Qwen3.8-Flash-Next

Qwen's open-weight multimodal MoE released 2026-08-26, and explicitly an early preview of the architecture behind Qwen4. Alibaba is doing again what it did with Qwen3-Next before Qwen3.5: ship the next generation's structural changes as open weights first, a year ahead of the family that will be built on them.

Shape: a 125B-parameter main model plus 51B N-gram embedding parameters, with 6B active per token. Native context 262,144 tokens, extensible to 1M with YaRN. The production variant is served as qwen3.8-flash on QwenCloud at $0.15 / $0.47 per 1M input/output tokens, with 1M context by default. Weights are on Hugging Face and ModelScope; a technical report ships in the GitHub repo.

Four changes, all aimed at cost

The post organises the release around attention, residual, embedding and optimization:

• Attention — GDN + QSA. Three of every four layers use Gated DeltaNet to compress history into a fixed-size state; the fourth is global attention, now Qwen Sparse Attention (QSA). QSA's indexer aggregates the sequence into micro-blocks and scores importance per block rather than per token, so the indexing itself gets cheaper as context grows, and each layer compresses independently instead of reusing indices across layers. Claimed 7.6× prefill / 4.9× decode attention-kernel speedups at 1M tokens, and 8.6× the prefill throughput of Qwen3.7-Plus at 1M context with a 90% prefix-cache hit rate.
• Residual — Gated Residual. The residual stream widens into four branches with a dynamic gate on reads and writes; the residual state can be held in FP8.
• Embedding — N-gram Embedding. A lookup keyed on the current token plus a few preceding ones, adding 51B parameters that are addressed deterministically, so they can sit in host memory and be prefetched asynchronously rather than occupying GPU memory.
• Optimization — Muon. Muon for the genuinely 2-D weight matrices, AdamW for embeddings, router and the low-rank gate parameters, with fused matrices split before orthogonalization.

Also disclosed: ultra-sparse MoE (a large expert pool, few routed experts per token, one shared expert) and multi-token prediction trained over several steps to raise speculative-decoding acceptance. The mechanism side of all this belongs cross-wiki to ../llm-inference-wiki (attention-mechanism, speculative-decoding, kv-cache); what matters here is the price it buys.

Where it lands on the market map

Every number below is Qwen's own, and the comparison set is chosen by Qwen.

• Against Claude-Opus-4.6 (Max): ahead on SWE-bench Pro 62.5 vs 53.4, SWE-bench Multilingual 81.0 vs 77.5, IFBench 81.3 vs 62.5, LiveCodeBench v6 91.9 vs 88.8; level on GPQA Diamond (91.7 vs 91.3); behind on HLE, 35.9 vs 40.0.
• Against its own family: it beats Qwen3.7-Plus (397B total / 17B active) on most rows while costing, Qwen says, about 1/9 as much to train.
• Multimodal rows are strong on agentic surfaces — AndroidWorld 84.5 vs Opus-4.6's 62.0, ERQA 72.3 vs 40.8, RealWorldQA 88.5 vs 73.9 — and thin where they matter least to a buyer.

What matters here is the 6B activated parameters producing those rows, not where the rows land. Sparse MoE already let total and active params diverge; N-gram embedding pushes another 51B of capacity off the per-token matrix-multiply budget entirely and into addressable memory. Capacity, compute and footprint are becoming three separate numbers, and a rate card of $0.15/$0.47 is what that separation looks like from the buyer's side — budget-tier pricing on a model Qwen benchmarks against the closed frontier (llm-api-pricing).

What the post does not say

Both items below were checked on 2026-08-30, two days after this page was written, and both had gone false. The originals are struck rather than deleted — the point of the caution was that these were inferences waiting on evidence, and the evidence arrived fast.

• No licence is stated anywhere in the release post. Qwen's recent open-weight releases have been Apache-2.0 (qwen), but that is an inference from the family, not a fact about this checkpoint — check the Hugging Face repo before repeating it. Checked 2026-08-30, and the caution was right. The Hugging Face model card states qwen-community-1.0, not Apache-2.0. That licence requires a separate agreement with Qwen to run the weights commercially as a hosted inference service or inside an AI work-assistant product — a restriction Apache-2.0 and MIT do not carry. Anyone who had inferred the licence from the family would have been wrong about the one term that decides whether a company can deploy it. This contradicts where open-weight-models places Qwen and is flagged there.

• No independent evaluation exists yet. All capability and throughput figures are first-party, and two of the headline benchmarks (CoWorkBench, RecreationBench) are Qwen's own in-house suites with no external baseline. HLE is judged by GPT-4o. Superseded 2026-08-30 — four days after release, Artificial Analysis has it on the board:

  Measure	Qwen3.8-Flash-Next	Median (open-weight)
  Intelligence Index	56	29
  Output speed	83.3 tok/s	66.6
  Time to first token	2.74 s	2.34 s
  Price, input	$0.15 / M	$0.50
  Price, output	$0.47 / M	$2.20
  Output tokens used in evaluation	200M	110M

  Roughly twice the median intelligence score at under a third of the median price, which is the shape the first-party post claimed and an outside party now reproduces. Two qualifications stay on the record: the index is a composite, so it does not confirm any individual first-party benchmark row; and the model spends 200M output tokens against a 110M median to get there, so per-task cost is not simply the rate card. The llm-benchmarks caveat still applies to the in-house suites (CoWorkBench, RecreationBench), which nobody outside Alibaba has run. Hugging Face shows 121,976 downloads in the last month and 4.37k likes on the base checkpoint.

• The Qwen4 preview framing is a claim about a model nobody outside Alibaba has seen.

First practitioner contact is already in the corpus: qwen38-flash-next-willison has simon-willison running community quantizations of it on a desk-side box within a day of release.

Related

qwen · qwen38-flash-next-willison · open-weight-models · moe-architecture · llm-api-pricing · llm-benchmarks · artificial-analysis · deepseek · glm-52 · kimi-k3