Qwen3-Next 80B A3B Instruct: Hybrid Attention, 3B Active

Most mixture-of-experts releases change one number: more experts, or more of them active. Qwen3-Next changed the attention layers too. Three out of every four layers in Qwen3-Next-80B-A3B use a linear-attention variant called Gated DeltaNet rather than standard attention, and the expert layout is unusually sparse.

The result is an 80B-parameter model that activates about 3B per token. The Instruct edition is the non-thinking one: it answers directly and never emits a reasoning block. Qwen released the pair in September 2025 under Apache-2.0, and a year on it is still one of the more interesting efficiency designs in the open-weight field.

What is actually inside

The Hugging Face model card is unusually specific. Qwen3-Next-80B-A3B has 48 layers arranged as twelve repeats of a four-layer block: three Gated DeltaNet layers followed by one gated standard-attention layer, each feeding a mixture-of-experts layer. The MoE has 512 experts, with 10 routed and 1 shared active per token. Alibaba Cloud's write-up puts that at roughly 3.7% of parameters active.

The same write-up gives the reasoning behind the mix. Linear attention is cheap but weak at exact recall; standard attention recalls well but its cost grows with context length. Keeping one full-attention layer in four is the compromise, and Qwen reports that Gated DeltaNet beat sliding-window attention and Mamba2 in its own in-context learning experiments. If the total-versus-active distinction is new to you, active vs total parameters covers why 3B active is the number that sets your compute bill.

There is also a native multi-token prediction module, intended for speculative decoding. It needs SGLang or vLLM; plain Hugging Face Transformers does not use it.

The efficiency claims, and which ones matter

Qwen makes two claims about the base model. The first is training cost: under 10% of the GPU hours of Qwen3-32B, with 9.3% quoted in the pre-training section, on 15T tokens. That is interesting to Qwen and mostly irrelevant to you.

The second is inference throughput against Qwen3-32B: nearly 7x faster prefill and nearly 4x faster decode at 4K context, and more than 10x on both beyond 32K. That one does matter, because it grows with context length. A model whose advantage widens as prompts get longer is a natural fit for retrieval-heavy work, long chat histories and log analysis. These are vendor figures on vendor hardware, so treat them as a direction rather than a promise for your own stack.

Context is 262,144 tokens natively. The card documents extending it to about 1M with YaRN scaling, with the caveat that static YaRN can hurt short-prompt quality, so enable it only for deployments that genuinely need the longer window.

How close it gets to the 235B

The useful comparison is Qwen's own Qwen3-235B-A22B-Instruct-2507, which activates seven times as many parameters. On the vendor's table the 80B model lands close on most rows:

• MMLU-Pro: 80.6 against 83.0
• GPQA: 72.9 against 77.5
• AIME25: 69.5 against 70.3
• LiveCodeBench v6: 56.6 against 51.8, a win for the smaller model
• Arena-Hard v2 (GPT-4.1 judged): 82.7 against 79.2
• RULER long-context average to 1M: 91.8 against 92.5

Against Qwen3-32B in non-thinking mode it is ahead on nearly every row, often by a wide margin. All of these are Qwen-run numbers, which is the usual reason to read them as a shortlist rather than a verdict.

Where it falls short: multi-turn agent work

The same table has a weaker section, and it is the one that matters if you are building agents. On the tau-bench customer-service suites the gap to the 235B opens up: TAU2-Retail 57.3 against 74.6, and TAU2-Telecom 13.2 against 32.5. Aider-Polyglot, a multi-language code-editing benchmark, is 49.8 against 57.3.

The pattern is that single-turn knowledge and generation hold up, while long, stateful, tool-driven conversations do not hold up as well. It still supports tool calling, JSON mode and structured outputs, but for an unattended agent loop the larger model, or a dedicated agentic one, is the safer starting point. Model routing and fallbacks covers the usual compromise: run the cheap model first and escalate the tasks it fails.

Running it yourself

All 80B parameters must sit in memory even though only 3B do work per token. At 16-bit precision that is roughly 160 GB of weights by simple arithmetic, before any KV cache. Qwen's serving examples use four-way tensor parallelism in both SGLang (0.5.2 or later) and vLLM (0.10.2 or later). The card suggests dropping the context to 32K if the server will not start, which is a sign of how much memory the full window wants.

The recommended sampling is temperature 0.7, top-p 0.8, top-k 20, with an optional presence penalty between 0 and 2 to curb repetition. Qwen suggests a 16,384-token output budget for most queries, which matches the maximum output that hosted listings show.

If you would rather not host it, OpenRouter listed it at $0.09 input and $1.10 output per million tokens at the time of writing. Output is the line that dominates, and as a non-thinking model it does not spend output tokens on reasoning you never see.

Instruct or Thinking

The two editions share an architecture and a context window, and differ in post-training. The Qwen3-Next 80B Thinking guide covers the other one in detail. A short rule:

• Pick Instruct for chat, drafting, extraction, summarisation over long inputs and anything latency-sensitive. It scores higher on Qwen's Arena-Hard v2 table and costs fewer output tokens per answer.
• Pick Thinking for competition-style maths, hard coding problems and multi-step analysis, where it scores well ahead (AIME25 87.8 against 69.5).

Before committing, run twenty of your own tasks through both and through the 235B, and compare cost per accepted answer rather than price per token. How to benchmark LLMs on your own work has a harness that makes that an afternoon rather than a project.