Nemotron 3 Super Guide: 120B Total, 12B Active, 1M That Holds

A 1M-token context window is easy to advertise and hard to use. Plenty of models accept that many tokens and then quietly stop paying attention to most of them. Nemotron 3 Super is interesting mainly because its published long-context numbers suggest it does not, and because it does so while activating only 12B parameters per token.

NVIDIA released it on 11 March 2026 as the second model in the Nemotron 3 generation, after Nemotron 3 Nano in December 2025 and before the larger Ultra. It sits in the middle of the family, and the middle is where most production agent work actually lands.

LatentMoE and multi-token prediction, briefly

Super has 120B total parameters and 12B active. Like the Nano, it interleaves Mamba-2 layers, which carry a fixed-size state rather than a growing key-value cache, with mixture-of-experts layers and a small number of attention layers placed for precise retrieval.

Two things are new. The first is what NVIDIA calls LatentMoE: tokens are projected into a smaller latent dimension before expert routing, which NVIDIA says lets the model use four experts for the inference cost of one. The second is a set of multi-token prediction layers with shared weights, which give the model native speculative decoding: it drafts several tokens ahead and verifies them in one pass. Speculative decoding explained covers why that speeds up output without changing it. NVIDIA also trained the model with NVFP4 quantisation for compute efficiency.

The card lists seven languages: English, French, German, Italian, Japanese, Spanish and Chinese. Pre-training used more than 25 trillion tokens.

The long-context result is the headline

On RULER, a retrieval-over-long-context benchmark, NVIDIA's card reports these scores for Super, Qwen3.5-122B-A10B and gpt-oss-120b:

• 256K — 96.30, 96.74, 52.30
• 512K — 95.67, 95.95, 46.70
• 1M — 91.75, 91.33, 22.30

Super and the Qwen model are level, and both hold up remarkably well to 1M. The gpt-oss-120b column shows what usually happens when a model is pushed past what it was built for. On AA-LCR, which tests reasoning across long inputs rather than retrieval, Super scores 58.31 against 66.90 for Qwen and 51.00 for gpt-oss-120b, so Qwen keeps an edge once the task needs thought as well as recall.

A practical note: the Hugging Face config defaults to 256K because of the memory cost, and serving the full 1M in vLLM needs an environment flag and a larger maximum model length. Filling 1M tokens is also expensive per request even when the model handles it well; long context degradation and the case for retrieval still apply.

Against Qwen3.5-122B and gpt-oss-120b

Outside long context, NVIDIA's own table is honest that Qwen3.5-122B-A10B leads on most rows:

• SWE-bench (OpenHands) — Super 60.47, Qwen 66.40, gpt-oss-120b 41.9
• Terminal Bench Core 2.0 — 31.00, 37.50, 18.70
• TauBench V2 average — 61.15, 74.53, 61.0
• GPQA without tools — 79.23, 86.60, 80.10
• HMMT February 2025 without tools — 93.67, 91.40, 90.00
• LiveCodeBench v5 — 81.19, 78.93, 88.00

So Super is clearly ahead of gpt-oss-120b on agentic coding and long context, behind Qwen on most agentic and knowledge measures, and competitive on maths. On Artificial Analysis's Intelligence Index it scores 26, two points above both gpt-oss-120b and its smaller sibling Nemotron 3.5 Lightning.

These are NVIDIA-run numbers across different harnesses, and vendor scaffolds tend to flatter. The ranking matters more than the decimals.

Controlling reasoning spend

Super gives you three levels of control, all through chat_template_kwargs:

1. Thinking off with enable_thinking: false, for extraction and formatting steps.
2. Low effort with low_effort: true alongside thinking, which the card says uses significantly fewer reasoning tokens than full thinking.
3. A hard budget with reasoning_budget, where the model closes its trace at the next newline before the budget, or abruptly at the budget plus 500 tokens.

For agent loops this matters more than the benchmark table. Reasoning tokens are output tokens, and output is the expensive side of the bill. The token cost of reasoning models shows how quickly unbounded thinking adds up across a long session. Start at low effort and raise it only on the steps where you have measured a gain.

What it takes to self-host

Super ships under the NVIDIA Nemotron Open Model License, which NVIDIA says permits commercial use; read it, because it is NVIDIA's own licence rather than MIT or Apache. The hardware needs depend on precision. The BF16 card lists eight H100 80GB GPUs as the minimum, or two B200 or B300 GPUs. The FP8 build needs two H100 80GB GPUs or fits on a single B200. An NVFP4 variant is also evaluated, with scores within about a point of BF16 on most rows.

That puts Super in a useful spot: big enough to be a capable planner, small enough that a two-GPU FP8 node is a realistic deployment. Through an API, OpenRouter lists it at $0.08 per million input tokens and $0.45 per million output at the time of writing, with a :free variant that is rate-limited and suited to evaluation rather than production load.

Super, Lightning or Ultra

• Pick Super as the planner or main worker in an agent system, especially when sessions run long and you need the context to stay reliable.
• Pick Nemotron 3.5 Lightning for the frequent, simple steps underneath it. It is roughly a quarter of the size, much faster, and only two points lower on the Intelligence Index.
• Look at Nemotron 3 Ultra when Super fails on the hardest tasks and you can afford a much larger deployment.
• Pick Qwen3.5-122B-A10B when raw agentic and coding accuracy matters more than staying in the NVIDIA stack, since NVIDIA's own table shows it ahead on most rows.

The fastest way to decide is to replay one long session from your own logs through Super and your current model, and compare where each one starts to lose track.