Nemotron 3.5 Lightning: A Fast Small Model for Agent Grunt Work

Most of the calls in a busy agent system are not hard. Reformat this output, check this tool result, extract these fields, decide which sub-task comes next. Sending every one of them to a frontier model is how agent bills get out of hand, and sending them to a weak model is how agents fall over. Nemotron 3.5 Lightning is NVIDIA's attempt at the model in between.

NVIDIA released it on 11 August 2026 as the first Nemotron 3.5 model and the direct successor to Nemotron 3 Nano 30B A3B. It keeps the same size and the same hybrid architecture. What changed is the post-training, and the results show up almost entirely in agentic work.

What changed from Nemotron 3 Nano

Lightning is a mixture-of-experts model with about 30B total parameters and roughly 3B active per token. NVIDIA's card says 30B and 3B; Artificial Analysis counts 31.6B and 3.6B. NVIDIA's Nano report used the same 31.6B figure and gave 3.6B active only when embeddings are included, so the gap is mostly a matter of counting. The layer stack mixes Mamba-2, MoE and attention, adds multi-token prediction layers, and supports up to a 1M-token context. Pre-training used an NVFP4 recipe on more than 20 trillion tokens.

On Artificial Analysis's Intelligence Index it scores 24, up from 15 for Nemotron 3 Nano. The bigger jumps are on agent benchmarks. Artificial Analysis measured Terminal-Bench 2.1 at 24 percent against 7 percent for the Nano, and an Elo of 824 on GDPval-AA v2, a gain of 334. NVIDIA's own harness tells the same story: SWE-bench Verified goes from 34.08 to 51.56 and BrowseComp from 13.74 to 36.97.

Output token usage per task stayed similar to the Nano, according to Artificial Analysis, so the gains did not come from simply thinking longer.

Speed is the product

The number that defines Lightning is throughput. Artificial Analysis measured a median of nearly 670 output tokens per second on a pre-release DeepInfra endpoint serving the final NVFP4 weights, and about half a minute per Intelligence Index task. In the same article, Qwen3.6 35B A3B took about 3.5 minutes per task and gpt-oss-120b about 3.4.

For an agent, wall-clock time compounds the same way cost does. A sub-agent that answers in a second instead of seven lets an orchestrator fan out, retry and verify within a latency budget that would otherwise allow one attempt. That is the job NVIDIA describes for it: the execution layer of multi-agent systems, where calls are frequent, short and checkable.

Where it still trails

NVIDIA's model card is unusually candid here. Its comparison table includes Qwen 3.6 35B A3B, Gemma 4 26B A4B, Nemotron 3 Super and GPT-OSS 20B, and Lightning leads none of the rows. A few gaps that matter:

• SWE-bench Verified — 51.56 against 70.12 for Qwen 3.6 35B A3B and 63.08 for Nemotron 3 Super.
• Terminal-Bench 2.1 — 24.58 against 44.38 for the Qwen model.
• HLE, text only, no tools — 11.72 against 19.56.
• AA-Omniscience — 17.50, lower than Nemotron 3 Nano's 20.15, so world knowledge did not improve with the agentic gains.

The fair reading is that Lightning trades some capability for a large speed advantage. On the Intelligence Index it matches gpt-oss-120b at 24 and sits two points below Nemotron 3 Super's 26, from a model about a quarter of Super's size. If the hard step in your pipeline is real software engineering, a stronger model should own it.

An executor, not a planner

The pattern that fits Lightning is an orchestrator-worker split. A stronger model plans, breaks the task down and reviews the result. Lightning does the many cheap steps: tool calls with clear schemas, extraction, classification, summarising tool output before it goes back into the planner's context, and first-pass checks. Orchestrator-worker patterns covers the structure, and model routing and fallbacks covers escalating to the bigger model when the small one fails.

Two settings from the card are worth copying. For coding agents, NVIDIA suggests force_nonempty_content: true in the chat template arguments, so a turn never comes back as reasoning with no answer. And thinking is on by default; turn it off with enable_thinking: false for formatting and extraction steps where a trace adds tokens without adding accuracy.

Licence and deployment

Lightning ships under OpenMDW-1.1, which Artificial Analysis describes as permitting commercial use without material restrictions. That is a change from the NVIDIA Nemotron Open Model License that covers Nemotron 3 Nano and Super, and it makes Lightning the easiest model in the family to build a product on.

NVIDIA publishes BF16 reference weights, aimed at fine-tuning, distillation and research, and a separate NVFP4 build that it recommends for production. A single H100 or A100 80GB runs the BF16 weights at a 256K context; the full 1M window needs a larger setup such as a B200, GB200 or eight H100s. vLLM and SGLang both serve it, with the qwen3_coder tool-call parser.

Through an API, OpenRouter lists it at $0.07 per million input tokens and $0.20 per million output, with cached input at $0.04, at the time of writing. A :free variant is also listed; it is rate-limited and good for trying the model, not for an agent fleet.

Lightning or something else

• Over Nemotron 3 Nano — almost always. Same footprint, better agent results, a more permissive licence. The Nano only makes sense if you have a fine-tune built on it.
• Over Nemotron 3 Super — when the step is frequent and simple enough that speed matters more than the last few points of accuracy. See the Nemotron 3 Super guide for the bigger sibling.
• Over Qwen 3.6 35B A3B — when latency is the constraint. When accuracy is, NVIDIA's own table points to the Qwen model.

Measure it the way you will use it: run your actual sub-agent prompts through Lightning and your current model, and compare completed tasks per minute and per dollar, not single-shot accuracy.