Nemotron 3 Nano Omni: One Small Model for Video, Audio and Screens

An agent that has to watch a screen recording, listen to a call and read a scanned contract usually ends up with three pipelines: a speech-to-text model, an OCR or vision model, and a language model to make sense of both. Each hand-off loses context and adds latency. Nemotron 3 Nano Omni puts all three inputs into one small model.

NVIDIA released it on 28 April 2026, with weights on Hugging Face and a paper on arXiv the day before. NVIDIA describes its job plainly: the eyes and ears of a multi-agent system, a perception sub-agent that turns pixels and sound into text a planning model can use.

What is inside

The language backbone is Nemotron 3 Nano 30B A3B, the hybrid Mamba-Transformer mixture-of-experts model, with about 31B total parameters and roughly 3B active per token. Two encoders feed it: C-RADIOv4-H for images and video frames, and NVIDIA's Parakeet speech encoder for audio. Output is text only.

The context window is 256K tokens. For thinking mode the card suggests capping the model length at about 210,000 tokens, which leaves room for the trace. Reasoning is on by default and toggled with enable_thinking, and the model also supports tool calling, JSON output and word-level timestamps when transcribing.

Input limits that shape the design

The model card sets concrete limits, and they decide how you build around it:

• Video — MP4 up to two minutes. At 1080p, sample up to 1 frame per second or 128 frames; at 720p, up to 2 frames per second or 256 frames.
• Audio — WAV or MP3 up to one hour, sampled at 8 kHz or higher.
• Images — JPEG or PNG.
• Language — English only.

The two-minute video cap is the one people hit first. A forty-minute meeting recording is not one request; it is twenty clips, each described separately, with the audio handled as one long track alongside. Design the chunking before you design the prompt, and keep timestamps on every chunk so the planner can stitch results back together.

English only is the other hard line. For multilingual audio or documents, this is the wrong model regardless of its scores.

What the benchmarks say

NVIDIA compares Omni with its previous vision-language model, Nemotron Nano VL V2. The biggest gaps are on agentic and document work:

• OSWorld, computer use — 47.4 against 11.1
• CharXiv Reasoning, charts — 63.6 against 41.3
• MMLongBench-Doc, long documents — 57.5 against 38
• OCRBenchV2 (English) — 67.04 against 54.8
• MathVista mini — 82.8 against 75.5

For audio and video, where the older model had no equivalent, NVIDIA reports Video MME at 72.2, Daily Omni at 74.52 and WorldSense at 55.4. It also claims the model tops six leaderboards for document intelligence and video and audio understanding, and up to nine times the throughput of other open omni models at the same interactivity. Those are vendor claims; the OSWorld jump is the one to verify first if you are building a computer-use agent, because it is the largest and the most directly useful.

Quantisation barely moves the scores

The card lists three precisions with their sizes: BF16 at 61.5 GB, FP8 at 32.8 GB and NVFP4 at 20.9 GB. In the quantised builds the vision and audio encoders stay in BF16, and only the language backbone is compressed.

The quality cost is small. Averaged over nine non-speech benchmarks in non-reasoning mode, NVIDIA reports 65.80 for BF16, 65.40 for FP8 and 65.43 for NVFP4. Speech recognition error rates are essentially unchanged across all three.

That changes where the model can run. NVIDIA's minimums are one H100 80GB for BF16, one L40S 48GB for FP8, and one RTX 5090 32GB for NVFP4, with DGX Spark and Jetson Thor also supported. A perception model that fits on a workstation card or an edge device can sit next to the camera or the call centre rather than in a distant data centre. Quantization explained covers what FP8 and 4-bit formats actually trade away.

The sub-agent pattern

Omni is not meant to be the brain of an agent. Its strengths are perception and description; the planning is better left to a stronger text model. A typical split looks like this:

1. The orchestrator receives a task that involves media: a screen recording of a bug, a support call, a batch of invoices.
2. It sends each chunk to Omni with a narrow instruction, such as transcribing with timestamps, listing UI elements and their state, or extracting fields as JSON.
3. Omni returns compact text. The orchestrator reasons over that text, never the raw media.

This keeps expensive tokens on the expensive model and keeps raw media out of the planner's context entirely. Subagents and delegation covers when this kind of split pays for itself.

For narrow perception jobs, try thinking off. The card's instruct-mode settings use temperature 0.2 with a 1,024-token output cap, and speech recognition runs in non-thinking mode by design. Thinking mode, with its suggested 16,384-token reasoning budget, is for questions that need inference across what the model sees and hears.

Licence and access

The weights ship under the NVIDIA Open Model Agreement, which the card says permits commercial use. NVIDIA also released datasets and training techniques alongside it. Runtimes include vLLM, TensorRT-LLM, llama.cpp and Ollama, with SGLang limited to BF16 for now.

On OpenRouter it is listed only as a :free endpoint at the time of writing. That is fine for trying it on your own media, but free endpoints are rate-limited and come with no throughput promise. For production, self-host or use one of the paid partner platforms NVIDIA lists.

The decision is straightforward. If your agents need to see and hear English-language media, and you want one model on modest hardware rather than three pipelines, Nano Omni is the obvious candidate to test. If you need other languages, image generation, or audio output, it does none of those.