Gemma 3 4B: What a Small Multimodal Model Is Actually Good For

Small models fail in a specific way. They rarely produce gibberish; they produce fluent, confident, plausible answers that happen to be wrong. If you judge a 4B model by chatting with it, it feels far more capable than it is, and that gap is where production problems come from.

Gemma 3 4B is a good small model. Google's own numbers say so, and they also say precisely where it should not be trusted. Read both halves before deploying it.

The smallest Gemma 3 that can see

Google released Gemma 3 on 12 March 2025 in 1B, 4B, 12B and 27B sizes. The 4B is the smallest that accepts images: the 1B is text-only with a 32K window, while the 4B gets the full 128K-token context and the same SigLIP vision encoder as its larger siblings, encoding each image as 256 tokens at 896 by 896 pixels.

It was trained on 4 trillion tokens, against 12 trillion for the 12B and 14 trillion for the 27B, and the model card gives a total output length of 8,192 tokens. The licence is the Gemma Terms of Use. Google's technical report makes a striking claim for it: that the instruction-tuned 4B is competitive with the previous generation's Gemma 2 27B, a model nearly seven times its size.

The number to read first: SimpleQA

The report's instruction-tuned table for the 4B includes MMLU-Pro 43.6, LiveCodeBench 12.6, GPQA Diamond 30.8, MATH 75.6, MMMU 48.8 and FACTS Grounding 70.1. Most of those are respectable for the size; MATH in particular is far above Gemma 2 27B's 55.6.

Then there is SimpleQA, a test of short factual questions answered from memory: 4.0. The 27B manages 10.0, which is also low, but the point stands. A 4B model has very little room to store facts, and when asked for one it does not know, it will usually invent something rather than decline.

Contrast that with FACTS Grounding at 70.1, which measures whether answers stay faithful to a document supplied in the prompt. The model is reasonably good at using information you give it and poor at recalling information you do not. That one distinction decides most deployment questions. Small vs large models covers why parameter count tracks stored knowledge far more closely than it tracks reasoning.

Where 4B is enough

• Classification and routing. Sorting tickets, tagging content, deciding which pipeline a request belongs in. The answer comes from a fixed label set, not from memory. LLM classification covers getting the taxonomy right first.
• Extraction from supplied text. Names, dates, amounts and fields pulled from a document into JSON.
• Image captioning and simple visual questions. Describing a photo, reading a clear label, checking whether a screenshot shows an error dialog.
• Safety filtering. Google built ShieldGemma 2, its image-safety classifier, on the Gemma 3 4B foundation — a reasonable signal of what the size is good at.

Where it is not enough: open-ended factual questions, code generation beyond small snippets, and anything where a confident wrong answer is costly. For those, move up to Gemma 3 12B or larger, or ground every answer in retrieved text.

Running it on a phone, a laptop or a CPU

The appeal of a 4B model is that it runs almost anywhere. The report puts the weights at 8 GB in bfloat16 and 2.6 GB at int4; Google's April 2025 quantisation-aware training release named phones among the targets for the 4B and 1B, with official builds for Ollama, LM Studio, MLX, llama.cpp and gemma.cpp.

As with every Gemma 3 size, context is the hidden cost. With a 32K-token KV cache, int4 rises from 2.6 GB to 7.3 GB. On a phone or a small laptop you will be running short prompts, and a single image already spends 256 tokens. That suits the jobs above, which are mostly short anyway.

Image documents need care at this size. The report shows DocVQA rising from 72.8 to 81.0 for the pre-trained 4B when Pan and Scan tiling is used, so if you are reading forms or receipts, check that your runtime supports it.

Hosted 4B is a different calculation

OpenRouter listed Gemma 3 4B at $0.05 per million input tokens and $0.10 per million output at the time of writing. The 12B was listed at $0.05 and $0.15. At those prices the hosted saving from dropping to 4B is small — the same input price, and a third less on output.

So the case for the 4B is mostly about where it runs, not what it costs per token. If you are calling an API anyway, the 12B is usually worth the marginal difference. The 4B earns its place on-device, offline, in a browser, or anywhere latency and memory are the binding constraints. OpenRouter's catalogue also does not list tool calling for the 4B, while it does for the 12B and 27B; if you need tool use through a hosted endpoint, check the provider before choosing it.

Gemma 3 4B or Gemma 4 E4B?

In April 2026 Google released Gemma 4, including an E4B model aimed at the same on-device niche. Google describes it as running with an effective 4B-parameter footprint, with a 128K context, image and video input, native audio input, and an Apache 2.0 licence instead of the Gemma Terms of Use.

For a new on-device project, test E4B first; audio input and the licence change alone are reasons to. Keep Gemma 3 4B where it is already deployed and passing evaluation, or where your runtime does not yet support the newer architecture.

Either way, the evaluation that matters is a fixed set of your real inputs with known correct outputs. A small model that scores well there is a bargain. One you only judged by chatting with it is a liability waiting for a confident wrong answer.