Gemma 3 12B: How Much of the 27B You Keep at Under Half the Size

Most people pick a model size by instinct: the biggest one that fits. That is a reasonable default when you know nothing else, but Google published enough data on the Gemma 3 family to do better, and the middle size is where the data is most interesting.

Gemma 3 12B is less than half the parameters of the 27B. On several of Google's instruction-tuned benchmarks it gives up only a few points, and on one it comes out ahead. Knowing which rows those are tells you which jobs you can move down a size.

Where the 12B sits in the family

Gemma 3 was released on 12 March 2025 in 1B, 4B, 12B and 27B sizes. The 12B shares the architecture of its larger sibling: a 128K-token context built from interleaved local and global attention layers, a SigLIP vision encoder that reads images as 256 tokens each, and multilingual training across more than 140 languages. Google trained it on 12 trillion tokens, against 14 trillion for the 27B and 4 trillion for the 4B.

The model card gives a total output length of 8,192 tokens. Like the rest of Gemma 3, it ships under Google's Gemma Terms of Use. Google's announcement lists function calling and structured output among the family's capabilities, and OpenRouter's catalogue lists tool calling for the 12B.

Row by row against the 27B

From the instruction-tuned table in Google's Gemma 3 technical report:

• MMLU-Pro — 60.6 for the 12B, 67.5 for the 27B
• LiveCodeBench — 24.6 against 29.7
• GPQA Diamond — 40.9 against 42.4
• MATH — 83.8 against 89.0
• Bird-SQL — 47.9 against 54.4
• MMMU — 59.6 against 64.9
• Global MMLU-Lite — 69.5 against 75.1
• FACTS Grounding — 75.8 against 74.9

Two patterns stand out. Broad knowledge and multilingual tasks show the clearest gap — five to seven points — which is what you would expect, since raw knowledge scales with parameters. Grounding, which measures whether an answer sticks to a supplied document, shows no gap at all; the 12B is marginally ahead.

That second result is the useful one. Summarising a supplied document, answering from retrieved passages and extracting fields from text all depend more on staying faithful to the input than on knowing things. Those are exactly the jobs where the smaller model is likely to be good enough.

For context, the 12B also beats Gemma 2 27B on every row above that has a Gemma 2 figure. The exception in the full table is SimpleQA, a short factual-recall test, where the older 27B scores 9.2 to the 12B's 6.3. A team still on Gemma 2 can often move down in size and up in quality at once.

The laptop GPU claim, checked

When Google released quantisation-aware trained builds in April 2025, it said the int4 12B targets laptop GPUs such as the 8 GB RTX 4060 Laptop. The report's memory table supports that for the weights — 24 GB in bfloat16, 6.6 GB at int4 — and complicates it for the workload: with the KV cache for a 32K context, int4 needs 21.5 GB.

So on an 8 GB card you are running short contexts, a few thousand tokens. That is still useful for local chat, code explanation and short-document work, but it is not the 128K model the spec sheet describes. On a 24 GB card the picture is much better, and the 12B leaves headroom that the 27B does not. Context length vs effective context covers why the usable number is usually well below the advertised one.

The price ratio on hosted endpoints

OpenRouter listed the 12B at $0.05 per million input tokens and $0.15 per million output at the time of writing, against $0.08 and $0.45 for the 27B. Output — usually the larger share of the bill for generation-heavy work — is a third of the price.

At these rates the absolute numbers are small, and for low-volume work the choice should be made on quality alone. At millions of requests a day, a three-times difference on output is a budget line, and the grounding result above suggests a lot of that traffic does not need the larger model.

Jobs it suits

• Retrieval-augmented answering — where faithfulness to supplied passages matters more than recall.
• Structured extraction — pulling fields from invoices, emails or tickets into a schema. LLM data extraction covers designing the schema first.
• Image description and simple document reading — with the caveat that 896-pixel inputs lose small text.
• Local development — a model that runs on a developer laptop for prototyping before moving to a hosted endpoint.

It is a weaker fit for code generation, where its LiveCodeBench score is low by current standards, and for questions that depend on broad factual recall without supplied context.

Picking between 4B, 12B and 27B

A simple rule: start with the 12B and move only when your own evaluation tells you to.

• Move up to Gemma 3 27B if failures cluster on knowledge, multilingual or maths-heavy prompts.
• Move down to Gemma 3 4B if the task is classification or short extraction and latency or device memory is the constraint.
• Look at Gemma 4 instead if you need agentic tool use or a permissive Apache licence; Gemma 3 predates both.

Run the same 30 to 50 real prompts through each size and record the failure type, not just the pass rate. Model routing and fallbacks covers sending easy traffic to the small model and escalating the rest, which is often where this comparison ends up.