Gemma 3 27B: The Open Multimodal Model Sized for One GPU

"Runs on a single GPU" is one of the most abused phrases in model marketing. It is usually true for the weights and false for the workload, because the memory a model needs grows with every token of context you give it.

Gemma 3 27B is a good model to learn this on. Google launched it in March 2025 as "the most capable model you can run on a single GPU or TPU host", and then published exactly the numbers you need to check that claim — which is more than most vendors do.

What changed from Gemma 2

Gemma 3 was released on 12 March 2025 in 1B, 4B, 12B and 27B sizes. Against Gemma 2 27B, the 27B changed in three ways that matter:

• Context went from 8K to 128K. Google achieved this by interleaving five local sliding-window layers for every global layer and raising the RoPE base frequency on the global layers, which keeps KV-cache growth manageable.
• It became multimodal. A SigLIP vision encoder of roughly 400M parameters lets it read images as well as text. The 1B model is text-only; the 4B, 12B and 27B all take images.
• It learned more languages. Google cites out-of-the-box support for over 35 languages and pre-trained support for over 140, from a training run of 14 trillion tokens.

The Hugging Face model card gives a total output length of 8,192 tokens. The licence is still the Gemma Terms of Use, not a standard open-source licence.

The hardware arithmetic

The technical report has a table every self-hoster should read. Weights alone, Gemma 3 27B needs 54 GB in bfloat16 and 14.1 GB with int4 quantisation. Add the KV cache for a 32,768-token context and those become 72.7 GB and 32.8 GB.

Read those two rows together. The int4 weights fit comfortably on a 24 GB consumer card — Google's April 2025 quantisation-aware training (QAT) release named the RTX 3090 specifically. But at a 32K context the total is already past 24 GB, and the full 128K window costs considerably more again. "Fits on one GPU" holds at modest context lengths, and stops holding well before the advertised maximum.

QAT is why the int4 build is usable at all. Rather than quantising a finished model, Google trained with quantisation in the loop, and reports that this cuts the perplexity drop at Q4_0 by 54% compared with naive quantisation. Official QAT builds are available for Ollama, LM Studio, MLX, llama.cpp and gemma.cpp. Quantization explained covers the mechanics, and KV cache explained covers why context, not weights, usually decides your hardware.

Vision at 256 tokens an image

Images are resized to 896 by 896 pixels and encoded as 256 tokens each. That is cheap on context but lossy for anything with small text: a dense screenshot or a scanned form loses detail when squeezed to that resolution.

Google's answer is Pan and Scan, an inference-time method that tiles an image into crops at closer to native resolution. The report's own table shows why it matters: on DocVQA the 27B rises from 85.6 to 90.4 with it, and on InfoVQA from 59.4 to 76.4. If you are doing document work, check whether your serving stack implements it; if not, you are getting the lower numbers.

What the benchmarks say

Google's instruction-tuned results for the 27B include MMLU-Pro 67.5, LiveCodeBench 29.7, GPQA Diamond 42.4, MATH 89.0, Bird-SQL 54.4 and MMMU 64.9. The report describes it as comparable to Gemini 1.5 Pro across benchmarks. On LMArena it posted a preliminary Elo of 1338 at launch, and Google's announcement claimed it beat Llama 3 405B, DeepSeek-V3 and o3-mini in those early human-preference votes.

The profile is strong on maths and multilingual knowledge and comparatively weak on code: a LiveCodeBench score below 30 is not a coding-agent model by 2026 standards. Treat it as a general assistant, document reader and multilingual workhorse rather than a programming tool.

On hosted endpoints it is cheap. OpenRouter listed it at $0.08 per million input tokens and $0.45 per million output at the time of writing, with cached reads at $0.04.

Has Gemma 4 made it obsolete?

For most new projects, yes. Google's Gemma 4 model card compares the newer Gemma 4 26B A4B directly with Gemma 3 27B: MMLU Pro 82.6 against 67.6, LiveCodeBench v6 77.1 against 29.1, and the agentic Tau2 benchmark 68.2 against 16.2. Gemma 4 also moved to the Apache 2.0 licence and doubled the context to 256K.

Gemma 3 27B still has a case in three situations. If you have a fine-tune or evaluation suite built on it, the migration is work. If you specifically want a dense model rather than a mixture-of-experts one — there is no expert routing to preserve when you fine-tune or quantise it — it remains a sound choice. And if your hardware budget is exactly one 24 GB card, the dense 27B at int4 is a known quantity; check what a Gemma 4 build needs at your context length before assuming it fits.

If you want something smaller from the same generation, Gemma 3 12B gives up surprisingly little on several benchmarks.

Testing it on your own hardware

1. Decide the longest context you actually need, then measure peak memory at that length, not at a short prompt.
2. Compare the QAT int4 build with bfloat16 on 30 of your real prompts. If you cannot tell them apart, run int4.
3. For images, test with and without Pan and Scan on your own documents.
4. Measure tokens per second under the concurrency you expect. A single user and ten users are different machines.

If the numbers work at your context length, it is a capable, cheap, well-documented model. If they only work at 4K, you have learned that before buying the GPU.