Gemma 2 27B in 2026: When an 8K-Context Model Still Makes Sense

Gemma 2 27B is still listed on model catalogues, still answers requests, and still turns up in tutorials written in 2024. The question is not whether it works — it does — but whether there is any reason to choose it over the models Google has shipped since.

For new work the honest answer is almost always no. For existing work the answer is more interesting, because a model you have already tuned prompts, evaluations or fine-tunes around has a switching cost that a spec sheet does not show.

What Gemma 2 27B is

Google released Gemma 2 on 27 June 2024 in 9B and 27B sizes, each as a base and an instruction-tuned model. The 27B is a dense decoder with 46 layers, trained from scratch on about 13 trillion tokens of primarily English web text, code and mathematics. The smaller 9B was trained with knowledge distillation; the 27B was not.

The architecture has a few details worth knowing if you self-host it:

• Alternating attention — local sliding-window layers with a 4,096-token window alternate with global layers that span the full 8,192-token context.
• Logit soft-capping — attention logits are capped at 50 and final logits at 30. At release this was incompatible with Flash Attention and SDPA, and Hugging Face recommended eager attention for fine-tuning.
• Precision — Hugging Face notes that the 27B instruction-tuned model gives erratic output in float16, so run it in bfloat16.
• Grouped-query attention — 32 query heads sharing 16 key-value heads, which trims KV-cache memory.

Input and output are text only. The weights are released under Google's Gemma Terms of Use rather than a standard open-source licence — permissive enough for commercial use, but a custom document with its own use policy. Open weights vs open source covers why that distinction matters.

How good it was

The technical report gives the pre-trained 27B an MMLU score of 75.2, against 79.2 for Llama 3 70B, and the instruction-tuned model 76.2. On the LMSYS Chatbot Arena it scored an Elo of 1218, just ahead of Llama 3 70B at 1206 — a notable result for a model two and a half times smaller.

That was the state of the art for its size in mid-2024. It is not the state of the art now, and the gap is large. Google's Gemma 3 report puts Gemma 3 27B ahead on every instruction-tuned row it shares with Gemma 2 27B: MMLU-Pro rises from 56.9 to 67.5, LiveCodeBench from 20.4 to 29.7, and MATH from 55.6 to 89.0. The same report claims Gemma 3 4B is competitive with Gemma 2 27B.

The 8K wall

The single biggest practical limitation is the context window: 8,192 tokens, shared between prompt and answer. OpenRouter listed the maximum output at 2,048 tokens at the time of writing.

Eight thousand tokens is roughly a long source file, or a few pages of a contract. It rules out whole-document summarisation, retrieval with many chunks, multi-turn agent transcripts, and almost any coding task that needs more than one file in view. Every Gemma 3 size from 4B upwards offers 128K, and Gemma 4's larger models go to 256K. Context windows explained covers what that difference buys in practice.

If your prompts genuinely fit in 8K — short classification, rewriting a paragraph, answering from a single retrieved passage — the window is not a problem. Measure your actual prompt sizes before deciding.

Why it now costs more than its successor

This is the part that surprises people. OpenRouter listed Gemma 2 27B at $0.65 per million tokens for both input and output at the time of writing. Gemma 3 27B was listed at $0.08 input and $0.45 output, and Gemma 3 12B at $0.05 and $0.15.

So the older, weaker, shorter-context model is the more expensive one on hosted endpoints. That inversion is common with legacy models: fewer providers keep them warm, there is less competition on price, and serving them is no longer anyone's priority. What a model upgrade costs covers the opposite case, where the new model is pricier; here the upgrade is cheaper on every axis except your own time.

Reasons you might still run it

There are a few legitimate ones:

• An existing fine-tune. If you trained adapters on Gemma 2 27B and they perform, retraining on Gemma 3 is real work and needs a fresh evaluation.
• A frozen evaluation baseline. Research and regression suites sometimes need the same model they reported on last year.
• Self-hosted and already paid for. If it runs on hardware you own and the prompts fit, the hosted price inversion does not apply to you. Hugging Face put it at about 56 GB in bfloat16 and about 18 GB at 4-bit.

None of those is a reason to start a new project on it.

Migrating to Gemma 3 without surprises

The natural replacement is Gemma 3 27B if you want the same size class, or Gemma 3 12B if cost matters more than the last few points of quality. A short checklist:

1. Collect 30 to 50 real prompts with the outputs you currently accept.
2. Run them through the candidate with the same system prompt and sampling settings. If you build prompts by hand, render them with the new model's own chat template from its tokenizer rather than reusing a string you wrote for Gemma 2.
3. Compare outputs side by side, paying attention to length and tone. Newer models are often more verbose, which affects both cost and downstream parsing.
4. Only then start using the larger window. A migration that changes the model and the prompt shape at once makes regressions impossible to attribute.

Most teams find the move is a configuration change plus an afternoon of checking. The rare exception is a heavily tuned fine-tune, and even then the newer base tends to be worth retraining on.