Llama 3.1 8B Instruct: The Small Default That Will Not Retire

Llama 3.1 8B Instruct is more than two years old, and by most aggregate measures it has been passed many times over. It still turns up in a surprising number of production systems: as a classifier, as a cheap first pass in a routing chain, and above all as the base model under a large share of community fine-tunes.

The reason is not that it is good at everything. It is that it is small, predictable, permissively enough licensed for most teams, and supported by every inference stack that exists. This guide covers what the model actually is, what Meta measured, and the jobs where it remains a sensible choice in 2026.

What Meta shipped in July 2024

Meta released the Llama 3.1 family on 23 July 2024 in three sizes: 8B, 70B and 405B. All three share the same recipe. They were pretrained on more than 15 trillion tokens of public data, use grouped-query attention, and support a 128K context window. The knowledge cutoff is December 2023, so anything after that is unknown to the model unless you put it in the prompt.

Officially supported languages are English, German, French, Italian, Portuguese, Hindi, Spanish and Thai. The model is text in, text out. There is no vision variant at this size; Meta's vision models arrived later with Llama 3.2 at 11B and 90B.

On OpenRouter it is listed at $0.05 per million input tokens and $0.08 per million output tokens at the time of writing. That is close to the floor for a hosted instruction model, which is a large part of its continued appeal.

What the model card numbers actually say

Meta's own instruct benchmarks put the 8B next to its larger siblings, which is the most useful way to read them. The gap tells you which tasks shrink gracefully and which collapse.

• MMLU (5-shot) — 8B 69.4, 70B 83.6, 405B 87.3
• HumanEval (pass@1) — 8B 72.6, 70B 80.5, 405B 89.0
• GSM-8K (8-shot CoT) — 8B 84.5, 70B 95.1, 405B 96.8
• MATH (0-shot CoT) — 8B 51.9, 70B 68.0, 405B 73.8
• IFEval — 8B 80.4, 70B 87.5, 405B 88.6
• BFCL (function calling) — 8B 76.1, 70B 84.8, 405B 88.5

Instruction following and function calling hold up well: the 8B is within ten points of a model roughly fifty times its size. Multi-step maths is where it falls away hardest. That pattern matches what most people find in practice. The 8B is good at doing what it is told with a clear format, and unreliable when the task needs several chained reasoning steps with no room for error.

These are vendor-run scores from mid-2024, and several of these benchmarks are now widely suspected of contamination in newer training sets. Use them to compare the three Llama 3.1 sizes against each other, not to compare the 8B against a 2026 model.

Tool calling works, within limits

Llama 3.1 introduced a proper tool-use format. Meta's prompt documentation describes three built-in tools (brave_search, wolfram_alpha and a code interpreter), an ipython role for passing tool results back, and custom tools defined as JSON with a name, description and parameters. The model emits the call; your code executes it.

One detail matters for agent builders: the documented custom-tool format supports a single tool call per turn. If your framework expects parallel tool calls, you will either serialise them or need a provider whose chat template adds that behaviour. Through an OpenAI-compatible endpoint the provider handles the template, but it is worth testing that the tools parameter is actually honoured rather than silently ignored.

In practice the 8B handles one or two well-described tools reliably. Give it fifteen overlapping tools and long multi-turn state, and selection errors climb quickly. That is a size limit, not a formatting problem, and no prompt will fully fix it.

The licence is why it is everywhere

The weights ship under the Llama 3.1 Community License, a custom commercial licence rather than an open-source one. For most teams the terms are workable. Three clauses are worth knowing before you ship:

• Companies with more than 700 million monthly active users must request a separate licence from Meta.
• Distributing a product built on it requires displaying "Built with Llama".
• If you use Llama outputs to train or fine-tune a model you distribute, its name must begin with "Llama".

That third clause explains a lot of model names on Hugging Face. Meta explicitly allowed outputs to be used for synthetic data and distillation, which made the 8B a common starting point for fine-tunes. If you are weighing it against an MIT or Apache model, the licence comparison sets out what each permits.

When to pick it, and when to move up

The 8B is a reasonable choice for high-volume, well-bounded work: intent classification, extracting fields into a fixed JSON shape, rewriting or normalising short text, and first-pass triage before a stronger model. It is also a sensible base if you plan to fine-tune on a narrow task, because the tooling around it is mature. If you are deciding between tuning and prompting, the usual rule is to exhaust prompting and retrieval before paying for a training run.

Move down to Llama 3.2 3B only if the model has to run on a phone or a CPU and memory is the binding constraint. Move up to Llama 3.3 70B when the task needs multi-step reasoning, code beyond small functions, or reliable tool selection across many tools.

The cleanest pattern is often both: send everything to the 8B first, check the output with a cheap validator, and escalate failures. Model routing and fallbacks walks through how to set that up without doubling latency.

A quick way to decide

1. Take fifty real inputs from the task, including the awkward ones.
2. Run them through the 8B with a strict output schema and a validator.
3. Count validator failures and read twenty passing outputs by hand.
4. If the failure rate is low and the passes are correct, the 8B is your default. If not, measure the same set on a 70B before assuming you need a frontier model.

OpenRouter lists a single endpoint id for this model, with no dated snapshots. Meta has not revised the weights under the same name, so behaviour differences between hosts come from quantisation and chat templates, not from a different model. If two providers disagree on your eval, check which precision each serves.