Llama 3.2 3B Instruct: The Smallest Llama That Can Call Tools

Small models tend to fail in the same place: they can write a passable sentence long before they can follow a schema. For Llama 3.2, the line between "writes text" and "does work" falls between the 1B and the 3B, and Meta's own numbers show it more clearly than most model cards do.

That makes the 3B the interesting one of the pair. It is still small enough to run on a recent phone, but it is the first size in the family where on-device agent behaviour, meaning a model that reads a request and calls the right function, becomes realistic rather than aspirational.

Specs in brief

Meta released Llama 3.2 on 25 September 2024. The 3B has 3.21 billion parameters and, like the 1B, was produced by pruning Llama 3.1 8B and distilling from the logits of Llama 3.1 8B and 70B. Pretraining used up to 9 trillion tokens; knowledge stops at December 2023.

Context is 128K tokens for the full-precision model and 8K for Meta's quantised on-device builds. Officially supported languages are the same eight as Llama 3.1: English, German, French, Italian, Portuguese, Hindi, Spanish and Thai. It is text only.

The jump from 1B is not linear

Compare Meta's BF16 instruct scores for the two small models. Most benchmarks improve by a respectable margin. Tool use improves by a different order:

• BFCL V2 — 1B 25.7, 3B 67.0
• Nexus (tool use) — 1B 13.5, 3B 34.3
• GSM8K (CoT) — 1B 44.4, 3B 77.7
• MGSM (multilingual maths, CoT) — 1B 24.5, 3B 58.2
• IFEval — 1B 59.5, 3B 77.4
• ARC-C — 1B 59.4, 3B 78.6

BFCL V2 more than doubles. Multilingual maths more than doubles. That is the profile of a capability that exists at 3B and mostly does not at 1B, rather than one that simply gets a bit better. For context, Meta reports 76.1 on the original BFCL for Llama 3.1 8B; the versions differ, so treat that as a rough guide rather than a like-for-like comparison.

The long-context numbers deserve a second look

One row in the model card is easy to skip and should not be. On InfiniteBench En.QA, a long-document question-answering test, the 3B scores 19.8 and the 1B scores 20.3. The larger model is no better. On the multiple-choice variant the 3B does pull ahead (63.3 against 38.0), and on multi-needle retrieval it reaches 84.7.

The reading is straightforward: the 3B can find things in a long input, but answering open questions that require synthesising across a long document is weak at this size regardless of the 128K window. A large advertised context and usable long-context reasoning are different properties. Context length vs effective context goes into why the two diverge, and it applies with particular force to small models.

In practice, keep the 3B on short inputs. If a task needs a long document, retrieve the relevant passages first and give it a few thousand tokens, not a hundred thousand.

What it costs to run on a device

Meta measured its quantised builds on a OnePlus 12 using ExecuTorch on the Arm CPU. Against the BF16 baseline of 7.6 tokens per second and 7,419 MB resident memory:

• SpinQuant — 19.7 tokens per second, 3,726 MB resident
• QLoRA — 18.5 tokens per second, memory down about 45%

Just under 4 GB of memory and around 19 tokens per second is workable on a flagship phone but not on a budget one, and it is noticeably heavier than the 1B. On a laptop or small server the picture is easier: a 3B model fits comfortably in memory on most modern machines, and self-hosting guidance on fitting models to hardware applies directly.

Through a hosted API it is cheap but not dramatically cheaper than an 8B, and the per-token prices listed for small models vary oddly between providers. If you have a network connection and no privacy constraint, price the 8B as well before assuming the smaller model saves money.

Choosing between 1B, 3B and 8B

The three models form a ladder, and the decision is mostly about where the model has to run.

• On a phone, single-step transforms (classify, extract, rewrite) — Llama 3.2 1B, for its lower memory and roughly 2.5 times the decode speed.
• On a phone, anything with tool calls, multi-step instructions or non-English users — the 3B.
• On a server or through an API — usually the 8B, which costs little more and is stronger across the board.

The 3B is the right answer when the device is the constraint and the task is more than a single transform. Outside that window, something else usually wins.

How to test it for an on-device agent

1. Write down the tools the assistant needs. Keep it to a handful with distinct names and clear descriptions; small models confuse overlapping tools.
2. Collect thirty real requests, including ambiguous ones and ones that need no tool at all.
3. Score three things separately: picked the right tool, produced valid arguments, and declined correctly when no tool applied.
4. Run the quantised build on the target phone and repeat. Quantisation can shift tool accuracy, and the hosted model is not what you ship.
5. Measure in the languages your users actually write in, since multilingual quality is where the 3B pulls furthest from the 1B.

Licensing follows the Llama 3.2 Community License: attribution, the 700 million monthly active user threshold, and the naming rule for derived models. The EU restriction in the Llama 3.2 acceptable use policy covers the multimodal models, not this text-only one.