Phi-4 Guide: A 14B Reasoner That Knows Few Facts

The usual assumption about a small model is that it is a worse version of a big one: a bit less of everything. Phi-4 breaks that assumption in a useful way. Its strengths and weaknesses are lopsided, and once you see the shape you can predict quite accurately which jobs it will do well and which it will quietly get wrong.

Microsoft Research announced Phi-4 in December 2024 on Azure AI Foundry and published the weights on Hugging Face under the MIT licence in January 2025. It is a 14B dense decoder-only transformer with a 16K-token context window. Nearly two years on, it is still worth understanding, because the design choice behind it, heavy use of synthetic training data, explains a lot about how small models behave.

What 14B dense and synthetic data buy you

According to the model card, Phi-4 was trained on about 9.8 trillion tokens over 21 days on 1,920 H100 GPUs. The mix leans on synthetic textbook-style data, alongside filtered public web documents, academic books and question-and-answer sets. Only about 8% of the data is multilingual, and the public-data cut-off is June 2024.

Dense means every one of the 14 billion parameters runs for every token. There is no expert routing, so memory and compute are both set by that one number. That is the main operational appeal: at 14B, a quantised Phi-4 fits on a single consumer or workstation GPU, and serving it is unremarkable. Which open model fits your GPU works through the memory arithmetic if you want to size it.

The synthetic emphasis is the more interesting part. Training on carefully generated reasoning material teaches a model to work through problems step by step. It does not teach it the long tail of facts about the world, because those facts are not in textbook-style exercises.

The benchmark shape tells you the job

Microsoft's own table makes the lopsidedness visible:

• GPQA: 56.1, the highest figure in the comparison table
• MATH: 80.4, level with a 72B Qwen 2.5 model at 80.0
• MMLU: 84.8, against 88.1 for GPT-4o
• HumanEval: 82.6, against 90.6 for GPT-4o
• SimpleQA: 3.0, against 39.4 for GPT-4o

That last line is the one to remember. SimpleQA measures short factual recall, and 3.0 means Phi-4 almost never knows the answer to an obscure factual question unaided. Pair that with the reasoning scores and the profile is clear: good at working things out, poor at knowing things.

These are vendor numbers from late 2024, and the field has moved a long way since. Use them to understand the shape, not to rank Phi-4 against current models. Treat them as the vendor's own measurements, run on the vendor's own setup.

Where Phi-4 is the right call

The model card lists memory- or compute-constrained environments, latency-sensitive work and reasoning tasks as intended uses, primarily in English. In practice that maps to a few concrete jobs:

• Grounded reasoning over supplied text. Give it the facts in the prompt and ask it to calculate, compare or decide. Its weak recall stops mattering when nothing needs recalling.
• Structured extraction. Pull fields from short documents into JSON. OpenRouter lists JSON mode and structured outputs for it; structured outputs and JSON mode covers how to validate what comes back.
• Maths and logic tutoring or checking where every step can be verified.
• Private, on-premises deployments where MIT licensing and a single-GPU footprint make it easy to approve.

It is also cheap through hosted APIs. At the time of writing OpenRouter lists Phi-4 at $0.07 per million input tokens and $0.14 per million output, which puts it in the range where cost is rarely the deciding factor.

Where it will let you down

Three limits are hard rather than soft.

The 16K window. That is small by any current standard. A long conversation, a full source file plus instructions, or a retrieved bundle of documents will exhaust it quickly. You will need retrieval and aggressive trimming, which RAG vs long context covers.

Open-ended factual questions. Anything that relies on the model already knowing an answer is a hallucination risk. Do not deploy Phi-4 as a general knowledge assistant without grounding.

Non-English and non-Python work. Microsoft says plainly that it is not intended for multilingual use, and that its code training was mostly Python with common packages. Output in other languages or less common libraries needs checking by hand.

How to decide in an afternoon

The fastest test is to split your workload into two piles. In the first, every fact the model needs is in the prompt. In the second, the model has to supply knowledge of its own. Run twenty examples from each pile through Phi-4 and through whatever larger model you currently use.

If Phi-4 holds up on the first pile, you have found work you can move to a much cheaper, easily self-hosted model. If it collapses on the second pile, that is expected, and it tells you to keep that work on a larger model or add retrieval. When a cheap model is enough covers how to split traffic once you know where the line falls.