Hermes 3 405B: The Steerable Llama Fine-Tune, Two Years On

Hermes 3 405B is not a stronger Llama 3.1 405B. On several standard benchmarks it is a weaker one. It is a differently behaved one, and the difference is deliberate: Nous Research trained it to follow the system prompt closely and adaptively rather than fall back on a fixed assistant persona.

That makes it a specialist's model in 2026. It is still on hosted endpoints and still a reasonable choice for some jobs, but only if you know what Nous traded away to get that behaviour, and how to prompt around it.

What Hermes 3 is

Nous released Hermes 3 in August 2024 in three sizes, fine-tuned from Llama 3.1 at 8B, 70B and 405B. The 405B model card describes it as the first full-parameter fine-tune of Llama 3.1 405B, with training compute from Lambda. The technical report, published on 15 August 2024, calls it a "neutrally-aligned generalist instruct and tool use model with strong reasoning and creative abilities".

The training data was mostly synthetic. The report's breakdown puts about 61% of tokens in general instructions, with smaller shares for domain expertise, maths, roleplay, coding, tool use and agentic or retrieval tasks, and a deliberate slice of steering and alignment data. The 405B model went through supervised fine-tuning only; the report says a DPO stage gave negligible gains at the larger sizes, so it was dropped.

On OpenRouter at the time of writing it lists a 131K context window, 16K maximum output and a knowledge cutoff at the end of 2023, priced at $1 per million tokens for both input and output.

The system prompt is not optional in practice

The model card says the system prompt is optional. The technical report is more candid. It notes that the 405B model is highly sensitive to the system prompt, and that with an empty one it does not necessarily adopt the helpful-assistant persona. The example in the report has it respond as though confused about who it is.

That is the flip side of the steerability Nous was aiming for. A model trained to become whatever the system prompt describes has less of a default to fall back on when the system prompt says nothing. The fix is simple: always send one, even if it is only a sentence stating the role, the audience and the output format. System prompts explained covers what to put in it.

The report also notes that the default assistant persona at 405B was not fully evaluated. If you deploy it behind a product, test the system prompt you will actually ship, not a generic one.

Where it beats stock Llama 3.1, and where it trails

The technical report compares Hermes 3 405B against Llama 3.1 405B Instruct. The pattern is mixed, and the losses are on the benchmarks many people care about most:

• Ahead: AGIEval 61.84 against 58.60, ARC-Challenge 69.45 against 66.04, GPQA 44.84 against 42.66, HellaSwag 90.19 against 88.34.
• Behind: IFEval 84.87 against 87.09, MATH level 5 30.85 against 35.98, MMLU 85.02 against 86.14, MMLU-Pro 54.14 against 63.51.
• Roughly level: MT-Bench 8.93 against 9.17, Winogrande, BoolQ and TruthfulQA within a point or so.

Two readings follow. The nine-point MMLU-Pro gap and the five-point MATH gap say that if your workload is hard reasoning or maths, stock Llama 3.1 405B Instruct is the better 2024 model. The IFEval gap is more surprising for a model sold on following instructions; IFEval tests verifiable formatting rules, which is a narrower thing than adopting a persona or a set of behavioural rules from a system prompt.

These are 2024 numbers on 2024 benchmarks, against a 2024 baseline. Every current open model beats both on most of them.

Tool calls and JSON in Hermes format

Hermes 3 uses ChatML and has its own tool-calling convention. Tool signatures go in the system prompt inside <tools> tags, and the model replies with a JSON object inside <tool_call> tags:

<|im_start|>assistant
<tool_call>
{"name": "get_weather", "arguments": {"city": "Phnom Penh"}}
</tool_call><|im_end|>

Your code runs the function and returns the result in a tool message wrapped in <tool_response> tags. There is also a JSON mode driven by a specific system prompt that supplies a schema. Nous publishes reference code for both in its Hermes function-calling repository.

On a hosted OpenAI-compatible endpoint, the provider translates between this format and the standard tools field, and that translation is where things break. Send a tool request and check you get a structured tool_calls array back, not tags in the content.

Running it yourself

The model card is blunt about hardware: the FP16 weights need more than 800 GB of VRAM. An FP8 build reduces that to around 430 GB, and the technical report's own evaluations were run in FP8. Either way it is a multi-GPU node, and for most teams a hosted endpoint is the only practical way to use it.

The licence is inherited from the base: the card tags it with Meta's Llama licence rather than an OSI-approved one. Read the terms before serving it commercially. Open-weight licences compared covers how Meta's terms differ from MIT or Apache.

Should you still use it

Nous has since released Hermes 4, also on Llama 3.1 405B, with optional reasoning in think tags and a much larger training corpus. For most new work that is the better Hermes; see the Hermes 4 405B guide. Hermes 3 still makes sense when you have prompts tuned for its behaviour and do not want to re-validate them, when you want a model that commits fully to a persona, or when you need a non-reasoning model with no thinking tokens in the output.

If the 405B price is the issue, the Hermes 3 70B guide covers the smaller sibling, which shares the format and the training approach. Whichever you pick, test with your own system prompt first; for this model family it changes the result more than the choice of size does.