Hermes 4 405B Guide: Llama 3.1 With a Reasoning Switch

Hermes 4 405B is easy to file as one more large open model. It is more useful to think of it as two behaviours sharing one set of weights: a direct-answer assistant and a step-by-step reasoner, chosen by the prompt rather than by switching models. Which one you get decides almost every number Nous Research published for it.

The technical report went up on arXiv on 25 August 2025, and OpenRouter lists the model from the following day. It is a post-train of Meta's Llama 3.1 405B base, not a new pre-train, and by late 2026 it is an older model. The reasons to still look at it are specific: it is unusually steerable, it refuses far less than most assistants, and its dense architecture is about as well understood as anything in the open-weight field.

What Nous changed on top of Llama 3.1

Nothing in the architecture. The 405B and 70B Hermes 4 models start from the matching Llama 3.1 checkpoints, and a 14B sibling starts from Qwen3 14B. What changed is the post-training data and the way it was produced.

The model card cites about five million training samples, against roughly one million for Hermes 3. Most of it is newly synthesised. The report describes rejection-sampling against roughly a thousand task-specific verifiers using Atropos, Nous's open-source reinforcement-learning environment manager, so a sample is kept only when a checker accepts it. Reasoning samples average around five times the tokens of non-reasoning ones, with thinking traces up to 16 thousand tokens.

Post-training ran on 192 NVIDIA B200 GPUs at a 16,384-token sequence length. That matters for long inputs: OpenRouter lists a 131,072-token context, but the ability to use that much context comes from the Llama 3.1 base rather than from anything Hermes 4 was trained on. Test it on your own long documents before relying on it.

Two models in one checkpoint

Reasoning is switched on through the chat template, with a thinking=True flag or a supplied deep-thinking system prompt. The model then writes a trace inside <think> tags before answering. Through an API, this usually surfaces as a reasoning toggle on the request.

The gap between the two modes is not small. From the report's main results table, reasoning mode first and direct mode in brackets:

• MATH-500 — 96.2 (73.8)
• AIME 2024 — 81.9 (11.4)
• AIME 2025 — 78.1 (10.6)
• GPQA Diamond — 70.6 (39.4)
• LiveCodeBench v6 — 61.4 (28.1)
• MMLU-Pro — 80.6 (58.3)
• IFEval (loose) — 81.5 (84.9)

Two things follow. For maths, code and hard science questions, direct mode is a different and much weaker model, so any comparison that does not say which mode was used is close to meaningless. And instruction following goes slightly the other way: direct mode scores a little higher on IFEval, which fits the intuition that a long trace gives the model more room to wander from a strict format.

The obvious cost is output tokens. A reasoning answer can be many times longer than a direct one, and on a model priced per output token that difference is the bill. Reasoning models explained covers when the extra thinking pays for itself.

The refusal question

Nous built its own evaluation, RefusalBench, which measures how often a model declines requests that assistants commonly refuse, with a few categories such as self-harm and exploitation scored in reverse so that refusing them counts in the model's favour. Hermes 4 405B scores 57.1 in reasoning mode and 43.2 in direct mode. In the same chart, Grok 4 scores 51.3, DeepSeek V3 28.1 and Llama 3.1 405B, the base it was trained from, 21.7.

Read that carefully. It is a vendor-designed benchmark, and a high score is a statement about behaviour, not quality. For fiction, role-play, red-teaming research and frank discussion of sensitive topics, fewer refusals is exactly what you want. For a customer-facing product, it means the safety layer is your job: put a separate classifier in front of and behind the model rather than trusting the system prompt alone.

The report also notes that Hermes 4 is sensitive to framing: changing the assistant role token in the Llama 3 template to a first-person identifier noticeably changed its behaviour. Steerability cuts both ways, so pin your template and system prompt and version them.

What a dense 405B costs to run

Every one of the 405 billion parameters is used for every token. That is the opposite of the mixture-of-experts designs that dominate 2026, where a model with a similar or larger total activates only a few tens of billions per token. Dense vs mixture-of-experts explains why that difference drives serving cost and throughput.

In practice self-hosting means a multi-GPU node. Nous publishes BF16 weights and an FP8 build, and GGUF quantisations exist for local tools, but none of that makes a 405B dense model cheap. Through an API, OpenRouter lists it at $1.00 per million input tokens and $3.00 per million output at the time of writing, which is reasonable for its size but well above what many smaller open models cost per token. Whether those smaller models match it on your tasks is something to measure, not assume.

Tool calls and structured output

Hermes models use their own tool format: definitions go inside <tools> in the system prompt, and the model emits calls inside <tool_call> tags, which are added tokens in the vocabulary, so a streaming parser can catch them cleanly. vLLM supports this with its hermes parser. Calls can be interleaved with reasoning in a single assistant turn.

The card also says the model was trained to produce JSON that matches a supplied schema and to repair malformed objects. Treat that as helpful, not as a guarantee, and still validate every object. The same Hermes format underpins its predecessor, covered in the Hermes 3 405B guide, so migration between the two is mostly a prompt change.

When Hermes 4 405B is the right pick

• Pick it for creative writing, role-play and research where refusals get in the way, and where you want explicit control over when the model thinks.
• Pick it if you already run Llama 3.1 405B infrastructure and want a drop-in model with stronger reasoning on the same footprint.
• Look elsewhere when cost per task dominates. Newer mixture-of-experts models activate a fraction of the parameters per token, so benchmark one or two of them against it on the same tasks before paying for dense compute.
• Check the licence. The card lists Meta's Llama 3 licence, inherited from the base model, which carries Meta's conditions and is not a permissive licence like MIT.

Whatever you decide, run it in both modes on twenty of your own tasks before committing. How to benchmark LLMs on your own work has the setup, and the mode gap above is the first thing your results should confirm or contradict.