Hermes 3 70B: What the Fine-Tune Changed in Llama 3.1 70B

A fine-tune is a trade, not an upgrade. Every behaviour you train in pushes on something that was already there, and the useful question about any community fine-tune is what it gave up to get what it gained. Hermes 3 70B is a good case study because Nous Research published the answer.

Its technical report puts Hermes 3 70B side by side with the stock Llama 3.1 70B Instruct it was built from. The gains and the losses are both real, and which one matters depends on what you are building.

What the fine-tune did

Hermes 3 is Nous Research's August 2024 family of Llama 3.1 fine-tunes at 8B, 70B and 405B. The aim, in Nous's words, was a model trained to follow system and instruction prompts exactly and adaptively, with stronger multi-turn conversation, roleplay, tool use and long-context coherence than Hermes 2.

The data was mostly synthetic. The report's breakdown is dominated by general instructions at around 61% of tokens, followed by domain-expert material, maths, roleplay, coding, tool use and agentic or retrieval tasks, and a small slice of steering and alignment examples. Training was supervised fine-tuning on a cluster of H100s; the report says a DPO stage gave negligible gains at 70B and 405B, so the released 70B is the SFT checkpoint.

It keeps the Hermes conventions: ChatML formatting, tool calls as JSON inside <tool_call> tags with signatures declared in the system prompt, and a JSON mode driven by a schema in the system prompt.

The scorecard against stock Llama 3.1 70B Instruct

From the technical report, Hermes 3 70B against Llama 3.1 70B Instruct:

• Clear gains: AGIEval 56.18 against 48.26, TruthfulQA 63.29 against 59.91, MuSR 50.67 against 47.08, HellaSwag 88.19 against 86.42.
• Clear losses: MATH level 5 20.80 against 29.24, IFEval 81.21 against 87.25, MMLU-Pro 47.24 against 52.94, MMLU 79.09 against 82.27, GPQA 37.67 against 40.09.
• Level: MT-Bench 8.99 against 8.93, with ARC and BoolQ within a couple of points.

The losses are concentrated in maths, hard knowledge questions and strict format compliance. The gains are on reasoning-style exams and truthfulness. Conversation quality, as MT-Bench scores it, is unchanged.

In practice that gives a clear rule. If your workload is maths, structured extraction with strict format checks, or hard factual questions, stock Llama 3.1 70B Instruct was the better model and fine-tuning did not help; the Llama 3.1 70B Instruct guide covers it. If your workload is persona-driven chat, creative writing or anything where the system prompt defines behaviour, Hermes is the reason to switch.

The long-context caveat

Hermes 3 lists a 131K context window, inherited from Llama 3.1. The technical report adds a detail worth knowing: training samples were packed to 8,192 tokens, matching the length Llama 3.1 was mostly trained at, even though the base supports far more.

That does not mean the model stops working past 8K. The long-context ability comes from the base model, and fine-tuning on shorter sequences generally preserves it. It does mean none of the Hermes-specific behaviour was trained on long inputs, so the claim of improved long-context coherence is one to verify rather than assume. If your prompts routinely run past 30K tokens, put a few long documents through it with questions whose answers sit in the middle before committing.

Running it yourself

Seventy billion dense parameters is the size where self-hosting stops being a cluster decision and becomes a hardware purchase. Nous publishes official GGUF and FP8 builds alongside the BF16 weights, and the model card shows loading in 4-bit through bitsandbytes and serving through vLLM.

Which precision fits depends on your GPUs and how much quality loss you will accept; which open model fits your GPU works through the memory arithmetic, and quantization explained covers what you lose at each step. The licence is Meta's Llama licence, inherited from the base, so read its terms before serving it commercially.

70B or 405B

The obvious alternative is the larger sibling. On OpenRouter at the time of writing, Hermes 3 70B lists at $0.70 per million tokens for input and output alike, and the 405B at $1. Both list a 131K window and 16K maximum output.

A 43% price difference for a model nearly six times the size is a smaller step than usual, and the report shows where the extra size goes. The 405B leads the 70B by about six points on MMLU, seven on GPQA and ten on MATH level 5. On MT-Bench the two are effectively tied, 8.99 for the 70B against 8.93 for the 405B.

So the rule is the same one that applies against stock Llama, one level up:

• Pick 70B for conversational, persona and creative work, where the scores say size adds little, and for anything you want to self-host.
• Pick 405B when knowledge and reasoning depth matter and you are on a hosted endpoint anyway. The Hermes 3 405B guide covers its quirks, including its sensitivity to an empty system prompt.

Either way, this is a 2024 model with a knowledge cutoff OpenRouter puts at the end of 2023. For new projects, test it against a current open model on twenty of your own prompts before choosing it on reputation.