Hy-MT2-1.8B: A 440 MB Translation Model for On-Device Work

Most translation features still send every string to a cloud API. That is fine until you need offline support, a latency budget measured in tens of milliseconds, or a guarantee that user text never leaves the device. Hy-MT2-1.8B is aimed squarely at that gap.

It is the smallest of three translation models Tencent's Hy team open-sourced on 21 May 2026, alongside Hy-MT2-7B and Hy-MT2-30B-A3B. All three are Apache 2.0 and ungated on Hugging Face. This guide covers what the 1.8B does well, where its limits are, and when to stop squeezing it and use a bigger sibling.

A translator, not a small chat model

The first mistake people make with a model this size is to treat it as a general assistant that happens to translate. It is the other way round. Hy-MT2 is trained for one job: translating among 33 languages, per the technical report, while following translation instructions such as terminology, style and format constraints. The model card ships no default system prompt and expects you to use its published prompt templates.

That narrow scope is why a dense model this small can be useful at all. It does not need to know how to write code or plan a trip. It needs to map text between languages faithfully, keep placeholders intact and respect a glossary. Ask it to summarise or reason and you will get the quality you would expect from any small model.

The 440 MB figure, and what it buys you

The headline claim is about size. Using Tencent's AngelSlim toolkit at 1.25-bit quantisation, the report says the 1.8B model needs about 440 MB of storage and runs about 1.5 times faster. Tencent publishes that build as a GGUF for llama.cpp, alongside FP8, standard GGUF and 2-bit GGUF variants.

440 MB is small enough to bundle in a desktop application or download on first launch of a mobile app. That is the real proposition: translation that works on a train with no signal, with no per-request cost and no data leaving the device.

The catch is that 1.25 bits per weight is an extreme setting, and Tencent's headline figures are about storage and speed, not quality loss per language. If the cost is uneven, rare languages, long sentences and exact terminology are the places to look first, because they are where a translation feature gets judged. Quantization explained covers the mechanics. The practical rule is simple: evaluate the exact quantised file you will ship, not the BF16 original, and compare the 2-bit and 1.25-bit builds on your own text before picking one.

What "beats commercial APIs" does and does not mean

Tencent's report says the 1.8B surpasses mainstream commercial translation APIs, naming Microsoft and Doubao, overall. That is a vendor claim on vendor-chosen test sets, and the model card shows the results as a chart rather than a table of numbers, so it is hard to check which language pairs carry the average.

Treat it as a signal that the model is worth testing, not as evidence it will beat your current provider on your pairs. An aggregate across 33 languages can hide a weak result on exactly the pair you care about. If your traffic is mostly English to Japanese, test English to Japanese.

Working inside an 8K window

Hosted listings for every Hy-MT2 size, including OpenRouter, show an 8,192-token context and 4,096 maximum output tokens, and the model card's recommended settings cap generation at 4,096 tokens. For UI strings and chat messages that is ample. For documents it means chunking.

Chunking translation is not free. Split mid-paragraph and you lose the referent for a pronoun; split between sections and terminology can drift. Three habits help:

• Split on paragraph or section boundaries, never on a token count alone.
• Pass a glossary with every chunk, using the model's terminology template, so terms stay consistent across chunks.
• Count tokens in the source language, not English. Many scripts tokenise into more tokens per word, so an 8K window holds less text than you expect; token costs by language explains why.

When to step up to the 7B or 30B-A3B

The 1.8B is the right pick when it has to run on the user's hardware, or when you are translating very high volumes of short strings where the 7B's extra cost would add up. Tencent's recommended sampling for the 1.8B and 7B is identical (temperature 0.7, top_p 0.6, top_k 20, repetition penalty 1.05), so swapping between them in a pipeline is a one-line change.

On a hosted API the price argument mostly disappears. OpenRouter lists the 1.8B at $0.044 input and $0.177 output per million tokens at the time of writing, against $0.074 and $0.295 for both the 7B and the 30B-A3B. Translation is cheap at all three prices. If you are calling an API anyway, start with the larger models and only drop to the 1.8B if the quality is indistinguishable on your test set.

Pick the 7B if you want a model that still fits on one consumer GPU with GGUF builds available but has more headroom on hard pairs. Pick the 30B-A3B if quality is the priority and you are calling it over an API or have a GPU with room for 30B of weights.

Before you ship it on-device

1. Build a test set of a few hundred real strings per language pair, including placeholders, markup and product terms.
2. Run it through the BF16 model, then the 2-bit and 1.25-bit builds, and have a fluent reviewer score a sample blind.
3. Check placeholder and tag preservation automatically; a dropped {count} is a bug, not a style issue.
4. Measure memory and latency on the lowest-spec device you support, not on a development laptop.
5. Keep a server-side fallback to a larger model for pairs where the small one is not good enough.

For the broader prompting side of machine translation, including glossaries and register, see LLM translation: placeholders, glossaries and register.