Command R7B: Cohere's Small RAG Model and Its Licence Catch

Small models are usually judged on the same general benchmarks as large ones, and they usually lose. That framing misses what Command R7B was built for. Cohere did not pitch it as a small all-rounder. It pitched it as a fast, cheap model for one specific stage of a retrieval pipeline: reading the documents you retrieved and writing a grounded answer from them.

If that is your job, R7B is still worth understanding. If you want a general assistant, a coding agent or long-form writing, it is the wrong tool, and this guide says so plainly further down.

What Cohere actually shipped

Command R7B arrived in December 2024, with the dated identifier command-r7b-12-2024. Cohere described it as the "smallest, fastest, and final model" in its R family, which is worth noting: there is no R7B successor to wait for. Later Cohere work moved to the Command A line.

The Hugging Face model card gives it 7 billion parameters (the file metadata rounds to 8B, a common discrepancy once embeddings are counted), a 128K-token context window and support for 23 languages, including English, French, Spanish, German, Japanese, Korean, Arabic, Chinese, Hindi and Hebrew.

The architecture has one detail worth knowing. Three of every four layers use sliding-window attention with a 4,096-token window and rotary position embeddings. Every fourth layer uses global attention with no positional embeddings, so it can attend across the whole sequence. The practical effect is a smaller KV cache on long inputs than a fully global 7B model would need, which is exactly what you want when the input is a stack of retrieved passages. KV cache explained covers why that memory dominates long-context serving.

Built for the answer step, not the whole pipeline

The model card is unusually specific about how R7B expects to be used:

• RAG. It was trained for the final generation step of retrieval-augmented generation. Documents go in through the chat template, and Cohere suggests short chunks of roughly 100 to 400 words rather than whole files.
• Tool use. Tools are described with JSON schema, and the template has slots for a tool plan, the calls and their results, so multi-step tool conversations have a defined shape.
• Two chat modes. A conversational mode that is chattier and asks follow-up questions, and an instruct mode that gives concise, task-focused answers.
• Code. Cohere highlights SQL and code translation, and recommends low temperature or greedy decoding for code tasks.

One default trips people up: there is no system preamble set out of the box, and the card recommends adding one. If your first test produces rambling, conversational output where you wanted terse extraction, check the preamble and the mode before blaming the model.

The licence is the first thing to check

The open weights are published under CC-BY-NC-4.0, plus Cohere Labs' acceptable use policy. NC means non-commercial. You can download the weights, study them, fine-tune them for research and run them on your own hardware for non-commercial work. You cannot put them behind a paid product without a separate arrangement.

Commercial use goes through Cohere's own platform or a hosted provider that has a deal with Cohere. That is a meaningful difference from the MIT-licensed open models covered in open-weight model licences compared. If your plan was "self-host a small model to avoid per-token costs in production", R7B does not fit that plan, however well it fits on the GPU.

Limits to plan around

Output length. OpenRouter lists a maximum output of 4,000 tokens. For a grounded answer with citations that is plenty. For generating a long report, a migration script or a full document, it is a hard ceiling, and you will see truncation rather than a graceful stop.

Age. The catalogue gives a knowledge cutoff of August 2024. In a RAG system that matters less than usual, because the facts arrive in the retrieved documents, but it still shapes the model's assumptions about libraries, APIs and recent events when a document leaves a gap.

Benchmarks. Cohere's launch materials compared R7B with other small open models of late 2024 on general leaderboards. Treat any such table with care: those are 2024 comparisons against 2024 models, and general benchmarks say little about grounded-answer quality, which is the thing you are buying it for. The test plan below measures that directly.

Price. It is listed on OpenRouter at $0.0375 per million input tokens and $0.15 per million output tokens at the time of writing. Prices move, so check before you budget. At that rate the cost of a RAG answer is dominated by how many retrieved chunks you stuff into the prompt, not by the model.

When to pick it, and when not to

R7B makes sense when all of these are true: the answers must be grounded in documents you supply, the questions arrive in several languages, volume is high enough that per-token cost matters, and you are calling it through a commercial API rather than self-hosting for profit.

It is the wrong pick in three common cases:

• Agentic coding. Cohere's own coding model is North Mini Code, a 2026 mixture-of-experts model with Apache 2.0 weights, a 256K window and a 64K output cap. Use that, not R7B, for anything that edits a repository.
• Harder reasoning over the documents. If answers require combining evidence across many passages, step up to the larger Cohere models, starting with Command A.
• Commercial self-hosting. The licence rules it out, as above.

There is only one variant in the catalogue, the dated 12-2024 snapshot. A dated identifier means the weights behind it should not change, which is a useful property for a pipeline you have already tuned around one model's behaviour.

How to evaluate it on your corpus

Generic leaderboards will not tell you whether R7B answers your users' questions well. A small, honest test will:

1. Take thirty real questions with known answers from your own documents, including at least five whose answer is not in the corpus at all.
2. Run your real retriever, then pass the top chunks to R7B with a preamble that demands citations and a refusal when the documents do not contain the answer.
3. Score three things separately: whether the answer is correct, whether every cited passage actually supports the claim, and whether it refuses on the unanswerable five.
4. Repeat in each language your users write in. Twenty-three supported languages is a training claim, not a guarantee of equal quality.

If it refuses cleanly and cites accurately, a small model is doing a small model's job well. If it invents answers on the unanswerable questions, no price is low enough. Building a docs chatbot that refuses to guess goes further into that refusal behaviour.