Mistral puts Ministral 3B and 8B on devices with 128k context

Mistral AI introduced Ministral 3B and Ministral 8B on October 16, 2024, two models the company built for on-device computing and at-the-edge use and announced together as les Ministraux. Mistral says Ministral 3B already outperforms its own Mistral 7B on most benchmarks, and reports the base model scoring 60.9 on MMLU against 52.4 for Gemma 2 2B and 56.2 for Llama 3.2 3B.

Both models handle up to 128k tokens of context, which the announcement says was capped at 32k when run on vLLM. Ministral 8B uses what Mistral calls an interleaved sliding-window attention pattern, where a layer attends only to a limited span of recent tokens instead of the whole sequence, which the company says makes inference faster and lighter on memory.

Mistral reports the instruction-tuned Ministral 3B at 8.1 on MTBench, 64.3 on Arena Hard and 77.4 on HumanEval, the last of those above Gemma 2 2B at 42.7 and Llama 3.2 3B at 61.0. Ministral 8B reaches 8.3 on MTBench and 70.9 on Arena Hard in the same table, while Gemma 2 9B stays ahead on WildBench at 43.8 against 41.3. All of these are Mistral’s own numbers, produced with an internal framework the company says it used to re-evaluate every model for the comparison.

Mistral names on-device translation, smart assistants that work without an internet connection, local analytics and autonomous robotics as the applications customers and partners asked for, citing demand for local, privacy-first inference. The company also positions the two models as intermediaries in front of larger ones such as Mistral Large, tuned to parse input, route tasks and call APIs across multiple contexts in multi-step agent workflows.

Both models went live through Mistral’s API on the day of the announcement, at $0.04 per million tokens for Ministral 3B and $0.1 per million for Ministral 8B, with input and output priced the same. Mistral published weights for Ministral 8B Instruct only, for research use, and lists that model under both the Mistral Research License and the Mistral Commercial License, while Ministral 3B appears under the commercial license alone with no weights release. Anyone wanting to self-deploy either model has to contact Mistral for a commercial license, and the company says it will help with lossless quantisation for specific use cases.