BitNet b1.58 gives every weight three values and runs 3B in 2.22 GB

Microsoft Research and the University of Chinese Academy of Sciences published BitNet b1.58 on February 27, 2024, a language model in which every single weight is one of three values, -1, 0 or 1. At 3B parameters the authors measure it at 2.22 GB of memory against 7.89 GB for a full-precision Llama-style model of the same size trained on the same number of tokens, a 3.55 times reduction, with a slightly lower perplexity of 9.91 against 10.04.

Three possible values need log2(3) bits to store, which is roughly 1.58 and where the name comes from. What that buys is a change to the matrix multiplication that dominates the work of running a model. A normal model multiplies each weight by an activation in floating point and sums the products, but multiplying by -1, 0 or 1 is only a sign flip, a skip or a copy, so the paper states that the matrix multiplication of BitNet “only involves integer addition”. Activations stay at 8 bits.

The models are trained from scratch at this precision rather than trained in full precision and squeezed down afterwards, which is how most low-bit models on phones are produced today. The authors trained BitNet b1.58 and their own reproduced Llama baseline on the same data for the same number of tokens at each size, and compare the two directly. They report that BitNet b1.58 starts to match the full-precision baseline in perplexity at 3B and on zero-shot end tasks from 3B as well, where it averages 50.2 against 49.7.

The efficiency figures widen with size. Memory drops 2.60 times at 700M, 2.93 times at 1.3B and 3.55 times at 3B, and latency per output token falls 1.23 times, 1.67 times and 2.71 times across the same three sizes. For throughput the authors ran 70B models on two A100 cards and report a batch size of 176 against 16 and 2977 tokens per second against 333, an 8.9 times gain. They also calculate that on 7nm chips the arithmetic for matrix multiplication costs 71.4 times less energy, because integer addition is far cheaper than floating-point multiply-accumulate.

The paper names no model release and its arXiv listing marks it as work in progress. The authors argue the approach suits CPUs, “which are the main processors used in edge and mobile devices”, and say the lower memory and energy cost is what would let such models run there. They close by calling for new hardware and systems designed specifically for 1-bit models.

Update, April 16, 2025. Microsoft released BitNet b1.58 2B4T, a 2B model trained from scratch on 4 trillion tokens at ternary precision, which the team calls the first open-source native 1-bit model at that scale. The report puts its non-embedding memory at 0.4 GB against 1.4 GB for Gemma-3 1B and 2.6 GB for Qwen2.5 1.5B, measures CPU decoding at 29 ms per token on a 13th Gen Intel Core i7-13800H using 8 threads against 41 ms and 65 ms for the same two models, and estimates energy at 0.028 J against 0.186 J and 0.347 J. Across the benchmark suite it averages 54.19, above Llama 3.2 1B at 44.90 and just under Qwen2.5 1.5B at 55.23. The weights are on Hugging Face, with a custom CUDA kernel for GPUs and CPU inference through bitnet.cpp, the runtime Microsoft published in October 2024 and measured at 2.37 to 6.17 times faster than the baseline on x86 CPUs and 1.37 to 5.07 times faster on ARM.