What If Two Bits Were Enough?
Here’s a wild thought: what if I told you that storing a neural network weight requires less than two bits? Not two bits per layer. Not two bits per neuron. Two bits per weight. And what if I told you that a model trained this way actually works?
You’d probably think I was selling you something.
But BitNet and its kin are doing exactly that. They’re storing weights as ternary values: -1, 0, or 1. That’s technically 1.58 bits (thanks, information theory). And the kicker? These models are competitive with full-precision models on tasks that matter. You can run a 100-billion-parameter model on a laptop CPU. No GPU. No special hardware. Just a MacBook and some patience.
This is quantization taken to its logical extreme. And it’s about to change what “local AI” means.
The Quantization Ladder (Quick Recap)
You’ve probably heard the terms. Full precision is float32 (32 bits per weight). That’s the baseline, bloated and slow but accurate. We’ve climbed down the ladder:
- float32: 32 bits per weight. Your model is gigantic.
- float16: 16 bits. Still quite large. Good for GPUs.
- int8 (8-bit): 8 bits. What typical phone models use. Getting snappy.
- Q4_K_M (4-bit): 4 bits with some clever scaling. Where llama.cpp lives. This is the sweet spot most people find today.
- 1-bit: -1, 0, or 1. Sounds like a joke. Isn’t.
Each step down, you lose precision. But here’s the secret: if you train the model knowing it will be 1-bit (instead of quantizing a pre-trained model), you can recover most of what you’d lose. It’s not a hack. It’s not a shortcut. It’s architecture.
Enter BitNet b1.58
Microsoft Research dropped the paper in late 2024. They called it BitNet b1.58: 1-bit LLM, Trained with All 1s and −1s. The name is a flex, but the paper is serious.
Here’s what they did: instead of training a model in float32 and then squashing it to 1-bit (which sounds like trying to fit a whale into a shoebox), they trained models from scratch where every weight could only be -1, 0, or 1. The model learned to work within those constraints during training.
The results were wild. A 3.3 billion-parameter BitNet model matched or beat a 13 billion-parameter Llama 2 on benchmarks. A 7 billion-parameter BitNet hit parity with Llama 2 7B. They scaled to 70 billion parameters.
How? Matrix multiplication with ternary weights is just addition and subtraction. No floating-point arithmetic needed. Your CPU can handle this all day. No quantization overhead. No dequantizing before each operation. Just pure arithmetic.
The Hardware Implications (This Is Why It Matters)
Here’s why I’m excited, and why this isn’t just academic navel-gazing:
A 1-bit model can run on older hardware that hasn’t had an update since 2015. A Raspberry Pi 5. An Intel NUC from 2018. A MacBook Air from 2017. That’s not hyperbole. That’s feasible.
Why? Because matrix multiplication complexity drops from O(n²) floating-point operations to O(n²) integer additions. Your CPU laughs at that. No specialized GPU needed. No VRAM bottleneck. No quantization artifacts sneaking into your results because the model was trained to expect them.
The implications cascade:
- Mobile: A phone can run a capable LLM entirely on-device. No cloud calls. Privacy by default.
- Edge: Routers, cameras, IoT devices. Everything gets local inference.
- Deployment: No Nvidia monopoly. Intel, ARM, old Xeons—everything is viable.
- Cost: A GPU-less laptop runs a 100B model. The economics of AI flip.
1-Bit Bonsai: The First Production Play
BitNet was the proof of concept. But 1-Bit Bonsai (early 2026) is the first model that hit the “actually deployable” bar for real tasks. It’s smaller, faster, and built with inference in mind from day one.
Bonsai models come in compact sizes (7B, 13B, 32B) and are trained specifically for consumer hardware. Think of it as BitNet’s production sibling.
How to Run a 1-Bit Model Today
llama.cpp supports BitNet models. Here’s how to get started:
# Download a BitNet GGUF model from HuggingFacehuggingface-cli download --repo-type model ggml-org/models \ --include "*.gguf" bitnet-b1_58-3b-gguf
# Or wget it directlywget https://huggingface.co/ggml-org/bitnet-b1_58-3b-gguf/resolve/main/model.gguf# Run it with llama.cpp./llama-cli -m bitnet-b1_58-3b.gguf \ -n 256 \ -p "Why is quantization important?" \ --threads 4That’s it. No GPU. Four CPU threads. You’ll see tokens/sec that rival GPU inference from two years ago.
For a performance comparison, here’s what you’re looking at (rough tokens/sec on CPU):
Model Hardware Tokens/sec PowerBitNet 3B M2 CPU 12-18 5WBitNet 7B i7-12700K 8-12 25WLlama 2 7B Q4 M2 GPU 20-25 10WFull Llama 2 7B RTX 4090 80-100 320WThe BitNet 3B isn’t faster than a quantized Llama, but it is close, and it runs on hardware you already own. Scale to 32B or 70B, and the story flips: a 32B BitNet on a CPU crushes a quantized Llama 2 13B on the same hardware.
What You’re Not Getting (Yet)
1-bit models aren’t ready to replace your favorite 70B model. They’re not competitive for complex reasoning tasks. They’re not state-of-the-art on benchmarks.
But here’s the thing: most people don’t need the bleeding edge. You need fast, local, private inference on your hardware. For classification, summarization, coding, and creative writing at 7B-13B scale? BitNet works. For edge devices, NPUs, and Raspberry Pi clusters? BitNet is the only game in town.
The Future Is Weird And Hardware-Agnostic
What excites me is the direction. We’ve spent the last three years assuming “bigger model, bigger GPU.” BitNet flips that. It says: “What if we made the model work with what people actually have?”
This opens doors:
- NPU support explodes (because NPUs are just integer arithmetic anyway).
- Phones and tablets become first-class AI inference targets.
- Hobbyists can build Raspberry Pi clusters that would’ve cost $50k in GPU money.
- Developing countries with older hardware suddenly have access to capable models.
The quantization ladder isn’t done. There’s probably a 0.5-bit future, or some exotic ternary scheme we haven’t thought of yet. But 1-bit feels like the practical floor where quality and efficiency meet.
Download a BitNet model. Fire up llama.cpp. Run it on your 2015 laptop. Then ask yourself: why did we ever think we needed a $10,000 GPU?
The endgame of quantization is making AI accessible. And 1-bit models just made it personal.
