A 4-bit Model Just Beat Its Full-Precision Original: What That Means for Your Inference Bill

Key takeaways

• Quantization-Aware Healing (QAH) distills a compressed, 4-bit model directly from the original full-precision teacher, not from the intermediate recovered checkpoint most pipelines use.
• Applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, the QAH model beat its own bfloat16 (16-bit) version on 7 of 9 benchmarks.
• The 4-bit model runs on roughly a quarter of the weight memory of its 16-bit source, and about half the compute per token of the original 120B teacher.
• For founders paying per-token API prices or running their own inference, this points at a widening gap between good-enough quality and the compute it actually costs to deliver it.

Why this matters beyond the benchmark table

Every provider price cut, every new efficient model release, and every open-weight compression trick eventually shows up in one place: your monthly inference bill. Say you are running a support-ticket triage feature at 2 million tokens a day. A technique that halves compute per token without hurting output quality does not change your prompt or your product, but it can meaningfully change your margin on that feature.

That is the real story behind Quantization-Aware Healing. Multiverse Computing's team found that when a model has been both structurally compressed (fewer layers, heads, or neurons) and quantized to 4-bit, the standard recovery methods, quantization-aware training or quantization-aware distillation, both run into a ceiling. They typically re-train or re-distill from the already-degraded recovered checkpoint, which caps how good the final 4-bit model can get.

How does Quantization-Aware Healing work?

QAH skips that ceiling by distilling straight from the original, full-precision, pre-compression model, treating it as a frozen teacher. The compressed, quantized student never sees hard labels, only the teacher's output distribution, matched through a KL-divergence loss. Because that comparison is architecture-agnostic, a much smaller 4-bit student can still learn from a much larger 16-bit teacher.

Applied to GPT-OSS 120B, compressed down to 60B parameters and quantized to MXFP4, the QAH model beat its own bfloat16 60B version on long-context reasoning (+7.4 points on AA-LCR), math (+5.6 on AIME 2025), agentic coding, tool use, science knowledge, instruction following, and coding. It lost narrowly on two benchmarks, MMLU-Pro and SciCode, by less than a point and a half.

What does this cost in practice?

The efficiency numbers are the part that should catch a founder's attention. At 4-bit precision, the healed model uses roughly four times less weight memory than its bfloat16 counterpart. Running at half the parameter count of the original 120B teacher, it roughly halves compute per token. Multiverse notes that for model families shipping in bfloat16 rather than already-quantized formats, the combined savings from smaller size and lower precision would land closer to eight times less compute per token.

None of that is a guaranteed price cut from your provider tomorrow. But it is a preview of where the unit economics of serving LLMs are heading: models that cost less to run without asking you to accept worse output. If you price a product per seat or per usage tier, that shift changes what good margin looks like a year from now, not just today.