Open-Weight vs Frontier LLMs: How to Model the Real Cost Difference

Key takeaways

• Open-weight models have narrowed the capability gap to months, not generations, which turns 'can we use one' into 'should we, at what cost'.
• A frontier API price is a bundle: compute, plus refusal training, classifiers, API-level controls, uptime and evaluation work.
• When you self-host open weights, you inherit every part of that bundle that you still need, and you pay for it in engineering time instead of tokens.
• The honest comparison is cost per successful request at your quality bar, not cost per million tokens.
• Break-even depends on volume and utilisation more than on the headline rate.

Why is the open-weight question suddenly a pricing question?

For most of the last few years the answer was easy: frontier models were meaningfully better, so you paid the frontier price. That gap has compressed. A recent SaferAI evaluation of Z.ai's open-weight GLM-5.2 put it only a few months behind leading closed models on the capability dimensions it tested, while noting that the safety mitigations shipped alongside it were far thinner.

That second half is where the cost story lives. Once capability is roughly comparable, the remaining difference between an open-weight model and a frontier API is largely operational, and operational differences are budget lines.

What does an API token price actually buy you?

It helps to unbundle the number on the pricing page. When you pay a per-token rate to a frontier provider, you are buying at least five things:

1. 1Inference compute, the raw GPU seconds.
2. 2Capacity and elasticity, so you are not paying for idle hardware between traffic spikes.
3. 3Serving reliability, uptime, autoscaling, and someone else's on-call rotation.
4. 4Safety mitigations, refusal training, classifiers, and API-level controls.
5. 5Evaluation and documentation, the pre-deployment testing and system cards that your enterprise buyers and auditors ask about.

Only the first item disappears cleanly when you switch to a cheaper model. The other four move onto your roadmap.

How do you model the switch from a frontier API to self-hosted open weights?

Build the comparison in four steps rather than one:

Step 1: measure your real token shape. Input and output tokens are usually priced very differently, and most products are far more input-heavy than founders assume. Model your actual ratio, not a round number.

Step 2: convert self-hosting to an effective per-token rate. Take monthly GPU cost, divide by tokens actually served, and be honest about utilisation. A cluster running at 30 percent utilisation has an effective token cost roughly three times its theoretical one.

Step 3: add the bundle you inherited. If you need refusal behaviour, content filtering, or an audit trail, that is engineering headcount and eval spend. Amortise it across your token volume the same way you would any other fixed cost.

Step 4: compare cost per successful request. If a cheaper model needs more retries, longer prompts, or a stronger fallback model on hard cases, the per-token saving partly evaporates. Say a cheaper model is 70 percent less per token but needs a second pass on one request in five: the real saving is materially smaller than the headline.

When does self-hosting an open-weight model actually win?

The pattern is consistent. Self-hosting wins when volume is high, traffic is predictable enough to keep utilisation up, the task is narrow, and the quality bar is set by your own product rather than by an external auditor. It loses when volume is spiky, the team is small, or the buyer is an enterprise that will ask what your safety framework is.

There is also a middle option that gets skipped too often: running an open-weight model through a third-party inference provider. You keep most of the token saving and the model portability, and you do not take on the serving problem.

What is the hidden line item most founders miss?

Governance. The mitigations a frontier provider applies at the API level stop being enforceable the moment weights run on someone else's hardware, which is exactly the freedom that makes open weights attractive. If your product genuinely needs guardrails, that work does not vanish with the API bill. It just moves from a line you rent to a line you own, and owned lines do not scale down when traffic drops.

The useful discipline is to price the model swap as a full P&L change rather than a rate change. You can model the token side of this in Calcaas and see where the break-even actually sits for your volume.

Takeaway: open weights change what compute costs, not what running a reliable AI product costs, so compare the whole bundle before you migrate.