Provider Substitution: The AI Cost-Cutting Lever Microsoft Just Pulled

Key takeaways

• Microsoft has begun handling a percentage of Office prompts with in-house MAI models rather than third-party APIs, per a Bloomberg report.
• This is partial routing, not a clean break: OpenAI and Anthropic still power many of its services.
• Provider substitution is one of the strongest levers for AI gross margin, because inference is usually the largest variable cost per request.
• You do not need to switch 100% of traffic to save; routing even a slice of prompts to a cheaper model moves your blended cost per request.
• The same math that works for Microsoft works for a two-person SaaS: model it before you migrate.

What did Microsoft actually change?

Microsoft has started using its homegrown MAI models to answer a portion of user prompts inside Excel and Word, according to Bloomberg. Previously the company marketed Office 365 as heavily powered by OpenAI and Anthropic models. It is not walking away from those providers; it is diversifying so that some requests run on cheaper in-house inference. At its Build conference it introduced seven new MAI models, including an agentic coder and a text-to-image generator, giving it more in-house options to route to.

Why is provider substitution such a powerful cost lever?

For most AI products, inference is the biggest variable cost: it scales with every request, unlike a fixed engineering salary. When the model call is your dominant cost of goods sold, the price you pay per million tokens flows straight into gross margin. Swap a frontier model for a cheaper one on the requests that do not need the flagship, and you cut COGS without touching headcount or raising prices.

That is why the giants are moving in the same direction. After a burst of 'tokenmaxxing' earlier this year, Amazon, Uber, Meta and Accenture have all reportedly tightened AI spending. Some teams are even testing cheaper Chinese models for agentic work. The common thread: the token bill got large enough to manage like a real line item.

How much can partial routing actually save?

The trap is thinking substitution is all-or-nothing. Microsoft is routing 'a certain percentage' of prompts, not every one. That partial approach is the model to copy, and the math is straightforward.

Say, illustratively, your premium provider costs P per million tokens and an in-house or cheaper model costs a fraction of that. If you route a share r of your traffic to the cheaper model, your blended cost becomes:

blended = (1 - r) x premium + r x cheaper

For example, if a flagship call costs roughly 10x a lightweight model, moving even 40% of low-complexity prompts to the cheaper option can cut your blended inference cost by a meaningful double-digit percentage, while your hardest prompts still get the flagship. The exact number depends on your traffic mix, which is precisely why you model it rather than guess. Treat these figures as illustrative and plug in your own current provider rates.

What should founders take from this?

Three things. First, audit which requests actually need a frontier model: many 'AI features' are classification, extraction or short rewrites that a small model handles fine. Second, treat routing as a dial, not a switch, and start with the safest slice of traffic. Third, watch quality, not just price: a cheaper model that forces retries or human cleanup can erase the savings.

The strategic point is that provider choice is now a margin decision, not just an engineering one. Microsoft has the scale to build MAI in-house; smaller teams get the same lever by mixing providers and open models. The discipline is identical: know your cost per request before and after.

Before you migrate a single endpoint, model the blended cost across providers and routing ratios. You can run that comparison in Calcaas in a few minutes.