Inference Is the Most Important Market in Software
Short summary
AI inference spending will overtake databases as the largest software market by 2027, fundamentally breaking the economics that made traditional SaaS profitable. This shift forces application companies to manage massive infrastructure costs that will compress margins and demand new operational approaches.
- Inference market grows from $25b (2025) to $350b (2027), passing databases at roughly 2x the size, making it the dominant software category.
- Gross margins collapse below SaaS norms (72%) because inference costs become a major COGS line item, forcing vendors to either build proprietary models to reduce token overhead or accept margin compression.
- Customer-owned infrastructure (BYOK) trades top-line revenue for gross margin—vendors get higher percentage margins but on drastically smaller contracts.
- Token price wars won't solve the problem; capability improvements keep resetting frontier model prices upward, so companies must optimize through smaller models, better routing, and efficient harnesses.
Full summary
Inference—the cost of running AI models—is about to become the biggest software market, surpassing databases. In 2025, companies spent $25 billion on inference; by 2027, that balloons to $350 billion, nearly double what databases will reach. This isn't just growth—it's structural. Every software application is becoming an inference reseller, which fundamentally breaks the economics that made database companies valuable. A 2% market share in databases during the mid-2010s was enough to go public. That math doesn't work when your biggest cost is token consumption.
This shift destroys the unit economics of traditional SaaS in three ways. First, inference bills will dwarf subscription fees, meaning companies need to charge based on usage rather than seats, which breaks sales compensation models and makes budgeting terrifying for customers. Second, gross margins collapse. Classic SaaS sits around 72% margins, but when you're passing through massive infrastructure costs, that evaporates unless you build proprietary tools to compress tokens or ride a price war down. Third, customers who bring their own GPUs or API keys force vendors to choose: take a huge revenue hit or keep margins thin. The price war helps but doesn't solve this. Token costs drop 47% quarterly, yet frontier models keep resetting prices upward—GPT-4's blended price jumped from $0.96 to $11.25 in 18 months before the next reset.
The parallel is straightforward: software companies will act like long-haul truckers watching diesel prices spike. They'll obsess over cost per token the way truckers obsess over fuel economy. That means smaller models, better code to reduce token waste, and smarter routing. The constraint creates the opportunity. Companies that figure out how to run inference efficiently will win. Those that don't will watch their margins disappear.