There are at least three inference markets

One way to slice the inference market is to cluster workloads by intelligence saturation. How much intelligence and consequently what type of model do you need for each task?

At least three markets with distinct characteristics emerge from this line of thinking.

Personal inference

First is what I'd call personal inference. These are simple inference workloads primarily related to your personal data. Think of tasks like summarizing an email, organizing a folder, or drafting a routine reply. How much more performance would you get from a smarter model? Once a particular model is good enough, making it substantially smarter adds little incremental value.

This inference bucket has the best opportunity to make marginal cloud inference costs disappear altogether, driven by privacy concerns and the availability of smaller models that run locally on consumer devices.

Startups in this category can't just markup tokens and call it a business model. Instead, they have to compete on the harness around the model: permissions, memory, integrations, and personalization.

Enterprise inference

Enterprise inference, on the other hand, is extraordinarily varied. There are a broad range of use cases across various complexity levels. A document OCR task may be saturated by a small specialized model, but a frontend coding task may require a mid sized open source model. Furthermore, a long horizon planning agent may need a frontier model.

In this world, it doesn't make much economic sense to send every query to the same model. Instead, we're heading towards a future where enterprises have a portfolio of models (open source, custom post trained, and frontier) and match the right task to each model.

The actual mix of inference traffic will differ by enterprise, but the stack required to support this multi-model world is starting to crystallize: automated evals establish which models can reliably handle which tasks, a router sends each task to the appropriate model, and a feedback loop scores outputs and updates routing policies as necessary.

For startups, this routing paradigm creates an opportunity to arbitrage the cost-quality frontier as a monetization strategy. Enterprise inference is therefore fundamentally an allocation problem: use no more intelligence than a task requires.

Industrial inference

At the other end of the spectrum is what I'd call industrial inference. These are workloads where we are nowhere close to intelligence saturation so more model capabilities and more inference-time compute spend can produce large improvements in the probability of solving a task. And because solving the task can be enormously valuable, there is correspondingly high willingness to pay for marginal intelligence.

I find the ARC-AGI benchmark to be one of the cleanest examples of this dynamic. Take the exact same model and vary only how much reasoning it is allowed to do and you get meaningful improvements. On ARC-AGI-3, GPT-5.6 Sol scores 0.33% at low reasoning effort, 1.07% at medium, 2.15% at high, 6.99% at xhigh, and 7.78% at max. Increasing inference-time reasoning on the same underlying model produces a more than 20x improvement, yet the model still solves fewer than one in ten tasks. 

ARC-AGI matters not because the benchmark itself represents a market, but because it shows us the shape of workloads that can still absorb much more intelligence. I am especially interested in markets where the output of reasoning changes the physical or computational world. For example, chip design, drug discovery, materials science, logistics, or industrial engineering. In these markets, small improvements in solution quality can translate into enormous downstream value.

The inference market won't be one giant pool of interchangeable tokens. It will fragment according to a simpler question: how much intelligence can a task economically absorb?