The head of technology at a medium-sized software company charted model spending over the past year: training costs were flat, reasoning costs tripled, and the reason for the change was simple-models went from being called occasionally to being called every interaction.

This changed the competitive dimension of the industry. Size was still important, but customers were starting to ask a different set of questions: how much was the delay per request, how much per thousand calls, and whether peak periods would hold up.

Cost curve determines product form

When the cost of a single inference drops by an order of magnitude, products that are not established will suddenly be established. For example, real-time subtitles, long document summaries, and meeting minutes are organized. These functions can only be used as value-added services when the cost is high, and only when the cost is lowered will they have the opportunity to become default configurations.

"The capacity of the model determines the upper limit, and the unit cost determines whether it can enter the daily life," said a product owner.

The action on the supply side is also obvious. The procurement structure of the accelerator card has changed from "a small amount of high performance" to "a large amount of medium performance" to match the distribution characteristics of the inference load; the location of the data center has begun to pay more attention to electricity prices and network delays than pure land costs.

A new watershed

Analysts believe that the future differentiation will not appear in "there is no model", but in "whether the cost can be controlled within the gross profit". This is an opportunity for small and medium-sized teams, but also a pressure: the technical threshold has been lowered, and the operational threshold has been raised.

"We're no longer competing with who has a bigger model," said the head of the product."We're competing with who can make the experience more stable for the same price."