Token pricing is in a state of flux, characterized by a supply crunch and market instability. As we navigate these turbulent waters, it is crucial to understand the dynamics at play and anticipate future shifts. Currently, the market is rife with speculation about ‘time to power,’ but the fundamental question remains: Can foundation models secure sustainable pricing power and strategic leverage, or will they devolve into low-margin, commodity infrastructure? At present, all signs point to the latter.
Current Market Dynamics: Challenges in Supply and Demand
The current situation is undeniably temporary. On the supply side, we anticipate over a trillion dollars in data center capital expenditures, with significant semiconductor investments following suit. Inference efficiency is advancing rapidly, and new models exhibit varying levels of efficiency in token usage. Conversely, the market has been capacity-constrained since 2022, driven primarily by successful product-market fit in software development—a relatively niche field. If consumer use cases with hundreds of millions of daily active users emerge, today’s infrastructure would be insufficient at any price. The future use cases and their token requirements remain unknown.
Understanding Inference and Training Costs
Inference currently boasts gross margins of 40–50%, taking into account server depreciation or rental costs. However, we lack clarity on asset life expectancy and the significant training costs of new models, which currently exceed revenue. Inference is a marginal cost, whereas training is a fixed cost. Achieving profitability depends on high revenue, but the future trajectory of training costs is uncertain. Furthermore, it is unclear how much of the recent surge in usage yields a return on investment (ROI) or what prices users might be willing to pay for future applications.
Forecasting Equilibrium: A Complex Task
In theory, one could model token pricing from a bottom-up perspective by considering each variable—such as chip availability and performance—and projecting their integration into data centers. However, this approach is akin to attempting a five-year forecast for the broadband market in 1998—too many unknowns make long-term predictions unreliable. Ultimately, token pricing is a function of supply and demand, balancing sellers’ marginal costs against buyers’ ROI, but the specifics remain unclear.
Top-Down Approach: Examining Broader Trends
Alternatively, a top-down perspective reveals broader trends. The key questions include how many users will pay for cutting-edge models, whether the technological frontier continues to advance, and whether fierce competition will persist among frontier models. Additionally, we must consider how much value high-end use cases derive from frontier models compared to other complementary elements, such as tooling, data, and support.
Exploring Potential Outcomes: Commoditization vs. Market Dominance
The future of token pricing varies significantly depending on the use case. In one scenario, a few dominant players might control the market, enjoying significant pricing power. In another, models resemble databases—numerous and commoditized, with value derived from applications built on top. While it’s tempting to draw parallels with industries like mobile data or semiconductor manufacturing, these analogies lack predictive value. Each industry and technology is unique, and AI will forge its own path.
Conclusion: The Path Forward Remains Uncertain
The landscape of token pricing is fraught with uncertainty. While current trends point toward commoditization, unexpected developments could reshape the market. Regulatory changes, technological breakthroughs, or shifts in competitive dynamics could alter the trajectory. Until then, businesses and investors must remain vigilant and ready to adapt to emerging opportunities and challenges.
