Token pricing is in a state of flux, characterized by a supply crunch and market instability. As we navigate through these turbulent waters, it’s crucial to understand the dynamics at play and anticipate future shifts. Currently, the market is rife with speculation about ‘time to power’, but the fundamental question remains: can foundation models secure sustainable pricing power and strategic leverage, or will they devolve into low-margin, commodity infrastructure? Presently, all signs suggest the latter.
Current Market Dynamics: Supply and Demand Challenges
The current situation is undeniably transient. On the supply side, we anticipate over a trillion dollars in data center capital expenditures, with significant semiconductor investments following suit. Inference efficiency is rapidly advancing, and new models exhibit varying efficiency levels in token usage. Conversely, the market has been capacity-constrained since 2022, driven primarily by successful product-market fit in software development—a relatively niche field. If consumer use cases with hundreds of millions of daily active users emerge, today’s infrastructure would be insufficient at any price. The future use cases and their token needs remain unknown.
Understanding Inference and Training Costs
Inference currently boasts 40-50% gross margins, accounting for server depreciation or rental costs. However, we lack clarity on asset life expectancy and the significant training costs of new models, which currently surpass revenue. Inference is a marginal cost, whereas training is a fixed cost. Achieving profitability hinges on high revenue, but the evolution of training costs is uncertain. Furthermore, it’s unclear how much of the recent surge in usage has a return on investment (ROI) or what prices users might be willing to pay for future applications.
Forecasting the Equilibrium: A Complex Task
In theory, one could model token pricing from a bottom-up perspective by considering each variable, such as chip availability and performance, and projecting their integration into data centers. However, this approach resembles attempting a five-year forecast for the broadband market in 1998—too many unknowns render long-term predictions unreliable. Ultimately, token pricing is a function of supply and demand between sellers’ marginal costs and buyers’ ROI, but the specifics remain elusive.
Top-Down Approach: Examining Broader Trends
Alternatively, a top-down perspective reveals broader trends. The key questions include how many users will pay for cutting-edge models, whether the technological frontier continues to advance, and if fierce competition will persist among frontier models. Additionally, we must consider how much value high-end use cases capture from frontier models versus other complementary elements, such as tooling, data, and support.
Exploring Potential Outcomes: Commoditization vs. Market Dominance
The future of token pricing varies significantly by use case. In one scenario, a few dominant players might control the market, enjoying significant pricing power. In another, models resemble databases—numerous and commoditized, with value derived from applications built on top. While it’s tempting to draw parallels with industries like mobile data or semiconductor manufacturing, these analogies lack predictive value. Each industry and technology is unique, and AI will forge its own path.
Conclusion: The Path Forward Remains Uncertain
The landscape of token pricing is fraught with uncertainty. While current dynamics suggest a trend towards commoditization, unexpected developments could reshape the market. Regulatory changes, technological breakthroughs, or shifts in competitive dynamics could alter the trajectory. Until then, businesses and investors must remain vigilant, ready to adapt to emerging opportunities and challenges.
