Prominent investor Mary Meeker's latest AI report reveals a key contradiction in the industry: the continuous surge in the cost of training AI models has reached the billion-dollar level, while the cost of inference has plummeted by 99% due to breakthroughs in hardware and algorithms. This extreme divergence in cost structure is reshaping the commercial landscape of the entire AI industry.
This report not only showcases the spectacular scene of AI technology development but also reveals a harsh reality: the AI industry is undergoing an unprecedented capital-intensive competition, and only a few leading players can afford to participate.
Training Costs: The Beginning of a Billion-Dollar Arms Race
Dario Amodei, CEO of Anthropic, made a shocking prediction: the training cost for advanced large language models in 2024 has reached $100 million, with some in-training models even approaching $1 billion. More remarkably, he expects that the first projects with training costs exceeding $1 billion will emerge between 2025 and 2027.
This prediction is not without basis. Report data shows that from 2016 to 2024, the training cost of cutting-edge AI models increased approximately 2,400 times, jumping from the million-dollar level to hundreds of millions of dollars. This exponential growth has created an "arms race" where only top players can participate, pushing a large number of medium and small-sized AI companies out of the core competitive track.
The surge in costs stems from an insatiable demand for computational resources. Training the most advanced AI models requires thousands or even tens of thousands of high-end GPUs running continuously for months, with hourly electricity and hardware depreciation costs reaching tens of thousands of dollars. This heightened capital threshold is gradually concentrating the research and development of AI foundational models among a few tech giants.
Inference Costs: Hardware Revolution Driving Application Proliferation
In stark contrast to the skyrocketing training costs, inference costs are experiencing a collapse-like decline. Stanford University research data shows that within the past two years, the cost per million tokens for inference has dropped by 99%, which is astonishingly steep.
The energy efficiency improvements of NVIDIA GPUs have been particularly significant. The energy consumption for generating a single token using Blackwell GPUs released in 2024 is reduced by an incredible 105,000 times compared to Kepler GPUs released in 2014. This accelerated pace of hardware iteration is directly driving the widespread adoption of AI applications.
The impact of low-cost inference is immediate. Tools like ChatGPT have been able to quickly gain hundreds of millions of users, largely thanks to the dramatic reduction in inference costs, allowing these applications to provide services to users at extremely low marginal costs. This change in cost structure is sparking developers' innovation enthusiasm, driving rapid implementation of AI applications across various vertical fields.
Commercial Challenges: The Harsh Balance Between High Input and Low Pricing
The extreme differentiation in cost structures presents unprecedented commercial challenges for AI model providers. On one hand, companies must continue to invest heavily in model training to maintain technological leadership; on the other hand, market competition forces them to offer inference services at extremely low prices.
OpenAI's financial situation perfectly illustrates this dilemma: its computing expenses grow almost in sync with revenue growth, and there is even a risk of losses. Even for cash-rich tech giants like Microsoft and Amazon, their free cash flow margins face immense pressure after increasing AI investments.
Mary Meeker cleverly drew parallels with historical cases like Amazon and Tesla in her report, pointing out that current AI companies are at a critical strategic juncture: they must find a delicate balance between "burning money" and building technological barriers.
Network Effects: The Only Path to Sustainable Profitability
The report emphasizes that, under the current cost structure, only AI companies capable of forming strong network effects can achieve sustainable profitability. This means that pure technological advantages are no longer sufficient; companies must also build user stickiness, data flywheels, and ecosystem moats.
The importance of network effects lies in its ability to create economies of scale. When the user base reaches a critical point, the rate of decline in marginal costs will exceed the rate of reduction in marginal revenues, thus fundamentally improving profitability. This explains why companies like OpenAI and Google are actively building developer ecosystems and application platforms.
Industry Shakeout: A New Landscape of Increased Differentiation
Mary Meeker's report actually foreshadows a major shakeout in the AI industry. The continuous rise in training costs will further increase industry entry barriers, leaving only well-funded leading enterprises to compete in foundational model development. At the same time, the decline in inference costs will spawn a wave of application innovations based on existing models, providing new opportunities for small and medium-sized enterprises and startups.
This trend toward differentiation may lead to an "hourglass-shaped" structure in the AI industry: one end consists of a few model providers mastering core technologies, while the other end comprises numerous companies focusing on application innovations. Companies in the middle layer will face risks of being squeezed out.
The current cost structure contradictions in the AI industry are both the inevitable result of technological development and a profound reflection of commercial competition. In this unprecedented technological revolution, understanding and adapting to these changes in cost structures will become the key factor determining the survival of businesses.