Even before the emergence of GPT-5, OpenAI seems to have begun training GPT-6, but power supply might be a "roadblock" issue.Kyle Corbitt, co-founder and CEO of AI startup OpenPipe, revealed that he recently had a conversation with a Microsoft engineer responsible for the GPT-6 training cluster project, who complained that deploying infiniband-level connections between GPUs across regions was a painful process.
When Corbitt asked why the training cluster couldn’t be centralized in the same region, the Microsoft engineer replied, “Oh, we’ve tried that, but if we place more than 100,000 H100 GPUs in one state, the power grid will collapse.”
What does 100,000 H100 GPUs mean? As a reference, a report from market research firm Factorial Funds shows that OpenAI’s DALL-E video model Sora requires 720,000 H100 GPUs at peak times—according to the engineer, this quantity is enough to crash the power grid of seven states.
Can Data Center Profitability and Construction Time be Determined by Power Supply?
During the recently concluded S&P Global Cambridge Energy Week 2024 (CERAWEEK), energy industry executives from around the world discussed the advancement of AI technology in the industry and the huge demand for energy brought about by AI.
“By 2030, the electricity consumed by AI will exceed household electricity consumption.” Toby Rice, CEO of America’s largest natural gas producer, EQT, cited such a prediction in his speech.
Bill Vass, Vice President of Engineering at Amazon Web Services, pointed out that the world adds a new data center every three days.
Bill Gates also stated that electricity is the key to determining whether data centers can be profitable, and the amount of electricity consumed by AI is staggering. The use of AI will drive energy demand, and chip design and power supply may constrain AI development in the coming years.
This is not an unfounded concern—due to the faster construction of new data centers compared to new power plants, a supply-demand gap has begun to appear. According to CBRE Group, Inc., a commercial real estate services company, the construction time for data centers has been extended by 2 to 6 years due to delayed power supply.
“The Energy Beast” of AI
The title of “The Energy Beast” for AI is not without merit.
Sam Altman of OpenAI has lamented about the energy, especially electricity, needed for AI. At the Davos Forum earlier this year, he said that AI development requires breakthroughs in energy, and AI will bring about electricity demand far beyond expectations.
Data shows that ChatGPT consumes over 500,000 kilowatt-hours of electricity per day to process approximately 200 million user requests, which is more than 17,000 times the daily electricity consumption of American households. As for search giant Google, if it uses AIGC for every user search, its annual electricity consumption will increase to around 29 billion kilowatt-hours, even exceeding the annual electricity consumption of countries like Kenya and Guatemala.
Looking back to 2022, when AI had not yet caused such a widespread frenzy, data centers in China and the United States accounted for 3% and 4% of their respective total societal electricity consumption.
With the gradual increase in global computing power, a report from Huatai Securities predicts that by 2030, the total electricity consumption of data centers in China and the United States will reach 0.95/0.65 trillion kWh and 1.7/1.2 trillion kWh, respectively, which is more than 3.5 times and 6 times that of 2022. Under an optimistic scenario, by 2030, the electricity consumption of AI in China/US will reach 20%/31% of the total societal electricity consumption in 2022.
Analysts further point out that due to the uneven distribution of data centers, regional power shortage risks will appear first (such as in Virginia, USA). Considering the almost zero growth in US electricity historically, AI will become an important driving factor for the return of positive growth in electricity consumption in developed overseas regions.
Where Does the Incremental Electricity Come From?
With power shortages, “new electricity” is naturally needed, but where does it come from? Amid the global trend of carbon neutrality, clean energy represented by photovoltaics and wind power seems to be the first choice, but this is only the “ideal choice.”
“We cannot build 100 gigawatts of renewable energy (power plants) in a few years. It’s a bit difficult.” Ernest Moniz, former US Secretary of Energy, admitted.
Toby Rice, CEO of EQT, added that tech companies need reliable electricity, which renewable energy sources like wind and solar cannot provide. As for large nuclear facilities (currently only one under construction in the United States), they have always been costly to build and time-consuming. “Tech companies won’t wait 7-10 years for these infrastructure projects, so natural gas is the only option.”
This executive from the US natural gas giant said that tech companies planning to build data centers have inquired about purchasing natural gas from EQT. Rice was asked, “How fast can your delivery be?” “How much natural gas can we get?”
US Stocks “No Longer in the Shadows”
First, it was the “GPU shortage,” and now it’s the “power shortage.” The road to AI development is not smooth sailing.
It is worth noting that US stock investors who want to seize the AI wave have turned their attention to this area.
Over the past year, the stock prices of Vistra Energy, one of the largest electricity producers and retail energy suppliers in the United States, Constellation Energy, the largest energy company in the United States, and NRG Energy, the largest green electricity company in the United States, have all more than doubled, and they have reached historical highs this week.
Looking at the year-to-date and annual percentage changes over the past year, although these three companies’ performance is not as good as that of “the strongest stock on the surface,” Nvidia, they have outperformed Microsoft, the “company behind OpenAI.”
