How much electricity do AI generators use?

It is well known that machine learning is a lot of energy. All those AI models powering email digests, regicide chatbots, and videos of Homer Simpson singing nu-metal add up to a hefty server bill, measured in megawatts per hour. But it seems no one – not even the companies behind the technology – can say exactly what the costs are.

Estimates do exist, but experts say these figures are partial and contingent, offering only a glimpse of AI’s overall energy consumption. This is because machine learning models are incredibly variable and can be configured in ways that dramatically change their energy consumption. Furthermore, the organizations best placed to draft a bill – companies like Meta, Microsoft and OpenAI – are simply not sharing the relevant information. (Judy Priest, CTO for cloud operations and innovations at Microsoft, said in an email that the company is currently “investing in developing methodologies to quantify the energy consumption and carbon impact of AI, while working on ways to make large systems more efficient , in both training and application.” OpenAI and Meta did not respond to requests for comment.)

An important factor we can identify is the difference between training a model for the first time and deploying it to users. Training in particular is extremely energy intensive and uses much more electricity than traditional data center operations. For example, training a large language model like GPT-3 is estimated to cost just under 1,300 megawatt hours (MWh) of electricity; approximately the same amount of energy as is used annually by 130 American households. To put that into context, streaming an hour of Netflix requires approximately 0.8 kWh (0.0008 MWh) of electricity. That means you would have to watch 1,625,000 hours to use the same amount of power needed to train GPT-3.

But it’s hard to say how such a figure applies to today’s state-of-the-art systems. Energy consumption could be higher because AI models have been steadily increasing in size for years and larger models require more energy. On the other hand, companies could use some of the proven methods to make these systems more energy efficient – ​​which would moderate the upward trend in energy costs.

The challenge in making up-to-date estimates, according to Sasha Luccioni, a researcher at French-American AI company Hugging Face, is that companies have become more secretive as AI has become more profitable. Go back a few years and companies like OpenAI would publish details of their training regimes – what hardware and for how long. But the same information simply doesn’t exist for the latest models, such as ChatGPT and GPT-4, Luccioni says.

“With ChatGPT we don’t know how big it is, we don’t know how many parameters the underlying model has, we don’t know where it’s running… It could be three raccoons in a trench coat because you just don’t know what’s under the hood .”

“It could be three raccoons in a trench coat because you just don’t know what’s under the hood.”

Luccioni, author of several articles examining the energy consumption of AI, suggests that this secrecy is partly due to competition between companies, but is also an attempt to deflect criticism. Statistics on energy consumption for AI – especially the most frivolous use cases – naturally invite comparisons to the waste of cryptocurrency. “There is a growing realization that none of this is free,” she says.

Training a model is only part of the picture. After a system is created, it is rolled out to consumers who use it to generate output, a process known as ‘inference’. Last December, Luccioni and colleagues from Hugging Face and Carnegie Mellon University published a paper (currently awaiting peer review) that contained the first estimates of the inferred energy consumption of various AI models.

Luccioni and her colleagues ran tests on 88 different models, spanning a range of use cases, from answering questions to identifying objects and generating images. In each case, they performed the task a thousand times and estimated the energy costs. Most of the tasks they tested use a small amount of energy, such as 0.002 kWh to classify written samples and 0.047 kWh to generate text. Using our hour of Netflix streaming for comparison, these equate to the energy consumed watching nine seconds or 3.5 minutes respectively. (Remember: that’s the cost of running each task 1,000 times.) The numbers were significantly higher for image generation models, which used an average of 2.907 kWh per 1,000 inferences. As the paper notes, the average smartphone uses 0.012 kWh to charge – so generating a single image using AI can consume almost as much energy as charging your smartphone.

However, the emphasis is on “can” because these numbers don’t necessarily generalize to all use cases. Luccioni and her colleagues tested ten different systems, from small models producing small 64 x 64 pixel images to larger models producing 4K images, and this resulted in a huge spread of values. The researchers also standardized the hardware used to better compare different AI models. This does not necessarily reflect real-world implementation, where software and hardware are often optimized for energy efficiency.

“This is certainly not representative of everyone’s use case, but at least now we have some numbers,” says Luccioni. “I wanted to put a flag in the ground that said, ‘Let’s start from here.’”

“The generative AI revolution comes at a planetary cost that is completely unknown to us.”

The research therefore provides useful relative data, but no absolute figures. For example, it shows that AI models require more power to generate output than to classify input. It also shows that everything that has to do with images is more energy intensive than text. Luccioni says that while the contingent nature of this data can be frustrating, it tells a story in itself. “The generative AI revolution has planetary costs that are completely unknown to us and the range is particularly indicative to me,” she says. “The tl;dr is that we just don’t know.”

So it’s difficult to determine the energy cost of generating one Balenciaga Pope because of the morass of variables. But if we want to better understand planetary costs, we need to take different paths. What if, instead of focusing on model inferences, we zoomed out?

This is the approach of Alex de Vries, PhD candidate at VU Amsterdam, who calculates Bitcoin’s energy consumption for his blog Digiconomist, and which used Nvidia GPUs – the gold standard of AI hardware – to estimate the industry’s global energy consumption. As De Vries explains in commentary published in Joule last year, Nvidia accounted for about 95 percent of sales in the AI ​​market. The company also releases energy specifications for its hardware and sales forecasts.

By combining this data, De Vries calculates that the AI ​​sector could consume between 85 and 134 terawatt hours annually by 2027. That is approximately the same as the annual energy needs of De Vries’ home country, the Netherlands.

“You are talking about AI electricity consumption that may amount to half a percent of global electricity consumption in 2027,” says De Vries. The edge. “I think that’s a pretty significant number.”

A recent report from the International Energy Agency offered similar estimates, suggesting that electricity consumption by data centers will increase significantly in the near future thanks to the demands of AI and cryptocurrency. The agency says current data center energy consumption is around 460 terawatt hours in 2022 and could increase to between 620 and 1,050 TWh by 2026 – equivalent to the energy needs of Sweden or Germany respectively.

But De Vries says it is important to put these figures in context. He notes that data center energy consumption has been fairly stable between 2010 and 2018, accounting for about 1 to 2 percent of global consumption. (And when we say “data centers” here, we mean everything that’s part of “the Internet”: from companies’ internal servers to all the apps you can’t use offline on your smartphone.) Demand has certainly increased during this period . says de Vries, but the hardware became more efficient and thus compensated for the increase.

His fear is that things could be different for AI, precisely because of the trend among companies to simply deploy larger models and more data for every task. “That is a really deadly dynamic for efficiency,” says De Vries. “Because it creates a natural incentive for people to just add more computing resources, and as models or hardware become more efficient, people will make those models even bigger than before.”

The question of whether efficiency gains will offset rising demand and usage is impossible to answer. Like Luccioni, De Vries laments the lack of available data, but says the world cannot simply ignore the situation. “It’s been quite a task to figure out which way this is going and it’s certainly not a perfect number,” he says. “But it is enough basis to issue a warning.”

Some companies involved in AI argue that the technology itself could help with these problems. Speaking on behalf of Microsoft, Priest said AI “will be a powerful tool to advance sustainability solutions,” and emphasized that Microsoft is working to “achieve sustainability goals of carbon negative, water positive and zero waste by 2030.”

But one company’s objectives can never capture the entire industry-wide demand. Other approaches may be necessary.

Luccioni says she would like to see companies introduce energy star ratings for AI models so that consumers can compare energy efficiency in the same way they do for appliances. For De Vries, our approach should be more fundamental: should we use AI for certain tasks at all? “Because given all the limitations that AI has, it’s probably not going to be the right solution in a lot of places, and we’re going to waste a lot of time and resources trying to figure that out the hard way,” he says.

Leave a Reply

Your email address will not be published. Required fields are marked *