At that point in history, combustion engines and cars had been around for almost four decades, and engines had become good enough in the previous decade to enable building aircraft. Also imagine a Ford engineer at that conference forecasting engine progress that would soon make it practical for widespread air travel. (Yes, a Ford engineer because that company pioneered not only aviation engines, but one of the first practical passenger aircraft, the 1925 TriMotor.i\u201d And imagine if that engineer had suggested that global air traffic would become so common that aviation in the foreseeable future would consume nearly as much energy as the entire U.S. used in 1925 for all purposes. A year later, 1926, saw the launch of America\u2019s first regularly scheduled, year-round commercial passenger service. The rest, as they say, is history. Fast forward to an actual conference in 2022\u2014one that took place several months before the November unveiling of ChatGPT\u2014where the CTO of AMD, a world-leading AI chipmaker, talked about how powerful AI engines have become and how fast they were getting adopted. He also showed a graph forecasting that by 2040, AI would consume roughly as much energy as the U.S. does today for all purposes.ii If those trends continued, AI would end up gobbling up most of the world\u2019s energy supplies. Of course, the trends won\u2019t continue that way, but that doesn\u2019t change the fact that, as another engineer at AI chipmaker ARM said at that conference: \u201cThe compute demand of [AI] neural networks is insatiable.\u201d And the growth of AI is still in the early days, equivalent to aviation circa 1925 or, in computer-history terms, to the pre-desktop 1980s era of mainframes.<\/p>\n\n\n\n
Inference as Efficient Energy Hog<\/h1>\n\n\n\n
It\u2019s not news to the computer engineering community that AI has a voracious energy appetite. AI-driven \u201cinference,\u201d rather than conventional \u201ccalculation,\u201d is the most power-intensive use of silicon yet created.iii That reality is starting to leak out because of the popularity of today\u2019s first-generation AI. We see headlines about AI having a \u201cbooming \u2026 carbon footprint\u201div and that it \u201cguzzles energy.\u201dv<\/p>\n\n\n\n
Consider, for example, the results of one analysis of what it takes to just build, never mind operate, a<\/p>\n\n\n\n
modest AI tool for one application, i.e., the equivalent of the energy used to build an aircraft, not fly it. The analysis found that the training phase\u2014training is how an AI tool is built\u2014consumed more electricity than the average home does in 10 years.vi A different analysis looked at what it likely took to build a bigger AI tool, say to train ChatGPT, finding that it used as much electricity as an average home in a century.vii Of course, like automobiles and aircraft, energy is also used to operate AI, something engineers call \u201cinference,\u201d to, say, recognize an image or an object, or give advice or make a decision, etc. Energy use by inference is likely tenfold or more greater than for training.viii As one researcher put it, \u201cit\u2019s going to be bananas.\u201d<\/p>\n\n\n\n There are few intersections of the world of bits and atoms\u2014of software and hardware\u2014that so dramatically epitomize the inescapable realities of the physics of energy, and the challenges in guessing future behaviors and thus energy demands. No one can guess the number of important, or trivial, things where we will want to use AI. And, in fact, for most people in the (lucky) wealthy nations, most energy is used for things other than mere survival. Aviation, for example, is dominated by citizens traveling for fun, vacations, to see family; business travelers account for well under 20 percent of global air-passenger miles (even less now, during the recovery period from global lockdowns). However, unlike aircraft, which are specialized machines used to move goods and people, AI is a universal tool, a \u201cgeneral purpose technology\u201d in the language of economists, and thus has potential applications in everything, everywhere. It\u2019s far harder to forecast uses of general-purpose technologies.<\/p>\n\n\n\n AI tools will be put to work for much more than just fine-turning advertising, or performing social media tricks, or creating \u201cdeepfakes\u201d to spoof hapless citizens, or making self-driving cars (eventually) possible. AI\u2019s power and promise, as its practitioners know, are leading to the potential for such applications as hyper-realistic vehicle crash-testing, or monitoring and planning ground and air-traffic flows, or truly useful weather forecasting. The most profound applications are to be found in basic discovery wherein AI-infused supercomputers plumb the depths of nature and simulate molecular biology \u201cin silico<\/em>\u201d (an actual term), instead of in humans in order to both accelerate discovery and even, eventually, to test drugs. The number and nature of potential applications for AI is essentially unlimited.<\/p>\n\n\n\n Analysts have pointed out that the compute power\u2014 and derivatively energy\u2014devoted to machine learning has been doubling every several months.ix Last year, Facebook noted that AI was a key driver causing a one-year doubling in its datacenter power use.x And this year, Microsoft reported a 34 percent \u201cspike\u201d in water used to cool its datacenters, an indirect even if unstated measure of energy use.xi It\u2019s the equivalent of measuring the flow of water to cool a combustion engine rather than counting gallons of fuel burned. The need for cooling comes from the energy use.<\/p>\n\n\n\n It is an open secret that AI will drive a massive infrastructure buildout. As a Google VP observed, deployment of AI is \u201creally a phase change in terms of how we look at infrastructure.\u201d One article\u2019s headline captured the reality: \u201cThe AI Boom Is here. The Cloud May Not Be Ready.\u201dxii Every computer vendor, chipmaker, software maker and IT service provider is adding or expanding offerings that entail AI. It is a silicon gold rush last matched in enthusiasm and velocity during the great disruption in information systems of the 1990s with the acceleration from mainframes to desktops and handhelds. AI enthusiasm is also seen in the stock market where, odds are, history will repeat as well: a boom, a bust and then the long boom. It\u2019s the coming long boom that has implications for forecasting energy demands from AI.<\/p>\n\n\n\n To continue with our analogy, while infrastructure growth points to potential for future energy use, it doesn\u2019t predict actual outcomes any more than counting highway or runway miles is predictive of fuel use, except that the infrastructure is what enables the fuel to be used. Future historians will see today\u2019s Cloud infrastructure as analogous to the 1920s stage of transportation infrastructure in the pre- superhighway days, also a time of grass runways. Even before we see what the next phase of silicon evolution brings\u2014the kinds of services and social changes that will echo those brought by the automobile and aviation\u2014it\u2019s possible to have some idea of the scale of energy demand that will bring by considering the current state of today\u2019s information infrastructure.<\/p>\n\n\n\n Unlike automobiles and aircraft though, the energy used by digital engines is hidden from plain sight inside thousands of nondescript warehouse-like datacenters. There we find, in each one of them, thousands of refrigerator-sized racks of silicon machines. Each such rack burns more electricity annually than 15 homes, that\u2019s before the racks are filled with AI silicon.<\/p>\n\n\n\n Datacenters, in square footage terms, are the skyscrapers of the modern era, except that there are far more of the former and many more being built. And each square foot of datacenter uses 100 times the power of a square foot of skyscraper\u2014again, before the infusion of AI.<\/p>\n\n\n\n Datacenters of course are only useful if connected to people and other machines and vice versa. The world today has over a billion miles of information highways comprised of glass cables along with four million cell towers that forge an invisible, virtual highway system that is effectively another 100 billion miles long.xiii Machinery for transporting bits uses energy just as it does in the world of atoms. And the convenience of wireless networks comes with an energy cost up to 10 times more energy per byte,xiv not unlike a similar principle of physics that makes flying more fuel-intensive than driving. Flight-shamers might take note: In energy-equivalent terms, today\u2019s global digital infrastructure already uses roughly 3 billion barrels of oil annually, rivaling the energy used by global aviation. And that number is based on data that is a half-dozen years old. Since then, there\u2019s been a dramatic acceleration in datacenter spendingxv on hardwarexvi and buildingsxvii along with a huge jumpxviii in the power density of that hardware, all of that, again, before the acceleration as AI is added to that infrastructure. A single, simple AI-driven query on the Internet can entail over fourfold the energy use of a conventional query.xix<\/p>\n\n\n\n It\u2019s not that digital firms are energy wastrels. In fact, silicon engineers have achieved epic, exponential gains in efficiency. But overall demand for logic has grown at an even faster, blistering pace. You can \u201ctake to the bank\u201d that history will repeat here too: demand for AI services will grow faster than improvements in AI energy efficiency. The cloud is already the world\u2019s biggest infrastructure and seeing it expand yet by several-fold, or more, would be entirely in keeping with historical precedent.<\/p>\n\n\n\n Nonetheless, the forecasters and pundits who are preoccupied with reducing society\u2019s energy appetite always offer energy efficiencyxx as a \u201csolution\u201d to the energy \u201cproblem.\u201d They have it backwards. Efficiency gains have always been the engine that drives a growth, not a decrease, in overall energy use. It\u2019s a feature, not a bug, in technology progress, and one that is most especially true in digital domains. This seeming contradiction has been called Jevons Paradox after the British economist William Stanley Jevons who first codified the economic phenomenon of efficiency in a seminal paper published back in 1865. That paper was focused on the claim, at that time, that England would run out of coal given the demands for that fuel coming from a growing economy, growth that itself was caused by the fuel of industrialization. The solution offered by experts at that time was to make coal engines more efficient.<\/p>\n\n\n\n Jevons, however, pointed out that improvements in engine efficiency\u2014i.e., using less coal per unit of output\u2014would cause more, not less, overall coal consumption. Thus, the ostensible paradox: \u201cIt<\/p>\n\n\n\n is wholly a confusion of ideas to suppose that the [efficient] use of fuel is equivalent to a diminished consumption \u2026. new modes of [efficiency] will lead to an increase of consumption.\u201dxxi Some modern economists call this the \u201crebound effect.\u201dxxii It\u2019s not a rebound as much as it\u2019s the purpose of efficiency.<\/p>\n\n\n\n Put differently: the purpose of improved efficiency in the real world, as opposed to the policy world, is to make it possible for the benefits from a machine or a process to become cheaper and available to more people. For nearly all things for all of history, rising demand for the energy-enabled services outstrips the efficiency gains. The result has been a net gain in consumption.<\/p>\n\n\n\n If affordable steam engines had remained as inefficient as when first invented, they would never have proliferated, nor would the attendant economic gains and associated rise in coal demand have happened. The same is true for modern combustion engines. Today\u2019s aircraft, for example, are three times more energy efficient than the first commercial passenger jets. That efficiency didn\u2019t \u201csave\u201d fuel but instead propelled a four-fold rise in aviation energy use.xxiii<\/p>\n\n\n\n The same dynamic is at play with today\u2019s digital engines, the driving force of the 21st-century economy. In fact, the microprocessor represents the purest example of the Jevon\u2019s paradox. Over the past 60 years, the energy efficiency of logic engines has improved by over one billion <\/em>fold.xxiv Nothing close to such gains are possible with mechanical and energy machines that occupy the world of atoms.<\/p>\n\n\n\n Consider the implications from 1980, the Apple II era. A single iPhone at 1980 energy-efficiency would require as much power as a Manhattan office building. Similarly, a single datacenter at circa <\/em>1980 efficiency would require as much power as the entire U.S. electrical grid. But because <\/em>of efficiency gains, the world today has billions of smartphones and thousands of datacenters. We can only hope and dream that the efficiency of AI engines advances similarly.<\/p>\n\n\n\n A leading-edge AI chip today delivers more image processing capability than a supercomputer could just two decades ago. In a sign of our times, last year the silicon start-up Cerebras introduced a kind of Godzilla-class AI chip the size of an entire silicon wafer (think, medium pizza) with more than two trillion transistors and a 15-kilowatt power appetite. That\u2019s more peak power than used by three houses. But it offers more than a ten-fold gain in efficiency over the best AI chips. Competitors will follow. That\u2019s why the market for AI chips is forecastxxv to dominate semiconductor growth and explode some 700 percent in the next five years alone.<\/p>\n\n\n\n Of course, the Jevons paradox breaks down in a microeconomic sense, and for specific products or services. Demand and growth can saturate in a (wealthy) society when limits are hit for specific items regardless of gains in efficiency, e.g., the amount of food a person can eat, or the miles-per-day one is willing to spend driving, or the number of refrigerators or light bulbs per household, etc. But for such things, we\u2019re a long way from saturation for over two-thirds of the world\u2019s citizens. Billions of people in the world have yet to become wealthy enough to buy even their first car or air conditioner, never mind use an AI-infused product or service.<\/p>\n\n\n\n But one can understand why the \u201cgreen AI\u201d community is alarmed over what will come. Even before the Age of AI is in full swing, today\u2019s digital infrastructure already uses twice as much electricity as the entire country of Japan. We await the Cloud forecasts that incorporate the energy impact of the AI gold rush. As Deep Jariwala, a professor of electrical engineering at the University of Pennsylvania provocatively put it: \u201cBy now, it should be clear that we have an 800-pound gorilla in the room; our computers and other devices are becoming insatiable energy beasts that we continue to feed.\u201dxxvi Prof. Jariwala went on to caution: \u201cThat\u2019s not to say AI and advancing it needs to stop because it\u2019s incredibly useful for important applications like accelerating the discovery of therapeutics.\u201d There\u2019s little to no risk that governments will or can directly throttle AI development (though some have proposed as much). Ironically though, government energy policies could make AI expensive enough to slow deployment.<\/p>\n\n\n\n Consider a simple arithmetical reality: In a low-cost state, training a high-end AI requires buying about $100,000 in electricity, but you\u2019d spend over $400,000 in California. And the training phase for many applications is necessarily repeated as new data and information are accumulated. One can imagine, as some have proposed, doing the training at remote locations where electricity is cheap. That\u2019s the equivalent of, say, buying energy-intensive aluminum to build airplanes from places where energy is cheap (China, using its coal-fired grid, produces 60 percentxxvii of the world\u2019s aluminum). But most uses for AI require operating it and fueling it locally and in real-time, much as is the case to operate an airplane. Wealthy people will be able to afford the benefits from higher cost AI services, but that trend will only further widen the \u201cdigital divide,\u201d wherein lower income households are increasingly left behind.<\/p>\n\n\n\n There is some irony in fact that many in the tech community have joined with the \u201cenergy transition\u201d lobby to promote the expansion of power plants that are not only increasing the cost of electricity but making it more difficult to deliver it when markets need it. Despite the mantra that wind and solar are cheaper than conventional power plants, the data show that, in every state and every country, the deployment of more episodic power leads to rising electricity costs. The reason for that, in essence, comes from the cost of ensuring that power is delivered whenever markets and people need it, and not when nature permits it. It doesn\u2019t matter whether the reliability is achieved by maintaining what amounts to a duplicate, under-utilized existing grid (Germany\u2019s solution), or by using use more transmission lines and more storage. The results lead to far higher costs.<\/p>\n\n\n\n Policies that lead to higher costs and lower reliability for electricity will be increasingly in collision with the emerging demands for an AI-infused future. That may be the most interesting and challenging intersection of the worlds of bits and atoms. \u25d9<\/p>\n\n\n\n I https:\/\/simpleflying.com\/henry-ford-aviation-pioneer-story\/<\/a><\/p>\n\n\n\n ii https:\/\/semiengineering.com\/ai-power-consumption-exploding\/<\/a><\/p>\n\n\n\n iii https:\/\/www.technologyreview.com\/2019\/11\/11\/132004\/the-computing-%20power-needed-to-train-ai-is-now-rising-%20seven-times-faster-than-ever-before\/<\/a><\/p>\n\n\n\n iv https:\/\/www.bloomberg.com\/news\/articles\/2023-03-09\/how-much-energy-do-ai-and-chatgpt-use-no-one-knows-for-sure?sref=lHqvUqWg#xj4y7vzkg<\/a><\/p>\n\n\n\n v https:\/\/www.wsj.com\/articles\/artificial-intelligence-can-make-companies-greener-but-it-also-guzzles-energy-7c7b678<\/a><\/p>\n\n\n\n vi https:\/\/arxiv.org\/abs\/1906.02243<\/a><\/p>\n\n\n\n vii https:\/\/www.scientificamerican.com\/article\/a-computer-scientist-breaks- down-generative-ais-hefty-carbon- footprint\/<\/a><\/p>\n\n\n\n viii https:\/\/xcorr.net\/2023\/04\/08\/how-much-energy-does-chatgpt-use\/<\/a><\/p>\n\n\n\n ix https:\/\/arxiv.org\/pdf\/1907.10597.pdf<\/a><\/p>\n\n\n\n x https:\/\/www.eetimes.com\/qualcomm-targets-ai-inferencing-in-the-cloud\/<\/a><\/p>\n\n\n\n xi https:\/\/fortune.com\/2023\/09\/09\/ai-chatgpt-usage-fuels-spike-in-microsoft- water-consumption\/<\/a><\/p>\n\n\n\n xii https:\/\/www.wsj.com\/articles\/the-ai-boom-is-here-the-cloud-may-not-be- ready-1a51724<\/a>d<\/p>\n\n\n\n xiii https:\/\/www.researchgate.net\/publication\/318461301_Green_and_Sustainable_Cellular_Base_Stations_An_Overview_and_Future_Research_Directions<\/a><\/p>\n\n\n\n xiv https:\/\/www.researchgate.net\/publication\/228774201_Power_Consumption_in_Telecommunication_Networks_Overview_and_Reduction_Strategies<\/a><\/p>\n\n\n\n xv https:\/\/www.us.jll.com\/en\/trends-and-insights\/research\/data-center-outlook?utm_source=Twitter&utm_medium=Social&utm_content=2631630870&utm_campaign=COMPANY+NE%20WSnon-specific<\/a> <\/p>\n\n\n\n xvi https:\/\/www.nextplatform.com\/2019\/12\/09\/datacenters-are-hungry-for-servers-again\/<\/a><\/p>\n\n\n\n xvii https:\/\/www.construction.com\/construction-news\/<\/a><\/p>\n\n\n\n xviii https:\/\/www.datacenterfrontier.com\/cloud\/article\/11429232\/the-eight-trends-that-will-shape-the-data-center-industry-in-2020<\/a><\/p>\n\n\n\n xix https:\/\/www.nytimes.com\/2023\/04\/16\/technology\/google-search-engine-ai.html<\/a><\/p>\n\n\n\n xx <\/sup>https:\/\/www.scientificamerican.com\/article\/a-computer-scientist-breaks-down-generative-ais-hefty-carbon-footprint\/<\/a><\/sup><\/p>\n\n\n\n xxi Jevons, William Stanley, The Coal Question, <\/em>Macmillan and Co., 1865. https:\/\/www.econlib.org\/library\/YPDBooks\/Jevons\/jvnCQ.html<\/a><\/p>\n\n\n\n xxii Nordhaus, Ted. \u201cThe Energy Rebound Battle.\u201d Issues in Science and Technology, July 28, 2020. https:\/\/issues.org\/the-energy-rebound-battle\/<\/a><\/p>\n\n\n\n xxiii Larkin, Alice, Kevin Anderson, and Paul Peeters. \u201cAir Transport, Climate<\/p>\n\n\n\n Change and Tourism.\u201d Tourism and Hospitality Planning; Development 6, no. 1<\/p>\n\n\n\n (April 2009): 7\u201320. https:\/\/doi.org\/10.1080\/14790530902847012<\/a>.<\/p>\n\n\n\n xxiv Roser, Max, and Hannah Ritchie. \u201cTechnological Progress.\u201d Our World in Data, May 11, 2013. https:\/\/ourworldindata.org\/technological-progress<\/a><\/p>\n\n\n\n xxv https:\/\/www.eetimes.com\/iot-was-interesting-but-follow-the-money-to-ai-chips\/?image_number=1<\/a><\/p>\n\n\n\n![\"\"](\"https:\/\/dda.ndus.edu\/ddreview\/wp-content\/uploads\/sites\/18\/2023\/11\/AdobeStock_462112399-1024x683.jpeg\")
AI tools like ChatGPT are costly to train, both financially, due to the cost of hardware and electricity, and environmentally, due to the carbon footprint required to fuel data processing hardware.<\/figcaption><\/figure>\n\n\n\nAI as General Purpose Tech<\/h1>\n\n\n\n
Datacenters & Infrastructure Buildout<\/h1>\n\n\n\n
![\"\"](\"https:\/\/dda.ndus.edu\/ddreview\/wp-content\/uploads\/sites\/18\/2023\/11\/AdobeStock_272099186-1024x683.jpg\")
Jevons Paradox<\/h1>\n\n\n\n
Green vs AI<\/h1>\n\n\n\n
Transition Irony<\/h2>\n\n\n\n
![\"\"](\"https:\/\/dda.ndus.edu\/ddreview\/wp-content\/uploads\/sites\/18\/2023\/11\/cs-wafer-keyboard-comparison-1024x577.jpg\")
The Cerebras \u201cwafer-scale engine,\u201d measuring 8.5 inches on a side, is the world\u2019s largest chip, and is dedicated to computations prevalent in machine-learning forms of artificial intelligence. Photograph \/ Cerebras Systems.<\/figcaption><\/figure>\n\n\n\n
\n\n\n\nReferences<\/h2>\n\n\n\n