Power From The People
The next advances in AI will be constrained by availability of power, not chip technology.
Anyone following AI developments will have a bad case of motion-sickness. Not just new products, updates and improvements but apparent step-changes bouncing us around as we sit in the back seats wondering where we are going, wondering if the driver knows how to drive and wondering if there is indeed a driver at all.
The destination is unknown, but there is a consensus that who ever gets there first will have a huge advantage.
At a recent business forum on trade-relations one point stuck - that President Trump’s multi-azimuth and apparently chaotic policies have one coherent theme - which unsurprisingly is - competition with China. MAGA is more than a slogan, it is the recognition that you can’t be a (the?) global super power if you import energy, if you can’t manufacture anything and if all your goods have potential security flaws when they rely on components manufactured by your adversary.
Within this world-view, after energy, the most important by far is chip manufacturing because of their role in *everything*, as well as their role in the cutting-edge of AI development. Currently 60% of the world’s semiconductors are made in Taiwan and that number increases to over 80% for latest generation chips.
Reshoring the manufacturing will be difficult, but not impossible. The ability to stay at the bleeding edge is a question of scale - enormous profits get ploughed back into R&D - making the incumbents strong, and new entry hard. The big get stronger (TSMC, Nvidia) , anyone who hesitates or has a bad step (Intel) gets left behind.
TSMC (Taiwan Semiconductor Manufacturing Company) is a pure-play foundry. It’s a manufacturer that builds chips designed by other companies. TSMC doesn’t create its own chip designs or brands; instead, it takes blueprints from clients—like Nvidia, Apple, or AMD—and turns them into physical silicon using its advanced fabrication plants (fabs). Its strength lies in cutting-edge manufacturing processes (e.g., 3nm technology), massive production capacity, and precision. Think of TSMC as the factory that powers the chip industry.
Nvidia, on the other hand, is a fabless semiconductor company. It designs chips—most famously GPUs (graphics processing units)—but doesn’t own factories to make them. Nvidia’s expertise is in architecture and software, creating products like the GeForce graphics cards for gaming or the A100 chips for AI and data centers. It outsources the actual production to foundries like TSMC. Nvidia’s value comes from innovation in chip design and the software ecosystem (like CUDA) that makes its hardware powerful.
The Chinese company BYD may well be the first big electronics company (best known for its EVs) to develop its own inhouse chip fabrication plants (“fabs”).
Currently there is a “gap” with the Chinese dominating the manufacturing of N-1 generation of chips - which get used in everything - and the latest N generation being dominated by TSMC in Taiwan. Much of Trump’s MAGA is focused on competition with China - and a big part of that will be in the AI “arms race”. So what are the limiting factors?
Limiting.
Manufacturing is hard - really hard - but this is solvable, albeit not overnight but the wheels are in motion.
R&D - the US and its allies are still at the forefront although it is expected that the gap will close.
Computation power - the Chinese are very good at building big (see BYD below) - so we are likely to see big AI “data center” clusters being built in both countries.
Power - this is the one to watch. According to the podcast linked below, as we progress from Chat-GPT type queries to AGI1 tasks, so the need for power will increase by orders of magnitude. It is unclear how this demand will be met.
As we progress from google-searches to Chat-GPT type queries to AGI tasks, so the need for power will increase by orders of magnitude.
In her recent blog North to Canada Meredith Angwin pointed out the tension between residential and industrial needs. Quebec Hydro is a provincial utility (electricity, despite the name) and it manages demand - recognizing that retail electricity is direct to voters and that businesses that want to avail themselves of Quebec’s not-so-abundant green hydro-electricity must wait in line. Any new business needing more than 5MW needs special permission - and refusals have prevented some businesses from setting up in Quebec.
As AI advances and its power needs multiple, this kind of tension may become the norm.
The Balance of Power
Now if you want to make yourself feel insignificant, out-of-the-loop and lets face it… old - have a watch/listen to this. Most podcasts are characterised by lack of information density - this one could have been a bit shorter (its about 4.5hrs long!) but it is certainly not lacking in depth or detail….
Dylan Patel is the founder of SemiAnalysis, a research & analysis company specializing in semiconductors, GPUs, CPUs, and AI hardware. Nathan Lambert is a research scientist at the Allen Institute for AI (Ai2) and the author of a blog on AI called Interconnects.
My summary - a short version of the discussion below is this.
Simply replacing current Google searches with GTP-3 queries would be “physically impossible to implement” (ie would take more computing power and indeed more energy than currently available.)
Chat queries of this nature cost a few cents, AGI queries are 1000x to 10,000x more intensive and would cost $5-$20 per interaction - and I assume those figures use today’s electricity prices - with no effect of a supply squeeze.
The implication is that the AGI race will be kicked off by chip advances, but limited by access to cheap / available power.
With the best will in the world, the ability to ramp-up energy production will not keep up with needs.
Given the strategic importance of this one has to wonder at what point the balance between retail use and strategic use gets stressed. Will we see electricity rationing for households and businesses to help maintain an edge in AGI advances?
Extracts of the Podcast Transcript
emphasis added.
Dylan Patel(01:14:08) I think to some extent, we have capabilities that hit a certain point where any one person could say, “Oh, okay, if I can leverage those capabilities for X amount of time, this is AGI, call it ’27, ’28.” But then the cost of actually operating that capability-
Nathan Lambert(01:14:23) Yeah, this was going to be my point.
Dylan Patel(01:14:24) … is so, so extreme that no one can actually deploy it at scale en masse to actually completely revolutionize the economy on a snap of a finger. So I don’t think it will be a snap of the finger moment.
Nathan Lambert(01:14:35) It’s a physical constraint [inaudible 01:14:37].
Dylan Patel(01:14:36) Rather, it’ll be a, “Oh, the capabilities are here, but I can’t deploy it everywhere.” And so one simple example, going back sort of to 2023 was when being when GPT-4 came out, everyone was freaking out about search. Perplexity came out. If you did the cost on like, hey, implementing GPT-3 into every Google search, it was like, oh, okay, this is just physically impossible to implement. And as we step forward to going back to the test-time compute thing, a query for… You ask ChatGPT a question, it costs cents for their most capable model of Chat to get a query back. To solve an AGI problem though costs 5 to 20 bucks, and this is in-
Dylan Patel(01:15:19) This is 1,000, 10,000 X factor difference in cost to respond to a query versus do a task. And the task of AGI is not like it’s like… It’s simple, to some extent, but it’s also like, what are the tasks that we want… Okay, AGI, “What we have today”, can do AGI. Three years from now, it can do much more complicated problems, but the cost is going to be measured in thousands and thousands and hundreds of thousands of dollars of GPU time, and there just won’t be enough power, GPUs, infrastructure to operate this and therefore shift everything in the world on the snap the finger.
(01:20:02) Yeah. I mean, many of them are Chinese people who are moving to America, and that’s great. That’s exactly what we want. But that talent is one aspect, but I don’t think that’s one that is a measurable advantage for the US or not. It truly is just whether or not compute. Now, even on the compute side, when we look at chips versus data centers, China has the unprecedented ability to build ridiculous sums of power. Clockwork. They’re always building more and more power. They’ve got steel mills that individually are the size of the entire US industry. And they’ve got aluminum mills that consume gigawatts and gigawatts of power. And when we talk about what’s the biggest data center, OpenAI made this huge thing about Stargate, their announcement there, once it’s fully built out in a few years, it’ll be two gigawatts of power. And this is still smaller than the largest industrial facilities in China. China, if they wanted to build the largest data center in the world, if they had access to the chips, could. So it’s just a question of when, not if.
(01:21:18) Chips are a little bit more specialized. I’m specifically referring to the data centers. Fabs take huge amounts of power, don’t get me wrong. That’s not necessarily the gating factor there. The gating factor on how fast people can build the largest clusters today in the US is power. Now, it could be power generation, power transmission, substations, and all these sorts of transformers and all these things building the data center. These are all constraints on the US industry’s ability to build larger and larger training systems, as well as deploying more and more inference compute.
AGI (Artificial General Intelligence) is a hypothetical next level of AI that can understand, learn, and perform any intellectual task a human can, across diverse domains, without being limited to specific functions. It’s the “general-purpose” dream of AI, still unrealized.
👋