--:-- Erik Torenberg: NVIDIA is going to have better networking than you. They're going to have better HBM. They're going to have better process node. They're going to come to market faster. They're going to be able to ramp faster. They're going to have better negotiations with TSMC, SK Hynix, the memory and silicon side, or all the rack people, copper cables, everything. They're going to have better cost efficiency. So you can't just do the same thing as NVIDIA. You have to really leap forward in some other way. You have to be like 5X better.
--:-- [Intro music fades]
--:-- Erik Torenberg: Today, we're talking AI, hardware, chips, and the infrastructure powering the next wave of models with three people at the center of it all. Dylan Patel, founder and CEO of SemiAnalysis, one of the sharpest voices on chips, data centers, and the economics driving AI's explosive growth. Aaron Price-Ripe, general partner at A16Z, investing in the technologies and infrastructure shaping the future. Guido Appenzeller, partner at A16Z, with decades on the front lines of AI, cloud, and networking. From GPT-5's launch to NVIDIA's dominance, custom silicon, and the global race for compute, we're covering what's happening behind the scenes. Let's get into it.
--:-- Erik Torenberg: Dylan, welcome to the podcast.
--:-- Dylan Patel: Thank you for having me.
--:-- Erik Torenberg: We've been trying to get you for a while. You're a busy man, but it worked out. Guido, why don't you introduce why we're so excited to have Dylan on the podcast and what we're excited to discuss?
--:-- Guido Appenzeller: I think, Dylan, you've done an exceptional job covering what's happening in the AI hardware space, the AI semi-space, and now more and more the data center space as well. Currently, the most valuable company on the planet is an AI semi-company. The biggest IPO so far in AI was an AI cloud company. This is where it's happening. In any gold rush, the early days are the picks and shovels that make money. I think that's the stage we're in. So super excited to have you here today.
--:-- Dylan Patel: Awesome. Thank you. Happy to talk about my favorite topics.
--:-- Erik Torenberg: Amazing. Well, maybe let's start with GPT-5. We just had some of the research team from Christina and Isabella on here last week. You said it was disappointing. Why don't you share your reactions, what capabilities you were hoping to see, or overall?
--:-- Dylan Patel: I think it depends on what tier of user you are. If you're just using GPT-5 and before you were a $20 or $200 a month subscriber, you no longer have access to 4.5, which in my opinion is still a better pre-trained model for certain things. Or you no longer have access to O3, which would think for 30 seconds on average. Whereas GPT-5, even when you're using thinking, only thinks for like 5 to 10 seconds on average. That's an interesting phenomenon.
--:-- Basically, GPT-5 is not spending more compute per se. The model did get a little bit better on a vanilla basis. 4.0 to 5 is actually quite a bit better. But when you think about the curve of intelligence, the more compute you spend, the better the model gets. That's whether it's a bigger model—which GPT-5 isn't, you can see it's roughly the same size—or you think more.
--:-- OpenAI's first thinking models, the first few generations of O1 and O3, would think for a long time and waste a lot of tokens. When you look at Anthropic's thinking models, even when you put them in thinking mode, they think a lot less to get to the same results or better results. OpenAI optimized a lot. The silliest one I asked O3 once was, "Is pork red meat or white meat?" It thought for like 48 seconds. I was like, what are you doing? Just tell me the answer.
--:-- The nice thing is that GPT-5 will think a lot less, even if you select thinking manually. More importantly, they have the auto functionality, the router, which lets them decide whether to route to the regular model, route to maybe mini if you're at rate limits, or route to thinking and how much to think. In general, the thinking model will think less. So there's less compute going into a power user's average query than before.
--:-- Guido Appenzeller: Isn't it even more interesting that OpenAI can control how much compute to allocate to you? If we're in a high load situation, maybe tune the router a little bit so it's less. I have no idea what they're doing behind the curtain. But there's this meme out there that all they did—which is a meme, it's not true—is take all three plus a couple of smaller models, put a router in front, and offer that at a lower blended price. I think there's a little bit of that. Cost suddenly matters, and they figured out a way to steer that.
--:-- Dylan Patel: Yeah. They talked about how they've been able to dramatically increase their infrastructure capacity. I was regularly using O3 or 4.5, and now I'm forced to use auto, which sometimes gives me the O3 equivalent thinking model, but sometimes gives me just a regular base, which sucks.
--:-- For the free user, it's actually quite interesting. The free user was not getting thinking models pretty much ever. In many cases, they just opened the website and asked their query. Now sometimes their query gets routed there, so they get a way better model. But now OpenAI can gracefully degrade them if they need to.
--:-- I think the router points to the future of OpenAI from a business perspective. Look at the model companies. Anthropic is fully focused on B2B, API, code, cloud code. OpenAI has that business too, but the majority of their revenue is consumer subscriptions. They have no way to upsell or make money off all the free users. In any other consumer app, the free user pays via ads. But that's not compatible with AI. It's a helpful assistant. You can't make the user's result worse by injecting ads. Banner ads don't really work in AI either. So how do you monetize them?
--:-- With the router, they're getting really close to figuring out how to monetize that user. The new CEO of applications launched a product at Shopify that was an agent for shopping. It immediately clicks: if the user asks a low value query like "Why is the sky blue?" just route them to mini. The model can answer perfectly fine. That's a chunk of queries. But if they ask, "What's the best DUI lawyer near me?" all of a sudden, you're in jail, you have one shot, you ask ChatGPT. Soon enough, it'll be able to contact all the lawyers in the area, figure out their results, search their court filings, book the best lawyer for you, or book an airplane ticket, and negotiate a cut as part of that.
--:-- Of course they're going to take a cut. This is a much better way of monetizing the free user. It's like Etsy—10% of their traffic now comes from chat. OpenAI makes nothing off of that, but they will soon. Partially that's because Amazon blocks chat, but there's a way to make money from shopping decisions, whether it's booking flights or looking for items. For those, you say, "Free user, I don't care. I'm going to send you to my best model. I'm going to send you to agents. I'm going to spend ungodly amounts of compute on you because I can make money off of this." But if it's a query like "Help me with my homework," I'll send you a decent model. I don't need to spend money on you. That's how OpenAI can finally make money off the free user. I think that's the biggest thing about the router.
--:-- Guido Appenzeller: This is super interesting. I think this is the first time we've seen a new model launch where cost is the headline item. So far, it was always about who has the smartest model, who has the highest MLU score. Now, people who use models for coding for eight hours a day are surprised that a large context window with the best model creates thousands of dollars of cost a month. So cost matters. The parade between cost and performance is the new benchmark for model competitiveness. Is that what we're seeing here?
--:-- Dylan Patel: Definitely. OpenAI said they doubled their rate limits for big amounts of users. They've dramatically increased the number of tokens they're serving from this launch, which effectively says this is an economic release. It probably also means the tokens aren't all cheaper.
--:-- The funniest thing about this whole cost thing is we've seen it in the code space. Cursor had to pull away the unlimited plan. They had a super expensive plan with unlimited rates, then a weekly rate limit, now hour-based rate limits. I saw the craziest thread on Twitter where this guy said he changed his sleep schedule modeled after how sailors in a bay solo sailing take power naps when they get to the right spots so they can still be safe. Because Anthropic had to put rate limits that are not just week-based but number of hours-based, he basically sleeps multiple times a day in small chunks just to maximize usage.
--:-- There's also a leaderboard on Reddit where people compete to see how many tokens they're using through their subscription. There's a dude spending $30,000 a month. I'm going to find some developer in India to do pair programming with so I can get the day cycle, he can get the night cycle, and we both can maximize the quota for the account.
--:-- Guido Appenzeller: Is that the future? It's clear people are taking advantage of the negative gross margin subscriptions that are offered. Anthropic probably makes a positive gross margin off of my subscription. I don't code enough. But plenty of people are definitely losing money. This puts more pressure on usage-based pricing. If you have an underlying commodity that you're reselling that is that large a part of your cost of goods, you need to go to usage-based pricing.
--:-- Erik Torenberg: How much do you think the customer capture and stickiness for these code products is? Once you use an IDE, once you integrate one of the CLI products, how sticky is it?
--:-- Dylan Patel: That is a billion-dollar question. That's a very conservative estimate.
--:-- Guido Appenzeller: Andrew Parthy has this great slide where he says if you're building an agentic system today, fundamentally it's a loop. Half of the loop is the model thinking and trying to do something. The other half is the user verifying what the agent did, providing feedback, and trying to steer it in the right direction because we can't run forever. One half is the model provider trying to build the best model. The other half is about designing the best possible UI to enable the user to give feedback. There's value in that. So there's a certain amount of stickiness.
--:-- Take code editing. How can I most easily visualize what the code changes are? What they impact? Which files? For small changes, get very quick feedback versus complex ones get complex feedback? Some tools actually draw diagrams of what they do. I think this will be the battle. There's stickiness in that. How much exactly?
--:-- Dylan Patel: In that sense, people should be doing subscriptions to get people locked in instead of moving to usage-based pricing.
--:-- Guido Appenzeller: I think it's the customers that don't want usage-based pricing because it's so hard to guarantee. They want guarantees and are willing to commit to pretty high spend to avoid usage-based pricing. I think it's the model companies that want usage-based pricing. With consumers, it's frankly very hard not to have usage-based pricing because the variability is so massive. If it's us coding versus somebody who does this as their full-time job, you have a factor of 20 difference in usage. That costs a lot of money. For enterprises, we could see more flat fee pricing because we can average it out more. You know in a general sense how many hours a day a developer is programming. The vibe quotas are harder.
--:-- Erik Torenberg: Before we leave OpenAI, I want to ask a broad question. If someone was sitting here and saying, "Dylan, I'll listen to anything you tell me to do, any advice you have, as long as it makes OpenAI more valuable," what would you tell them?
--:-- Dylan Patel: I would say immediately launch a method for you to input your credit card into ChatGPT and agree that for anything it agentically does for you, it'll take X cut. Then launch that product for shopping. Everyone knows that Anthropic, OpenAI, and all the other labs are buying RL environments of Amazon, Shopify, Etsy, all the different ways to shop on the internet, airline websites. Now, just integrate my calendar. "I want to fly there on Thursday. Make sure I don't miss a meeting. Cool, book it." Do that integration super well. Know my preferences on whether I like aisle or window. Just take a take rate. I think this will make them so much money the moment they launch it.
--:-- I think they're working on it already. I'd like to hear how Sam Altman thinks about it because he shifted his tone massively on ads over the last six months. He used to be like, "No way." Now he's like, "Maybe there's a way to do it without harming the user." I think this is how you monetize the free user. That's probably what I'd tell him.
--:-- Erik Torenberg: A whole line of questions around this. He's coming on the podcast in a few weeks. I want to shift to NVIDIA. NVIDIA is having a monster year. They're up almost 70%. What are the possible paths from here? How do you see it playing out?
--:-- Dylan Patel: It depends on how you bet on continued growth. You guys have a good vantage point of how fast revenue is growing for a lot of these companies, especially the code companies, but even many other applications. I think we can clearly see the demand side is accelerating. On the training side, the race is on. Google is upping hugely. Just look at OpenAI and Anthropic and the compute they have and are getting this year from Google and Amazon for Anthropic, and from Microsoft, CoreWeave, and Oracle for OpenAI. 30% of the chips are going to them, just those two companies.
--:-- But 70% of the stuff—who's making off with that? One third is ads, whether it be ByteDance, Meta, or many others doing ads. The rest of that one third of chips are coming from mostly uneconomic providers. It's not an obvious bet that they're going to keep raising bigger and bigger rounds.
--:-- We talked about coding earlier. The Quen Coder 3 model is actually super cheap if you're running it on-prem or in the cloud with all these inference libraries. So the question is how much does it keep growing? The first third—OpenAI and Anthropic lab spend—is definitely skyrocketing. The second third—ads—is going to grow, but not like crazy. There could be an inflection point with Gen AI ads. I know Meta has been experimenting with it a lot. I could be convinced there's going to be a huge inflection and take rate there, where you start showing me personalized ads. Every person in an ad looks like me, except slightly better so I feel better, and I'm like, "I want to buy it."
--:-- Guido Appenzeller: I have no idea how this is going to scale. But if you ask how much it could scale, how much value are we creating here? Can we create enough value to actually keep growing for a long time? If you take AI software development, we know we can easily get about 15% more productivity out of a developer.
--:-- Dylan Patel: I don't think that's right. I think it's way higher.
--:-- Guido Appenzeller: No, the straight classical enterprise GitHub Copilot deployment gives you about 15%. We can do much more than that.
--:-- Dylan Patel: But do you know how bad GitHub Copilot is? Look at their revenue ARR chart. It's so funny. Cloud Code in three months has surpassed that. Cursor easily surpassed that. Even companies like Replin and Windsurf slash Cognition are going to pass them. What's going on?
--:-- Guido Appenzeller: Let's assume we can get 100%. We can double the productivity of a developer. About 30 million developers worldwide, give or take. Let's say $100K value add per developer. That might be a little high worldwide—the US is low, but worldwide is high. So it's $3 trillion. We're probably building technology that adds $3 trillion of GDP value. In theory, we could put that into GPUs because that's the main cost factor here. Just from a coding model. Ignoring every other use case. At least in theory, the value generation is here to keep growing.
--:-- How that translates to the industry is much more complicated. We've already seen AI's value creation exceed the spend. There's the famous "$300 billion problem" or "$200 billion problem," now "$600 billion problem." Soon it'll be the "trillion problem." There's some reality to that, but it ignores that infrastructure spend today is accounting for five years of revenue, not one. And the revenue looks like this, not like a flat line.
--:-- Dylan Patel: The main thing is that AI is already generating more value than the spend. The value capture is broken. I legitimately believe OpenAI is not even capturing 10% of the value they've created in the world already just by usage of chat. The same applies to Anthropic, Cursor, and whoever else. The value capture is really broken.
--:-- Even internally, what we've been able to do with four devs in terms of automation—our spend on the Gemini API is absurdly low. Yet we go through every single permit and regulatory filing around every data center with AI. We take satellite photos of every data center, label our dataset, recognize what generators people are using, cooling towers, construction progress, substations. All this is automated and only possible because of Gen AI. We do it with very few developers. The value I generate by selling this data and consulting with it is so high, but the company making the API gets nothing out of it.
--:-- There's a value capture challenge here that far exceeds the creation. As you get models like GPT-5 or open source models continuing to drive costs down, value capture is just harder and harder for these companies because they're making 50% gross margin on inference, or less in many cases.
--:-- Guido Appenzeller: In so many words, you're saying we're getting commoditized and therefore you can't capture the value, and thus you should temper your expectations of how much you can spend on GPUs.
--:-- Dylan Patel: No, I think there are still ways to inflect hugely on value capture. I mentioned ads as a huge value capture. But that needs to happen before we see massive increase. The other thing is there's a lot of capital that hasn't been spent. The hyperscalers can still grow CapEx 20 to 30% next year from what they're doing this year. Companies like CoreWeave and Oracle, because they're tapping capital markets, can raise way more than 20 to 30% CapEx. Then you go down the list further and the largest infrastructure funds in the world, like Brookfield and Blackstone, are turning all their eyes to investing even more into AI infrastructure. The sovereign wealth funds of the world, like G42, the Norway one, or GIC in Singapore, have barely started touching AI.
--:-- There's a whole lot more CapEx that can come without it being economically motivated day one. Economically motivated CapEx can only grow so much, but there's so much other capital where it's not clear from a spreadsheet based on real business that you should spend this much. But people will because they believe. I believe, I think you believe, people believe this will generate profit, but there's no 100% certain way to argue it.
--:-- Erik Torenberg: How threatened is NVIDIA by custom silicon?
--:-- Dylan Patel: I think that's the biggest thing. When we look at orders from Google, Amazon, and Meta, their custom silicon—not Microsoft, their custom silicon kind of sucks—but the other three are really upping their orders massively over the last year. Amazon is making millions of Tranium. Google is making millions of TPUs. TPUs are clearly 100% utilized. Tranium is not there, but I think Amazon will figure out how to do that, and Anthropic will. I think that's the biggest threat to NVIDIA: people figure out how to use custom silicon more broadly.
--:-- If AI is concentrated, custom silicon will do better. That's not even talking about OpenAI's silicon team. If AI is really concentrated, custom silicon will do better. But if it gets dispersed broadly because there are all these open source models from China and open source software libraries from NVIDIA and China, making deployment costs rock bottom, then potentially...
--:-- Erik Torenberg: Hear me out here. If Google's TPU is able to compete with NVIDIA, in theory it could do it on the open market. NVIDIA is worth more than Google these days. Shouldn't Google start selling the chips to everyone? In theory, they should be able to achieve a higher market cap.
--:-- Dylan Patel: I absolutely think so. I think Google is even discussing it internally. It would require a big reorg of culture, of how Google Cloud works, how the TPU team works, how the JAX software team and XLA software teams work. I totally think they could. It would just take them shaking themselves pretty hard to be able to do it. But I totally think Google should sell TPUs externally. Not just renting, but physically.
--:-- Erik Torenberg: It's kind of funny if a side hobby in theory has a higher company value potential than your entire business, especially as you think about the degradation of search.
--:-- Dylan Patel: If you were to ask Sergey, "Do you think selling chips in front racks is more valuable than cloud or Gemini?" He'd be like, "No, no, no. Gemini is going to be worth way, way, way more. It's just not yet today." Today you say NVIDIA is the most valuable. It's a concentration thing. If the world is super concentrated in terms of customers, NVIDIA will not be the most valuable company in the world. But if it gets dispersed more and more—which arguably we're starting to see with open source models getting better and better, and the ease of deploying them getting better—then you could argue NVIDIA will remain the most valuable company in the world for a long period of time.
--:-- Guido Appenzeller: Historically, software has eaten the world in most markets. In early networking days, Cisco was the most valuable company on the planet for a while. It's no longer. The guys that build services on top, like Google, Amazon, or Meta, eventually eclipsed them.
--:-- Dylan Patel: Which is why NVIDIA is making all these software libraries. They're trying to commoditize inference. You guys don't even have an inference API provider investment, do you? With all kinds of model providers, but I'm talking about a pure API provider investment. I think I talked to one of the team members about why you guys didn't invest in a Together or a Fireworks. The argument was that just serving models alone without making them will be commoditized. You have some in the stable diffusion ecosystem, like with File. It's a little bit different dynamics there. They tend to make much more compound models than the LM folks.
--:-- You guys don't have one of those API investments because you think it'll get commoditized because of software NVIDIA is making, because VLM and HDLang—open source software coming out of Berkeley that now has its own environments and is supported by many—this being commoditized means API providers aren't necessarily worth a ton.
--:-- Erik Torenberg: Shifting gears, what about the silicon startups? What's your take on those? There's a ton of capital flowing into that. Probably billions being invested in chip startups.
--:-- Dylan Patel: For sure. Companies like Etched and Revos, Madax, and others have gotten the amount of funding they've had without even launching a chip. In the past, silicon companies would make money or raise money, but they would at least launch a chip before getting a big round. Etched and Revos have raised a lot of money without ever launching a chip publicly. It speaks to how capital intensive it is if you're building an accelerator with so many moving pieces.
--:-- There are like 10 different AI accelerator companies out there that are newish in the last few years. There's a lot more. Then there's the old guard that continues to raise money: Grok, Cerebris, Samanova, Tenztorin, and so on. Graphcore got bought out by SoftBank, and SoftBank is dumping money into that effort too. There's a lot of capital being invested to dispel NVIDIA's top position. But it becomes challenging. How do you beat NVIDIA?
--:-- Dylan Patel: Like the hyperscalers, I think, are kind of lucky in that they can do mostly the same thing as NVIDIA. They're a captive customer, which is themselves. They can just win on supply chain, using cheaper providers. It's a margin compression exercise, essentially.
--:-- Erik Torenberg: Yeah, and maybe for certain workloads, like Meta for recommendation systems, they can specialize more.
--:-- Dylan Patel: But for the most part, it's like, no, we're targeting the same workloads. We can just simplify supply chain or in-house a lot of it and compress margin, and it'll be fine. But in the case of these other companies, they don't have a captive customer. So now you have to contend with: either I can use some custom silicon provider who's going to take a margin on top, and that's going to compress what I can sell for, or I can try and in-house everything. But then it's like, this is really hard. I'm going to do all the software design, all the silicon design, build all this different IP, manage the supply chain on chips, on racks, on everything. It ends up being a huge effort in terms of team size. And in the end, NVIDIA makes a 75% gross margin. AMD sells their GPUs for 50% gross margin, and they have a hard time out-engineering NVIDIA, and they're great at engineering. They still take more silicon area, more memory to achieve the same performance, and they have to sell for less, so their margin gets compressed.
--:-- Guido Appenzeller: That makes sense. Look, I think historically, if you look at it, typically, new entrants in markets didn't win by marginally improving on something existing. That happens sometimes, but more likely, they jumped on some kind of disruptive technology leap. They had a different approach, different technology. Is that possible here? I mean, to some degree, maybe it's oversimplifying, but I think part of the reason why the transformer model won was because it runs so incredibly well on GPUs. A recurrent neural network is similarly performing, but it runs terribly on a GPU. So did we sort of pick the model for the architecture? And now it's hard to come up with an architecture that really...
--:-- Dylan Patel: Well, it's hardware-software co-design. There's all this hype about neuromorphic computing. Theoretically, it's amazing and super efficient. But there's no ecosystem of hardware, no ecosystem of software. It would take tens of thousands of people who are the best in AI today focusing on that to even prove out if it's worthwhile. On the hardware side, on the software side, on the model side. And so you look at Grok, Cerebras, SambaNova — they all over-indexed to the models that were leading at the time when they designed their chips. They made certain trade-offs. They put a lot more memory on chip, and NVIDIA was like, "We're not going to do that."
--:-- Guido Appenzeller: A lot faster, at least. More SRAM instead of DRAM.
--:-- Dylan Patel: Yes, correct. They went SRAM instead of DRAM. But then they usually have less DRAM, so there's a trade-off. Because there's more SRAM on the chip, you have to have less compute on the chip. And they ended up losing because the model sizes got too big. You have this super weird dynamic where they bet on something that was actually better. I have no doubt that Cerebras would run certain types of models better than NVIDIA, or Grok, or Tesla's Dojo. Dojo runs certain types of models way better than NVIDIA's chips because they're optimized for that. But then, even in vision tasks, you use vision transformers now. Model sizes grew. So it ends up being a catch-22: you optimize for something.
--:-- Today, you have this new age of AI accelerator companies saying, "OK, we're going to optimize for transformers." By the time they started designing, they said, "Transformers are dense models that are this big. The hidden dimension is 8K, your batch sizes are this big, your sequence lengths are this big. So let's just make a super large systolic array to create maximum efficiency." And then look at DeepSeek, or look at what the labs are doing. Actually, their shapes are much smaller. You need to do a bunch of small matrix multiplies, not massive singular matrix multiplies per layer. So the chip you're designing for that is actually not super effective for that workload. The software is evolving constantly because of what works best on NVIDIA. You see that with what DeepSeek is doing, what Alibaba is doing, what the labs are doing internally. You even see this with Google — their open-source Gemma models make different decisions because the shapes of a TPU are different than a GPU. And the GPU and the TPU are actually not that far apart. Blackwell and TPUs are converging on similar designs.
--:-- But to beat NVIDIA, you can't just have the supply chain win. You don't have this captive customer. So you need to do something that will give you a 5x advantage in hardware efficiency for a certain type of workload, and then pray the workload doesn't shift. Because NVIDIA is also optimizing their architecture generation after generation. They've added a lot of stuff to make their chips way better for the existing models. They're taking large steps every year, every two years. You have to go way over there in left field and hope that models stay over there. Because you have to win by 5x — NVIDIA is going to have supply chain efficiency over you, time to market over you in terms of new process nodes or new memory or whatever technology. Even AMD got to 2 nanometer before NVIDIA. They had higher density HBM, they use 3D stacking, all these things on supply chain that should be better than NVIDIA, and yet they still lose.
--:-- Erik Torenberg: There's still the software angle.
--:-- Dylan Patel: NVIDIA is fantastic. Yeah, and then there's software as well. NVIDIA is going to have better networking than you, better HBM, better process node. They're going to come to market faster, ramp faster, have better negotiations with TSMC or SK Hynix on the memory and silicon side, or all the rack people, copper cables, everything. They're going to have better cost efficiency. So you have to be 5x better.
--:-- Erik Torenberg: But to be fair, if somebody had a viable competitor that was even marginally cost-competitive, I'd guess many of the big consumers of GPUs would immediately shift some revenue there just to have a number two.
--:-- Dylan Patel: That's AMD today.
--:-- Erik Torenberg: And Microsoft stopped somewhat.
--:-- Dylan Patel: Yeah, there's still pretty limited traction. Meta continues to buy from them, and Microsoft did buy a bunch and then they stopped. Because AMD gives you all these advantages, but it ends up still not being better on a performance-per-watt basis, and they have a way bigger software team that are somewhat competitive on all these dynamics. So you can't just do the same thing as NVIDIA and do it better, or try and execute better like AMD. You have to really leap forward in some other way. But the design cycle takes so long that models will shift. They're like, "What's the next generation of TPU and GPU look like? OK, let's optimize for that." And the research path is... yes, neuromorphic computing could be the most optimal thing for us to do. But no one's working on that because you have to advance in the tech tree you've chosen. If you restart the tech tree, you're going to be like, "Well, this sucks." And if it branches this way and you're over here, you're screwed because you have to be 5x better. The supply chain stuff means that 5x actually turns into 2.5x, and then NVIDIA can compress their margin a little bit if you're actually competitive, and then that 2.5x becomes like 50% better. So it ends up being way too difficult. And the software stuff takes your 5x and makes it so you're only 50% better.
--:-- Guido Appenzeller: Defense supply chain for sure.
--:-- Dylan Patel: Yeah, defense supply chain. And Lutnick himself said, "We had to do this for rare earth minerals." It's interesting. China has provinces that have rules saying the H20 is not efficient enough to be deployed, which is super bizarre because it's clearly the best AI chip China has. Huawei is still a little bit behind.
--:-- Guido Appenzeller: Well, what's interesting is that efficiency is so much less of an issue in China than here because they just have the power infrastructure to support it. So even if they're running less powerful chips, it doesn't really matter because China has such an infinite supply of power.
--:-- Dylan Patel: Which is a big challenge in America. Jensen keeps saying he couldn't give away H20 in America for free. I've literally heard companies say, "Yeah, no, I mean, I wouldn't because I only have this much power. How am I going to... in data centers ready to go over the next year? If I bought an H20, I'd literally have less compute capacity and then I'd lose." Even if it was free, it doesn't make sense. Whereas China doesn't care. They can build these things. They have the muscle.
--:-- Erik Torenberg: I'm curious how this all shakes out. China's posturing really hard. They even put out something like, "We're investigating to see if there's backdoors in the H20." There's no backdoor in the H20. GPU is usually firewalled from the public internet anyway. You step through stuff before you get to the GPU clusters, so a backdoor wouldn't even matter. I think it'll be interesting to see because China can definitely deploy way more power to AI the moment they decide to. But there are competing interests because they want Huawei to be better than NVIDIA.
--:-- Dylan Patel: And this is how NVIDIA argued to the administration. They said, "If we don't do this..." Within Triton, which is a common ML library, ByteDance has open-sourced some stuff that plugs into this that is super awesome. There are all these other libraries. It's not just models that China open sources. It's software for NVIDIA that Chinese companies open source. By NVIDIA selling GPUs, NVIDIA's argument was that they were able to stop Huawei from building up a software ecosystem, and the Western ecosystem is better. But on the flip side, if you believe the models deliver more economic value to society than the hardware — which I actually think they do, there's just a value capture problem today — then you're giving China way more by giving them H20s and soon a version of Blackwell that's cut down, like Trump said, versus selling them the chips. The economic value derived from selling them the chips is not as large as being able to somehow sell them AI services.
--:-- Erik Torenberg: So is China gatekeeping power for AI?
--:-- Dylan Patel: I don't think so. Even with H20 being sold into China, and future versions of the chip, we still see Chinese companies like Alibaba renting GPUs outside of China because the GPUs they can get outside of China are just so much better on a dollar-per-performance basis. Renting them, or even going through a Singaporean company that is effectively a Chinese company and building data centers and putting chips in them. I don't think China is limiting the power per se. Chinese companies are growing their CapEx way more than U.S. companies on a percentage basis next year. The absolute dollar number, obviously the U.S. companies are spending more still on AI. On a percentage basis, Chinese companies are growing more next year. And you still have the problem that dollars spent to AI output in tokens is going to be lower because these chips are worse. So power is not the gating factor. It's always capital. At least today. Now China can spend a lot more capital if they wanted to. They're subsidizing the semiconductor industry to the tune of $150-200 billion a year through SOEs, through CapEx that's not generating revenue. So it's not like they couldn't do this to the AI ecosystem. Meta's CapEx is like $60 billion, Google's CapEx is like $80 billion. They could totally spend way more than that on a single effort. They just haven't decided to.
--:-- And I just think for the US, our buildouts are constrained by power. Google has a ton of TPUs sitting, waiting for data centers to be powered and ready, as does Meta with GPUs. We posted about how Meta is now building these effectively tents.
--:-- Guido Appenzeller: Is this to some degree also coupled to their unwillingness to sell them to a broader ecosystem? I mean, if they want to be confined in their own data centers and they didn't run data center buildout for their own hyperscaler cases quickly enough, then yes, that constrains them. If they were on the open market, would we still be constrained?
--:-- Dylan Patel: Yeah, for sure. Because companies like CoreWeave — why is CoreWeave valuable? It's really because they build infrastructure really fast. Their software is nice, but a lot of their customers are bare metal. Just replace the GPUs whenever they're broken and network properly. They glue more aggressively. And they'll go anywhere.
--:-- Guido Appenzeller: Jensen likes them as well. That's very important. Because it unconcentrates the ecosystem, which is better for NVIDIA. Having worked at Intel, I know exactly what's going through his mind.
--:-- Dylan Patel: CoreWeave doesn't care. They're like, "Oh, crypto data center, I will convert it to AI data center." They bought a company for $10 billion that's doing crypto mining, which was worth $2 billion a couple years ago. It's not because their Bitcoin mining business is growing, it's because they have powered data centers. Anywhere and everywhere, people are trying to build powered data centers, and companies like CoreWeave and Oracle are moving to the... Actually, today, Google just bought 8% of a crypto mining company called Terrowolf. Not because they're getting into crypto mining. No, because they need the data centers. They need the power. All the hyperscalers have said, "Screw off to my sustainability pledges" because they need power as fast as possible. They're doing things that take a little bit longer to move the ship, but even if you didn't do it in your own self-built data centers, there's still a lot of challenges in the open market. There's a deficit. And that's constraining American chip buildouts heavily. Others could maybe do it a little bit faster like CoreWeave or others. Oracle's got an open mind as well. But it's still constraining U.S. buildouts heavily even though the capital's been spent. The chips are 60 to 80% of the cost of the cluster depending on what chips you're getting. They've already bought the chips. They just can't put them anywhere because the data centers aren't ready. So it applies to Google, applies to Microsoft, applies to Meta, applies to a lot of folks.
--:-- It's really hard to build power infrastructure in the U.S. Power, grid interconnections, transmission, substations, all of this stuff. Electrical contractors, electricians in Texas — if you're willing to be a travel electrician, it's like oil pay. It used to be that if you're physically adept, you could go make $100k in West Texas. But who wants to do that? Now it's like, you could go 200 miles away from Dallas to what's still a reasonable town and build a data center and work on the wiring within the data center and all this other stuff, the transmission stuff. Your pay is up 2x now versus what it was just a few years ago. This labor problem is a challenge too.
--:-- In China, they don't have any of these problems, but they just haven't spent the capital yet. But capital is an issue as well because of the scale of what's being spent. NVIDIA's revenue this year is going to be over $200 billion. Next year, it expects over $300 billion. Plus Google is going to spend $50 billion on TPU data centers. Amazon's going to spend tons on Trainium data centers. The scale of dollars is quickly growing to nation-state level stuff. And what's more important is being able to decide to spend the dollars and what's cost effective. China is still constrained by that, but they can smuggle chips in, they can build data centers outside of China, they can rent data centers outside of China and have the most cost-effective Blackwell chips. ByteDance is either the biggest or the second biggest customer of Google Cloud for a reason. And they're getting many Blackwells from them. And the same with Oracle and the same with Microsoft. All these other companies are renting tons of chips to China anyway because it's more cost-effective to do that than build it yourself. So it's not like China has this mentality where "we only have to..." — the government does, but the infrastructure companies don't. Alibaba, Tencent, ByteDance, et cetera.
--:-- Guido Appenzeller: So what's the end game for data centers? We need more power. We need more cooling. Will all data centers be next to a nuclear reactor, or lots of solar, or next to deep sea water that we use for cooling? Where is this heading?
--:-- Dylan Patel: I think cooling — the physical cooling of a data center — there's this whole narrative about how AI uses so much power. It's not really. Farming alfalfa uses 100x the water of AI data centers, even by the end of the decade. And alfalfa is worth very little. Cooling is not that... People have experimented with undersea data centers to reduce the cooling cost. That doesn't make sense. It's 5-10% savings, but then it's easier to get the water out of the ocean than put the data center into the ocean. If you want to service it, you're screwed. The same with power. We talk a lot about how the power is not actually that expensive. It's just hard to build. Hard to get to the right place, hard to convert it down to the voltages and all the stuff that chips need. So it's less the magnitude of power and more where it is and how it moves.
--:-- Guido Appenzeller: Well, the magnitude too. In terms of total worldwide energy consumption, AI data is still a fraction of a percent.
--:-- Dylan Patel: Even by the end of the decade, the U.S. will be like 10% of our power will be AI data centers. Of our electricity. In terms of energy, that's even a smaller fraction. But shifting to electric vehicles also, you can probably make a bigger swing than with all the AI data centers we can build. In Europe, that number's not moving up that fast. I think we need to build a lot more power, but it's not some crazy amount. It's just doing it properly that's the hard thing.
--:-- Guido Appenzeller: Yeah, I agree. And again, the cost of power — you go look at these deals people are signing. Even though the price has skyrocketed from a few cents a kilowatt hour for these massive purchases to $10, when you think about the full TCO of a cluster, the GPU cost, networking, all of this stuff far outstrips the power. And same with cooling.
--:-- Dylan Patel: What percentage is power? If you do a four-year amortized GPU data center, 80% of the cost if you're building Blackwell is capital. It's the GPU purchases, the networking, the physical data center conversion, power conversion equipment. All of this stuff is 80% of the cost. Then 20% is your land, your power, your cooling, your cooling towers, your backup power, your generators. It's nothing. Which is why it doesn't matter if you spend 10% or 50% more on that. Because at the end of the day, the expensive thing is the chips. This is why what Elon did would seem silly. They spent a lot more money on generators outside the data center and these mobile chillers to cool the water down for their liquid cooling instead of the more cost-effective option, because it got the data center up three months faster. That three months of additional training time is worth way more on a TCO basis. The performance you got out of the chips and the time to market was way faster. And therefore it was the right decision, even though this part of the data center bloated the cost. Everything else is still there and you're still paying for the chips. And if they were sitting idle, it's not worth it.
--:-- Guido Appenzeller: Just by bypassing the grid, bypassing anything to do with interconnect, anything to do with public utilities.
--:-- Dylan Patel: Exactly.
--:-- Erik Torenberg: What's your take on Intel? Where is Intel going?
--:-- Dylan Patel: I think the world needs Intel. The U.S. needs Intel. Samsung is doing worse than Intel on leading-edge process development, in my opinion, based on various customers in the industry having done test chips at Intel versus Samsung. I think the industry generally agrees that Intel is further along on the 2-nanometer class process technology than Samsung is. But both are way behind TSMC. And TSMC is a monopoly to some extent. The number one question people always ask is, "Why is TSMC not making more money? Why are they only raising prices next year 3% to 10% depending on what it is?" TSMC is a monopoly. They could raise a lot more, but they're good Taiwanese people rather than dirty American capitalists. If TSMC was managed by Americans — most ownership is actually American in terms of the stock, it's on the New York Stock Exchange — they would have raised prices a lot more. So there is this difficult thing: there's one island that controls all leading-edge semiconductors, and not just leading-edge, the majority of trailing-edge production as well. Something needs to be done. Intel is behind, but not absurdly so. If something were to happen to Taiwan, Intel would have the most advanced technology in the world. It's just not economic.
--:-- Guido Appenzeller: Can you keep Intel as one company if you want them to be competitive?
--:-- Dylan Patel: I think the process of splitting it would take so much executive time and so much executive effort that you would be bankrupt by then. That's the big challenge. I think Intel should be separate. But to properly split the company, the management time needed is absurd. Instead, you need Lip-Bu Tan, who's the CEO of Intel. There's a lot of drama around him because he's one of the greatest semiconductor investors ever. He's invested in so many different companies first. He was on the board of SMIC, which is China's TSMC effectively. Some of the biggest tool companies in China, he was the first investor in them. It's a multi-polar world there, and he's making good investments. But now people are getting mad about that. But he recognizes that he needs to not spend his time on splitting the company because then he never actually fixes the company.
--:-- Intel's problem is that it takes them five to six years to go from design to shipping the product, in some cases more. When they tape out a chip, they send the design to the fab, the fab brings back the chip, they go through 14 revisions in some cases. The rest of the industry goes through one to three revisions if they're good. They'll launch a chip in three years. But if you look at Intel today, they still don't have a competitive entry on the AI side. And they won't.
--:-- Erik Torenberg: So what does that mean for their offering? They're still doing great on CPUs. They don't have a good AI chip product. Is it long-term sustainable positioning as a standalone chip company?
--:-- Dylan Patel: IBM still makes more money every launch off of mainframes. So it's not like x86 is dead. You don't get the growth rates, but you could totally run this as a very profitable enterprise. I think the same with PCs. There's some turmoil, some Arm entry, some AMD competition. It can be a very profitable business if it had one third the people or half the people working on it. So Lip-Bu Tan needs to go into the design company and lay off a lot of people, keep all the good people, and make sure they're designing fast and launching from design conception to launch in two to three years, not five to six. And on the fabs, he has to do the same thing.
--:-- One of the heads of fab automation at Intel — I explicitly told Lip-Bu Tan, because we have a couple of ex-Intel people who are actually good in the company that worked on the fab side. We were like, "Who are the worst people?" It's like, "Oh, this guy sucks." I explicitly told Lip-Bu Tan. He had never talked to the guy because it was four layers down. The company has absurd amounts of hierarchy. He goes and talks to the guy and he's out. He figures out who's bad and who's good. He has to go in and... the vast majority of the team at Intel is the one who led the world in production and process technology for 20 years. But there's a lot of built-up crap. So he has to figure this out. He can't waste his time on structuring to split. I think it would be better if the company split. I just don't think he can spend the time to do that. The design side of the company — you're not really going to get into AI. You have to make some money there. But the fabs, I think, could truly become a competitor. But they're going to go bankrupt by the time anything can happen. So he has to figure out how to get capital, clean up all the crap, make the yields go up, make the product ship way faster. All of these things are basic problems. I think the goals are completely correct.
--:-- Guido Appenzeller: I think the big challenge, just reflecting back on my time there, is that right now, if you look at Intel, they have essentially software, the chip design, and then the core manufacturing part. They have three very different cultures. And it's very hard to get everything under one umbrella. I think you should even run the company separately. But you can't physically separate them entity-wise because it's going to take so long to sever all these things. He doesn't have time.
--:-- Erik Torenberg: Intel is literally going to go bankrupt if they don't have a big cash infusion, or they lay off like half the company. Some could argue you need to lay off 30% of the company anyway, but there are a lot of bad things that happen if that happens. And they need to spend a lot more on building the next generation fab, even if they fix the fab, and they don't have money for that.
--:-- Dylan Patel: So there are a lot more important problems than physically separating the company, even though I think long-term, yes, the fab has to be separate from the chips and design software. The chip design part of the company — that's going to make each company much more accountable and able to service their customers better. It's just that's going to take too long and they're going to go bankrupt by then.
--:-- Erik Torenberg: Awesome. But I hope someone does something right. Like you get a big capital infusion.
--:-- Dylan Patel: The big hyperscalers are muscled into thinking, "OK, wait. If TSMC eventually grows their margin to 75% because of the monopoly, plus they intake all this stuff like co-packaged optics and power delivery, all of a sudden the cost is going to spike. So we should actually just throw $5 billion at Intel each, right? Screw it." And that could actually give Intel enough of a lifeline to potentially get to something and maybe be competitive. That's the hope.
--:-- Erik Torenberg: Can we finish by finishing this game that we started when we gave Altman advice? If Jensen was here, what advice would you have for him?
--:-- Dylan Patel: If Jensen was here... I think he has a massive balance sheet. Jensen's free cash flow is ridiculous. The new Trump tax bill institutes something really incredible: you can depreciate all of the GPU cluster cost in year one. We put out a note about how the tax implications for Meta are like $10 billion a year, and across each of the major hyperscalers, it's massive. Well, NVIDIA is going to spend tons and tons of cash, or they're going to spend tens of billions of dollars of taxes. Why don't you get into the infrastructure game somehow?
--:-- Now, this is obviously going to be crazy because now they're buying their own GPUs and putting them in data centers and doing stuff, and they're competing with their own customers. But they're already doing that anyway because their customers are trying to make chips. They should accelerate the data center ecosystem with investments. We can have a very high degree of accuracy on what they're going to do next year in terms of revenue because it's just the number of data center watts that are being built. This is a harder thing to shift up and down. There's a little bit of share difference between how much is TPU versus GPU, but you have to accelerate the infrastructure and you need to spend all of this capital that you're building.
--:-- Do you want to go the route of doing buybacks and dividends? You're a loser if you do that. You can make more money by reinvesting and building a bigger company that's not just chips into the ecosystem or servers into the ecosystem, but actually controlling the infrastructure end to end somehow. So I think there's something he could do there with this massive war chest. NVIDIA has done some buybacks and some dividends, but the cash on their balance sheet keeps growing. They're going to have north of $100 billion of cash on their balance sheet by the end of this year, I think. So what are you going to do with that? I think there's something moving into the infrastructure layer much more that they could do if he really wants to be the king of the world, which I think he does.
--:-- Erik Torenberg: Sergey and Sundar?
--:-- Dylan Patel: I think they should open up the kimono on TPUs. Start selling them, open up the software, open source a lot more of the XLA software. There's OpenXLA and there's XLA, but the vast majority is closed source. Really open up the kimono on that and be a lot more aggressive. They're still pretty not aggressive on data centers. They're pretty not aggressive on a lot of elements of the company. The TPU team's next-gen designs are pretty not aggressive, partially because a lot of the TPU team has left to go to OpenAI. The best people that I knew — it was actually really annoying. I knew like four or five people and they all went to OpenAI. I think they could be a lot more aggressive in many ways across the company. They don't have to be, but they could.
--:-- Because AI — this ChatGPT, the shift of search queries, the monetizable ones especially from purchasing agents — is going to really screw Google long-term if they don't get their act together. I think they've gotten their act together on DeepMind. There are still some inefficiencies, but Sergey works within DeepMind a lot and they're driving hard. They're still a little bit behind. But physical infrastructure, TPU, and how much money they could make and how much they could take the wind out of everyone else's sails if they start selling TPUs externally and reorg around building data centers much faster so that they do have the most compute in the world — because they did, but now certain companies are going to surpass them potentially over the next few years if they don't really get their act together. So I think that's what I would say for them.
--:-- Erik Torenberg: Yeah, and also learn how to ship product.
--:-- Dylan Patel: Zuck. I think it remains to be seen what goes on with superintelligence, but they're trying to move super fast with the data centers. "Screw it, we'll build tents instead of physical data centers because we only need these for five years anyway." The superintelligence moves — you could say whatever you want, but trying to buy Thinky for $30 billion or SSI for $30 billion didn't work out. So then they spent not even that much on hiring all these people. I think he recognizes the urgency with the models and the infrastructure. If you read his website post about AI, I think he sees the vision. There are the wearables, integrating AI into that, being your AI assistant to do all this purchasing. I think he sees the vision, but I think he also needs to focus on actually releasing that faster. The products that they do outside of their core IP — every time they launch something, it's kind of mid. Meta Reality Labs is doing well, but I think they should go more explicit. Have a ChatGPT competitor, have a Codex competitor. Just start releasing way more products. They're really just focused on their individual gardens rather than branching outside of it.
--:-- Erik Torenberg: Do you think Apple should have that same sense of urgency? If Tim Cook was here, what would you tell them?
--:-- Dylan Patel: The funny thing is some of their best AI people are now at superintelligence. They're building an AI accelerator. They have AI models, but they're just way slower. They did mention on the last earnings call they're going to allocate more capital to this. But guys, Apple, you guys are going to lose the boat if you do not spend $50 or $100 billion on infrastructure.
--:-- Erik Torenberg: You don't think the grand jury will cut it?
--:-- Dylan Patel: I think more and more you'll see people — great, Apple has this walled garden, but they can only do so much to protect it. IDFA, they shut down ads or data sharing to Meta, but Meta made better models and now they have way more data and way more power over the user than they ever did before. It was good that Meta kicked the crutch off of them, or Apple did. But the same applies to AI. Yes, they have access to the text and they have access to this, but I think other people are going to be able to integrate user data, and agents will be able to integrate all this user data. They'll start to lose control of what the user experiences as more and more gets disintermediated by AI being the interface rather than touchpad and keyboard. I don't think they've truly realized what happens when the interface to computing is AI. They market it, but that's going to shift computing really heavily. They have great hardware and their hardware teams are working on awesome stuff and form factors. I just don't know if they get what is actually going to happen to the world in the next five years truly well enough, and they're not building fast enough for it.
--:-- Erik Torenberg: What about Microsoft to that end?
--:-- Dylan Patel: Microsoft has the same problem. I think they were super aggressive in '23 and '24, and then they pulled back heavily. Now OpenAI is slipping through their grasps. There's that whole thing there. They cut back on data center investments heavily. They were going to be the largest infrastructure company in the world by a factor of 2x, which you could argue maybe was too much and maybe it wouldn't have been economical. But they're losing grasp on OpenAI. Their internal model efforts are failing spectacularly. They're on LLM Arena right now and they're pretty decent there, but that's just a sycophantic model. MAI is failing. Azure is losing a lot of share to Oracle, CoreWeave, and Google. Their internal chip effort is by far the worst of any hyperscaler. They're just mis-executing.
--:-- GitHub — how is GitHub not the highest ARR software code model? They had the best IDE, the best source code repository, the best enterprise sales force, the best model company as a relationship, and they were the first to market. They had everything going for them. And there's just nothing. GitHub Copilot is failing. Microsoft Copilot is still crap.
--:-- Erik Torenberg: Yeah, it's unusable.
--:-- Dylan Patel: It's like, what is going on? You need to shake the crap out of the company. I think they win a lot because they have the best business-to-business relationship with so many enterprises on the planet.
--:-- Erik Torenberg: Yeah, but they end up not having the actual product to sell them, which is really scary. So they need to really work on product. Satya has done great on sales and stuff, but yeah.
--:-- Dylan Patel: If Elon was here, what advice would you give him? A lot of people at xAI are mad about the porn models. It's fine. You're going to make a ton of money off of this. This is how you accelerate the revenue of that company. But he's losing a lot of talent and axing a lot of good projects. Elon is a magnet to amazing talent and building stuff, so I won't bet against him. It seems like since he left the administration and focused on stuff again — but I think he's focused on a lot of things. I think RoboTaxi is starting to look good again. I haven't ridden one yet, but I have some friends who have ridden one that looks pretty decent. He could not make these snap decisions, which often are the reason why he's amazing, but some of these snap decisions are hurting him. I'm not sure if I can give Elon that much great advice because maybe it's just like focus on the products again, more. But he is working on that stuff a lot.
--:-- Erik Torenberg: Yeah. I think that might be a good place to wrap. Awesome. It was a great discussion. Dylan, thanks so much for joining us.
--:-- Dylan Patel: Thank you for having me.
--:-- Erik Torenberg: Thanks for listening to the a16z podcast. If you enjoyed the episode, let us know by leaving a review at ratethispodcast.com slash a16z. We've got more great conversations coming your way. See you next time. Thank you.
