--:-- Raghu Raghuram: We live in a world where demand for AI capacity tokens is near infinite and supply is incredibly constrained. So another way to put it is we all just live in Jensen's world.
--:-- Guido Appenzeller: You were doing, back then, M&A for the most valuable company on the planet.
--:-- Raghu Raghuram: The retro on the M&A process that I think worked well for Cisco was we were not afraid to fail. And we did it with high enough frequency that we got a few outliers. You bought this thing for $500 million, and it's doing a billion in earnings two years later.
--:-- Guido Appenzeller: When you see the AI build-out, how do you look back on your previous experiences?
--:-- Raghu Raghuram: If you look at the revenue ramp of Anthropic, OpenAI, they are definitely experiencing growth rates that were far superior to what we saw at the time.
--:-- Guido Appenzeller: Do I think the build-out that's happening now is useful and will be fully utilized, if not more? 100%. Will it be owned by the same people that own it today?
--:-- Raghu Raghuram: The last great infrastructure boom built the internet. This one is building the infrastructure for intelligence. What can the first tell us about the second?
--:-- Narrator: In this episode, a16z's Raghu Raghuram and Guido Appenzeller sit down with Hanabi Capital general partner and former Cisco executive Mike Volpe. They compare today's investment in GPUs and data centers with the fiber boom of the late 1990s and why Mike thinks the infrastructure will remain valuable even if a correction reshuffles who owns it. They also discuss NVIDIA, custom AI chips, networking, and what Cisco's acquisition strategy can teach founders and investors today. Mike's view? The opportunity underneath this build-out could ultimately be even larger than the internet boom.
--:-- Raghu Raghuram: Great to have you here on the podcast, Mike. It's a very opportune time to have you because you were one of the architects of the great internet build-out when you were at Cisco. And now, of course, we've got the great AI build-out happening. We and the audience would love to get your thoughts on then and now from all different dimensions: the actual build-out, the capitalization of it, the technology differences, et cetera. So let's start for those that don't know or remember. Mike was a senior exec at Cisco, overseeing a lot of the M&A and strategic moves that Cisco made in the late 90s. That was the time the internet got built out, so Mike was in the middle of it all. You were there when the internet started taking off. You guys rushed into a lot of great things, and the industry rushed to build a lot of bandwidth, which for some time was not useful. And then it turned out it was actually too little. There are some loose similarities to what's going on now. So when you see the AI build-out, how do you look back on your previous experiences?
--:-- Mike Volpe: Well, first of all, it's an absolute privilege to be here with two legends. Both you, Raghu, and Guido — I got the chance to work with Guido at least once. So it's fantastic to be here. As you said, it's an interesting time because history doesn't repeat itself, but it definitely rhymes. So there are some similarities.
--:-- Maybe it's useful to give some context on the observations of the late 90s build-out and what might apply in today's world versus what doesn't. Some of the similar themes: we were building out this technology called the Internet and the Web and all the things that rode on the Internet. We had expectations that this would be massively transformative to the world. We imagined a wide range of applications that would ultimately benefit from this build-out. Applications were different media types. First we just had pictures and text, then we had audio, then video, then high-quality video. Applications like commerce, applications like social networking. All of these things were imagined in the 90s but didn't exist yet.
--:-- In parallel, there was a very big surge in demand. At a minimum, there was a surge in demand for connectivity. On top of that, a reasonable surge in demand for bandwidth. And in parallel, there was a great influx of capital of two shapes and forms: equity and debt.
--:-- If you look back at what happened in the bandwidth world, particularly in fiber optic bandwidth, the increments of increase in the amount of bandwidth are quite different than compute. That's because we invented this technology called WDM that essentially takes a piece of fiber and says, if you used to put one gigabit on there or 10 gigabits on there, by putting in more colors of light in that same piece of fiber, you can now have 40 times or 80 times that capacity. So capacity increased quite dramatically with the advent of technological change. That's actually different because compute is ultimately bound by something that resembles Moore's law. The pace of improvement is more moderated — hard to believe, but it is actually more moderated in compute than it was at the time.
--:-- The other component that I think is different is that the practical applicability of AI today is higher than the practical applicability at the time of network bandwidth. That represents itself by the revenue ramp of the providers. If you look at the revenue ramp of Anthropic, OpenAI, I don't know what the numbers look like within Google, even the providers of the open source models like Fireworks, Together, Modal, et cetera, they are definitely experiencing growth rates — maybe not quite at the same margins, but certainly growth rates that were far superior to what we saw at the time.
--:-- Now, then what happens? At some point, there is a shock that some of these businesses built on the Internet are not good enough. That ripples through the industry. Ultimately, the players that were most saddled with debt went bankrupt. It's interesting to note that debt was buying assets — those fiber optic cables — and those assets actually turned out to be quite useful in the future. Except they were operated by somebody else because the guys that went bankrupt didn't have them anymore.
--:-- The first thing to recognize is the underlying assets that were built at the time of this bandwidth — and I suspect this compute this time — will be useful. There's no question that it'll be useful. Whether it will be owned by the same companies or not is an interesting economic question.
--:-- So that's a long-winded answer to say: do I think the build-out that's happening now is useful and will be fully utilized, if not more? 100%. Will it be owned by the same people that own it today? A lot of interesting dependencies. A lot of things that are outside of the control — macroeconomic and geopolitical — are really outside of the control of the builders themselves. When I talk about the builders, some of the balance sheets are pretty healthy. Last count when I was talking to folks at NVIDIA, there are 200 NeoClouds, and those balance sheets are not good.
--:-- Raghu Raghuram: Yes.
--:-- Mike Volpe: So I'll predict that there will be some — not all of them, of course — some better than others. But broadly, I expect that there will be an economic correction in the market, but the underlying assets will be quite useful for a long period of time.
--:-- Guido Appenzeller: Isn't there one other big difference from how it was back then, which is that everything is now much larger? Back then we were investing a small fraction of a percent in infrastructure for the internet. Today, we're investing 2% or so of our GDP in AI infrastructure.
--:-- Mike Volpe: Well, it is much bigger, but it's an interesting concept. At the 100,000-foot abstraction, what is it that is being built? Back then, it was communication. Today, we're building intelligence. The economic value of intelligence is probably higher than the economic value of communication.
--:-- Raghu Raghuram: Probably, yes.
--:-- Mike Volpe: Right? So I think it's unsurprising. I also think that, believe it or not, it's been nearly 30 years — the markets are more efficient now, which is why so much more risk capital is willing to make its way into AI, which even back then was not able to. So net, I'm actually more optimistic about this cycle, but I also believe that there will be some correction and some cleaning up of the industry structure.
--:-- Raghu Raghuram: The role of IT has also massively increased. Back then, in the late 90s, PCs were somewhat common but not ubiquitous. Today, software has eaten the world. For any company out there, IT is a very central part of what they do.
--:-- Mike Volpe: Cost of distribution of intelligence.
--:-- Raghu Raghuram: Oh, for sure. And not just any company, but 7 billion people have a supercomputer in their pocket.
--:-- Mike Volpe: Exactly. So there's much more evenly spread consumption of the underlying benefit that's being produced.
--:-- Guido Appenzeller: I don't fully recall all the puts and takes in the 90s and early 2000s on the financial side. Here, one of the recent shocks to the system was the rising interest rates and the scale of capital causing people to say, "This is going to crowd out everything else and we may not even have enough capital." During the bandwidth build-out and the internet build-out, that was not the case, correct? There was no sudden increase in interest rates?
--:-- Mike Volpe: No, there wasn't a massive increase in interest rates. There are a couple of big differences. One is that on a proportional basis, I believe more of the capital came from the balance sheet of the equipment vendors than today. Today, it's a more capital markets-driven build-out — private debt, public equities, et cetera. There is NVIDIA vendor — I don't think you're supposed to call it vendor financing, but it looks a lot like it. So there's a little bit of that, and there's some corporate financing, but it's largely equity, and it's not tied to specific equipment purchases. So the capital structures are a little cleaner now.
--:-- One thing that interestingly favors the 90s is that a lot of the money went into the ground in the form of fiber optic. When you put fiber optic in the ground, it's good for a long, long time. It's a usable asset for 20, 30 years. I think one of the deterministic factors is the compute you put in the ground — what is its actual life? This is a fascinating phenomenon. The dominant part of that compute is chips — GPUs. 80-plus percent. It's the big chunk of it.
--:-- I remember back when people were very excited about Hopper H100s. They were like, "Okay, maybe it has a five-year useful life and you're renting it out at $1.82 per GPU hour." That was four years ago. So the chip should be defunct now, or near defunct. Not at all defunct. Not only that, it rents at a higher rate — $2.50 or $3. So the interesting part is we thought this asset would have a five-year life, but partly because of AI itself, it's a useful asset for quite a bit longer. If you're in the business of financing an asset and that asset ends up having a much longer useful life, then you're the beneficiary because your business models were constructed on a shorter life. So in some sense, the fact that an unintended consequence of AI being good at software is that the underlying asset life is getting longer, thereby helping it to be more financeable. That's an interesting phenomenon that I didn't quite expect when this thing got underway.
--:-- Although the debt investors have not broadly accepted the extended life of a GPU. But the new clouds clearly believe that.
--:-- Guido Appenzeller: I'm not sure I have either. Right now, what's mostly happening is that we have a massive scarcity of an asset. So even previous generations are still worth a lot. If we ever get to a point where we have an abundance of GPUs, that might completely change. You might go back to those numbers. So I think the key metric is how much token revenue can you extract per watt.
--:-- Mike Volpe: Maybe, yes. So the Hopper is going to be a lot worse than Blackwell, which is going to be a lot worse than Vera Rubin. They're all just more efficient at getting dollars out of a watt. So in theory, at some point when you have sufficient Vera Rubins, it doesn't really make sense to run H100s anymore because it's just the worst way of converting electricity into dollars.
--:-- Guido Appenzeller: Correct. But the problem is that right now the demand is so high that even if the watt-to-dollar conversion is low, you're still going to use the asset. The scarcity is so significant. Ultimately, the scarcity is in the manufacturing of the semiconductor itself, and that doesn't look like it's changing anytime soon.
--:-- Mike Volpe: Correct. So scarcity persists, and the life of the asset continues to be long.
--:-- Raghu Raghuram: The other big difference: there was a period of time where there was unused bandwidth. Today, there is no such thing as an unused GPU.
--:-- Guido Appenzeller: There's a bit of unused energy. With better networking, you could get more flops out of a given GPU.
--:-- Mike Volpe: I was just going to say that. I'm confused by a factor in certain use cases and how people have built it, but certainly generally not.
--:-- Raghu Raghuram: Back when you were at Cisco, were you still there when Cisco was the most valuable company on the planet?
--:-- Mike Volpe: Yes, indeed, I was there.
--:-- Raghu Raghuram: So you were doing M&A for the most valuable company on the planet, and it worked pretty well, looking back?
--:-- Mike Volpe: I was very happy with it.
--:-- Raghu Raghuram: What would be your words of advice to NVIDIA in 2026? Whoever runs M&A there is in exactly the same spot you were.
--:-- Mike Volpe: Vishal is in the same spot I was. Look, the retro on the M&A process that I think worked well for Cisco was that fundamentally we were okay with making enough acquisitions that we got a statistically significant outcome. Some worked really well, some worked so-so, some didn't work at all. If you kind of drew it out, a third of them worked really well, a third were whatever, and a third failed. The benefit was that we were not afraid to fail. We did it with high enough frequency that we got a few outliers. Andy Bechtolsheim's company, Mario Mazzola's company — even for us, they were venture-like returns. You bought this thing for $500 million, and it's doing a billion in earnings two years later. That's amazing.
--:-- During the period that I was there — I joined in '94 and I was the one biz dev hire — by 2000, roughly 50% of Cisco's revenue came from acquired sources. That was incredibly useful. We never really spent much more on a given acquisition than 4% or 5% of our market cap. Most of the acquisitions were actually cheap relative to our market cap standards.
--:-- Raghu Raghuram: How much would that mean for NVIDIA? I don't even want to think about that.
--:-- Mike Volpe: My rationale at the time was: if your stock trades more than the price you paid for the acquisition in one day, then you can do as many acquisitions as you want. Nobody will care.
--:-- Raghu Raghuram: Were both of them done with equity?
--:-- Mike Volpe: All equity. Almost everything was equity. A handful of the small ones were cash. And Cisco — they became major product lines. It was not a little tuck-in that added a feature somewhere. It was a new product line that in some cases became the leading product line in the industry.
--:-- Now, you're a networking guy, so you'll appreciate this. One thing that's a little different about a networking acquisition versus others is that networking operates on open protocols. That means I could buy a switch company made by somebody else, and it immediately talks to my switches or routers freely because all the protocols are open. At Cisco, we had a couple of proprietary protocols that we enjoyed, but by and large, all the OSBF, BGP, ISIS, Ethernet switching, 802.whatever — they all worked. So you could immediately integrate the product into your sales team. You literally just change the badge on the thing. That allowed us to be super high velocity.
--:-- Guido Appenzeller: How does that contrast to Jensen's world today?
--:-- Mike Volpe: They've made some really clever acquisitions. Mellanox is the world's largest — that was amazing. It was genius. Now they're making other acquisitions that sometimes I scratch my head. I'm not entirely sure what Hugging Face is all about, for example. Maybe there's a rationale, but it's hard for me to see it. I don't think it matters. I think it's fine for them to take risk. Some of them will work. If they work as well as Mellanox, holy cow. Some of them are not going to work. The market has granted them a moment of incredible optionality. They get to do whatever they want, which is the optimal moment for them to take these risks.
--:-- I think they're being very clever. You can always critique any given acquisition. Trust me, I got critiqued endlessly at Cisco because I bought StrataCom and not Cascade, and I bought an optical networking company from Vinod that went nowhere. But in the end, our team did 70 acquisitions while I was there, and those turned into half the company's revenue. We were doing $30 billion in revenue, so that's $15 billion in revenue. The strategy worked. I think the team at NVIDIA is doing that. I think that's interesting and wise.
--:-- The degree to which they are financing the industry, we'll see.
--:-- Raghu Raghuram: That's different. In that sense, NVIDIA is basically trying to shape what an industry looks like.
--:-- Mike Volpe: Yes. And I think industries are powerful and they find their own shape. It's hard to reshape them.
--:-- Guido Appenzeller: Let me defend NVIDIA a little bit. Having been at Intel and having seen how the big hyperscalers essentially disintermediated Intel from the end customers — at the end, they could build better chips than Intel just because Intel had no idea anymore what the customer was doing with the chips — I have some sympathy for wanting to have more than three customers to sell to, or maybe four.
--:-- Mike Volpe: As do I.
--:-- Guido Appenzeller: You could argue: what is efficient capital allocation? Is it giving it back to investors, or could it be — at this time, which is probably once in a lifetime — put it all to work on future innovation?
--:-- Mike Volpe: I think both of you make very valid points. The thing I would say, Guido, is that an industry wants to look a certain way because when it does, it creates maximum utility and maximum economic value. Why are there three hyperscalers? Because it's a business where scale matters. Software, yes. Scale matters, and therefore the industry wants to concentrate. No matter how much you finance them, the little ones are always weaklings. If you're NVIDIA, you don't like that. You don't want three customers. You want 50 customers. So you fund them. But at some point, they're going to die, because whenever it's a scale business, the industry wants to consolidate. We could fund 100 airlines in America, but in the end, you're going to have three. My point is not that it's a bad idea. It's just that it's futile. Noble intentions.
--:-- Guido Appenzeller: Now, to your point, it could be a very good return. I don't know if they actually care about the return on capital. They might. And if they get out at the right time.
--:-- Raghu Raghuram: It's not even about the return on capital. I agree. It is to some degree shaping industries, deciding the shape of the industry. But on the other hand, it is enabling a lot of innovation as well, because our startups and lots of other startups do get access to that capacity now.
--:-- Mike Volpe: No, look, overall, I get why they're doing it. Many of our companies are beneficiaries of it. So thank you, Jensen.
--:-- Raghu Raghuram: Exactly. At VMware, we did probably not as many as you did, but maybe 50-plus acquisitions over the full lifetime of the company. The observation was that whether you were acquiring a 20-person company or a 2,000-person company, the amount of post-acquisition work in terms of integration, culture, business models, sales team, et cetera, was always a lot of work. How did you guys pull it off at Cisco?
--:-- Mike Volpe: Structurally, the way we did it was that we had a very small deal team — less than 10 people — whose job was to acquire. And we had a somewhat larger integration project management team that had rules and guidelines about how we did integrations. They had colleagues in the various business units, divisions, and functional units within Cisco that collaborated with them.
--:-- The important thing to recognize is that businesses are good at processes. They're very bad at one-time events. If we were going to do a large number of acquisitions, we needed to have repeatable processes that were performed over and over again. We had a centralized group that was the administrator of those processes.
--:-- More often than not, in the beginning at Cisco, there was nobody formally assigned to this. It would be like, "Raghu, it's your job. Go have fun with that integration. Let me know when it's done." And you're like, "Where do I even start?" That was the modality. We realized this was bad, so we created this process and it worked pretty well.
--:-- Where it got challenging was that we liked to buy small things, and small things became too small to be significant to the overall business because the business got so big that it just didn't make a difference anymore. After my time — I stopped doing M&A in 2000 and switched to running a business unit — they started buying bigger things. Then you get into the problem you're talking about. Big things are very hard to integrate. They've been around a long time, they have a long culture. We bought Scientific Atlanta, which was a disaster. We bought a bunch of things that didn't work very well. We also got into the mode of not fully integrating them — partially integrating them.
--:-- Guido Appenzeller: That's terrible.
--:-- Mike Volpe: It doesn't work very well. So the mitigation is to have a process, but when you stop respecting the process, you do have big problems like you talked about.
--:-- Raghu Raghuram: What about from the portfolio company side? You've got portfolio companies, we've got portfolio companies that, at least in recent memory, have had exits into being bought by other large companies. Two questions. One: if you're a portfolio company CEO, what would you watch out for? And sitting on the boards of these companies, how do you advise your portfolio companies these days about exits in general?
--:-- Mike Volpe: When you're in our business of being on boards and you're facing a situation where you need to sell — there's an opportunity to sell a business — it's one of three outcomes. One: your company's doing great, and it's a really bad idea to try to sell it. So you try to convince the CEO not to do it. Two: the company's struggling, you don't have a lot of finances, it's a nice way to get out, everybody's happy. Then you just help the deal happen. You manage a process.
--:-- Then you have the tweeners. You're not totally sure. I think there are a couple of important indicators that help you determine what to do. The first is: does the CEO, the founder, the founding team have the fire? Do they want to do this? Do they believe they can get through this intermediary period and bust open into the next phase? If you have a CEO or a leader who's like, "Maybe I should consider this. I'm a little tired. It's been a long haul," then let's just sell this thing and move on.
--:-- The second factor is you have to be pretty cold and objective about whether this is going to be an interesting enough business. Not, "Will it double from here?" Because at the end of the day, if it's double, I'll take the money today. But will it 10x from here? That gets to be a really hard decision because it's a little bit emotional for you. You're not a founder, but you've been working with this company for four, five, six years. Your heart's in it. You told your good friends to go work there. Yet you have to step away and objectively say, "This is not 10x from here." Then you help steer the ship. But I think the most important parameter is where the founder's head is at.
--:-- Raghu Raghuram: There is a certain element of mystery or unknown for a lot of the portfolio company founders on how these things actually happen. People think one day somebody calls another company and says, "Hey, I want to buy you." Versus the other thing where people have built relationships for a long time. How did you approach it at Cisco? And how do you advise your portfolio companies, even if they're expecting a 10x?
--:-- Mike Volpe: You do as good a job as you can building a network and knowing what's going on. It's basically cataloging the inventory, the way we used to think about it. Every company in the market is a potential acquisition. So you catalog them: where are they in their growth rate? What kind of founders do they have? Who are the backers? What are their aspirations? Who are their customers? Just be informed. The corporate development group had to do that, but I considered it everybody's job at the company to be constantly informed about what's happening around them.
--:-- At Cisco, we would try to weave the inventory against a strategy statement. For example: "We're a router company, but there are going to be more switches than routers. Therefore, we need to have a switch. What are the switching companies that are available?"
--:-- Mike Volpe: ...right, or optical networking is becoming important because of bandwidth, blah blah blah. What are the companies that are available? You often have the other way where companies come to you and say, "Hi, I'd like to get bought." I think broadly a bad idea, right? Because there's either an auction going on, or you should have thought about it earlier if you were interested in it. But basically, you almost never pay attention. We never tried to pay attention to inbound.
--:-- Flip side: you're now a small company or a startup. How do you think about that? I always encourage the founders of our companies to talk to big companies, to talk to competitors, and to always be in dialogue. Some people will say, "Well, you're shopping yourself." No, it's an inverse intelligence gathering mechanism. I even encourage my CEOs to talk to their direct competitors. Go to coffee. If you're clueless, you might leak some information towards them. Hopefully you're not clueless. And they might tell something to you that's interesting, or they might shape your view on an industry or something. So I encourage people to be out there always talking and seeing what's happening in the market. Usually you have a pretty good idea of, "Okay, my company is struggling a little bit. I think I'd like to sell." I kind of already know them because I've been in conversations with who's where.
--:-- So I would say, don't spend your entire day talking, but going on one or two coffees a week with the powers that be at NVIDIA, the powers that be at Amazon and OpenAI, et cetera, et cetera, when the right people are talking, do it. It's fine. The information exchange, you don't lose that much in that. They're already thinking about it, but you gain a lot by having that relationship.
--:-- Raghu Raghuram: I think this is really good advice. I keep telling my portfolio CEOs the same thing. If you don't have the cell phone number of the CEOs of each of your competitors, that's a problem. There will be a day when you need it.
--:-- Mike Volpe: Absolutely. So it's a network you need to build.
--:-- Raghu Raghuram: Yeah, yeah. Now, enough of this M&A stuff. Let's get back to actual technology. You spent 30 plus years in infrastructure. Very few people know more about infrastructure than you do, what you've done in the past. Summarize the landscape for us right now. When you think about infrastructure, opportunities versus where are the technology gaps that you see, and what founders should be really looking at.
--:-- Mike Volpe: Well, super interesting topic. I would say you want to think about the current shape of this industry in two dimensions. One is the vertical stack that starts with EUV equipment and goes all the way up to AI, applied AI and applications, and everything that sits in between that sandwich. And then you want to think about particularly the infrastructure side. What's the value chain of it?
--:-- The first thing you would say is right now we live in a world where demand for AI capacity, tokens, whatever, is near infinite and supply is incredibly constrained. Another way to put it is we all just live in Jensen's world. He's king of the hill, there's no question. I think the shape of the industry will begin to change when either the characteristics of demand change or the characteristics of supply change.
--:-- My view is right now the supply side, because so much of the limitation — well, there are some power limitations for sure, but the dominant limitation is silicon. Unless there's more fab capacity, that doesn't change, which means it's not likely to really change in the next two or three years, maybe five years out. You could imagine that there would be more fab capacity. You could put more wafers through the same machines, or you can make more ASML machines and so forth. But that's probably a governing factor of our industry, which means we live in a reasonably scarce world. And those who have fab capacity are actually kind of kingmakers of the infrastructure.
--:-- Raghu Raghuram: Yeah, I see that now. Five to one or seven to one or whatever. It's a massive economic difference. So people are going to get smarter about consuming what?
--:-- Mike Volpe: I do think that that will begin to at least put some price dependency on demand. And that's more likely to happen in the near term. So those are the dynamics that are going to govern our industry and everybody that sits in between — if you're a neocloud or you're an inference cloud or you're a model builder, you live between those two poles of supply and demand.
--:-- Now, for the infra itself, I think right now we still live in a world where you need to think of the entire piece of infra as a single system. It's quite different than the networking days of old, where you would assume a very variable amount of traffic going from one type of traffic to another, times of day, lots and lots of variability, much easier to statistically multiplex that workload. Today, you have lots of GPUs that are all working all the time, always connected with much more constant flows. Because the interrelationship between the various components are largely still proprietary — you're on NVLink or whatever — it's very hard to swap one thing out for the other.
--:-- In fact, today I would argue that probably the best architecture is the Google TPU architecture because it is literally thought of as a single system. An NVIDIA style architecture is migrating in the direction of where the TPU architecture is, but it still has a few pieces that you can kind of swap out. And then you have everybody else who's like, "I'm going to take a Cerebras chip and throw a wrist switch on top of it," and all that good stuff. At least in today's world, those systems have inefficiencies because they're not constructed as a system. They're more of a patchwork. I think that industry will evolve, but today people that can think of the whole thing as a single system and optimize the architecture of it through that have a competitive advantage.
--:-- Raghu Raghuram: Yeah. I mean, it seems we're moving from a card as the minimum unit to a rack as the minimum unit, or even a few racks in some cases.
--:-- Mike Volpe: A few racks. Yeah. That's true. The immediate part that they announced recently, that is the minimum unit mostly.
--:-- Raghu Raghuram: Well, but if you look at how Google does this thing with these toroidal, pass-through optical MEMS switches, it's literally like the whole data center is the minimum unit. So the row is the minimum unit.
--:-- Mike Volpe: Yeah, exactly. But my sense is that's where NVIDIA wants to go also.
--:-- Raghu Raghuram: Clearly. So the two dominant providers are all going to that systems approach versus... I mean, even the smaller startups are heading that direction. But jumping on that, how do you see AI chips evolving? Right now, Jensen, king of the hill, incredibly dominant. Google TPU, very successful, currently mostly sold by Google. It's expected to change. Not enough TPUs.
--:-- Mike Volpe: Not enough TPUs, exactly.
--:-- Raghu Raghuram: How is the future looking? Will we see more competitive silicon?
--:-- Mike Volpe: I think for sure. We all want that. And we all want to invest in it. But first of all, what I find interesting is that model architecture is becoming reasonably diverse. Some people are using FP4, FP8. Some people are using mixture of experts with lots and lots of experts. Some have fewer experts. Some are jamming up the parameter counts, some not so much. I actually think that there is an interrelationship between your chip design and the model that's optimized around it.
--:-- Raghu Raghuram: How do software co-design?
--:-- Mike Volpe: In some sense, to your point, is there going to be more diversity in chips? I think yes, because there is going to be more diversity in model designs. Depending on what kind of model you want to do, what kind of things you want that model to do, you can certainly select a different chip architecture for that.
--:-- Now, reasoning intensity has dominated to date what chip architecture should be, because reasoning intensity equals big parameter counts and big KV cache. That means you need to talk a lot between memory and chip. When the only way you can talk between chips are the edges of it, it's just less connections than if you can use the top and bottom of the chip. So I'm a big fan of 3D DRAM architectures. Some of the new stuff that's coming up with hybrid copper bonding is even more interesting because you can have even higher density. So I think that architecture has huge promise, so long as we continue to see these KV cache, in particular KV cache intensive use cases. It seems like the most value-added labor that occurs — labor or AI that occurs — is reasoning-driven labor. Therefore, that should favor architectures that have very good memory bandwidth.
--:-- We're investors in Cerebras. Cerebras did very well to go fast, but if the model doesn't fit in the SRAM, you need a lot of them.
--:-- Raghu Raghuram: Well, yeah, you need a lot. Because once you get off board of Cerebras, it's not going to go super fast.
--:-- Mike Volpe: Right. And also, because the chip, the wafer is so large, you're actually at a significant disadvantage in terms of the memory bandwidth once you go off chip.
--:-- Raghu Raghuram: So they run Sol on the Cerebras chip. That's a big model. So they've figured out how to fit it in. But my guess is OpenAI had to do some hard work to get that model to fit in the chip.
--:-- Mike Volpe: No, we are fans of 3D architectures as well, but the manufacturability of it is a lot more complex with those effects and so on and so forth.
--:-- Raghu Raghuram: No, I'm a big fan. I'm a mechanical engineer that studied thermodynamics.
--:-- Mike Volpe: There you go.
--:-- Raghu Raghuram: So heat is my friend.
--:-- Mike Volpe: That makes you one of the very few people who would say that.
--:-- Raghu Raghuram: So all the packaging divisions of TSMC and everybody else are calling you right now.
--:-- Mike Volpe: No, I agree. The packaging of those chips are going to be complex, and especially the thermal attributes of it. When we get to a world where you have both a logic chip and a memory chip in the same package, the different coefficients — well, they have the same coefficient of thermal expansion, but logic chips get a lot hotter. So the ability to put that all into the same package will be non-trivial, shall we say. Chips are hot again and they're getting hotter.
--:-- The other possibility, of course, is to keep them still separate but have optical interconnectivity between them. Are you believing that?
--:-- Raghu Raghuram: Yeah. I think yes. Optics though are very finicky things that need to be experimented with a lot. When you heat up optics, they don't like it. Photons don't like getting hot. In theory, you would like a substrate that had wave guides in it or something. But the minute the logic chip starts getting hotter and you get outside of the comfortable temperature range, you're going to have a variety of connectivity issues. So I think optics are interesting. If you can do it with copper, I'd do it with copper, is my view.
--:-- Mike Volpe: Even for scale-up racks, not just interconnect now, but even for...
--:-- Raghu Raghuram: No, no, I mean, obviously for scale-up racks, that's a different story. That just simplifies the cabling and so forth. I'm talking about it at the module. Inside the server.
--:-- Mike Volpe: Yeah, for when you start going top of rack and all that, you need it. Definitely, no question. You just don't have enough bandwidth.
--:-- So Cisco back then, most valuable company on the planet, early hardware leader in networking, which sort of powered all of the internet. At some point it wasn't most valuable company on the planet anymore. What triggered that? What can we learn from that?
--:-- Mike Volpe: What tends to happen when you have a bubble in demand, and perceived demand, is that customers over order because they know they're going to get cut back. Every part of our product line was out of capacity. We couldn't make enough switches. We had orders on top of orders, on top of orders. And we were, of course, putting orders on all of our supply chain. Because the demand felt so robust, we were taking the financial risk on those orders. We were like, "Here's the money, keep the money, and just get us this stuff as soon as possible."
--:-- Raghu Raghuram: What could possibly go wrong with that?
--:-- Mike Volpe: Yeah. So when demand dries up, it's not like people stop buying. They cancel all their orders. They're like, "Yeah, I ordered twice as much as I wanted to anyway, so I need half of half of that." So the orders went to like one quarter of what they were. You've taken the risk on the supply chain, and your demand falls through the roof.
--:-- I think we ended up in the December quarter. This is kind of memorable because it happened in December quarter. We are one month off. November, December, January. Usually what tends to happen with businesses that are a month off — December soft, and then the first week of January you start seeing coming back, and by the end of January, you've made it up. So we go through December, it's soft, and it's like, "No big deal, December is always soft." Week one, January, nothing happens. Week two, January, nothing happens. Week three, January, nothing happens. You're like, "Oh, shit." So the quarter came in, I don't know, maybe we missed by 35, 40%.
--:-- Raghu Raghuram: Wow.
--:-- Mike Volpe: Massive miss. For a company, by the way, that publicly traded for a decade, never missed a quarter. And then that's how it all corrected. It was difficult. Stock went from 80-some dollars a share to nine dollars a share, I want to say. So it was literally one-tenth the value over a period of six months. Very tough.
--:-- Raghu Raghuram: So look, every time when the workload changes enough at the top layer, the underlying infrastructure of software and hardware needs to change with it. With AI, we've seen this. We talked about compute. What's going to happen in networking? I mean, we've seen big changes in networking. We no longer have one network. We don't have suddenly the second backup network. Where did this come from? Did you just say Ethernet is dead?
--:-- Mike Volpe: There we go. Ethernet is dead.
--:-- Raghu Raghuram: No, no, no. I mean, Ethernet is a framing mechanism. It's coming back. It was dead, right?
--:-- Mike Volpe: Here I think that the core tenets — the world of telecom started with circuit switching. We made the phone, and the idea with the phone was that you had starting point A, starting point B, constant bandwidth between two, you connect the line and off it goes. Then we found out, well, this is reasonably unreliable because one line cut and you lose it. And obviously it doesn't scale very well in a world where you have uneven traffic patterns. So then we decided, let's go to packets. Packets are very convenient because they allow stat muxing, which is the big, big benefit of it.
--:-- Raghu Raghuram: David and Andy back then, right? Cheerleading it.
--:-- Mike Volpe: Yep. So that and variable length packets and IP packets essentially. So the world grew up building networking equipment assuming that you had uneven arrivals and departures into a device that spread out the workload. You would under-provision the device to connect more bandwidth because not everybody communicates at the same time through the thing, and the outputs of that inbound are highly variable. That's the internet. That's the workloads that we live with. It was almost dogma for a long time.
--:-- Fast forward today. What does an AI data center look like? It does not look like that. You don't have random arrival. Links are consistently fully utilized. GPUs want to be fully utilized. Traffic patterns are much more predictable than they ever were before, particularly in training. They're incredibly predictable. They're less predictable in inference, but you can see the traffic path, especially if you do good batching.
--:-- Raghu Raghuram: Good chunk of it is still very predictable, right? Especially if you do big batching.
--:-- Mike Volpe: Exactly. You can definitely make it more predictable. So the fundamental tenet of the workload has changed. Yet, if you look at the dominant networking architecture today, which is the Broadcom Tomahawk — the DUN acquisition that they did — it's a stat-muxed packet switching architecture.
--:-- Raghu Raghuram: Yes.
--:-- Mike Volpe: So I think that at some point for data center networking, we're going to have a new kind of architecture that assumes a high degree of predictability and full utilization, as opposed to an unpredictable, stochastic stat-muxed approach.
--:-- Raghu Raghuram: So essentially back to circuits is what you're saying.
--:-- Mike Volpe: Yeah. And optical circuit switches.
--:-- Raghu Raghuram: Or time division multiplexing works as well, right? Or combinations of those. You can have many modalities of multiplexing, but they're not stochastic. They're constantly timed somehow. You can do TDM, you can do whatever else you want.
--:-- Mike Volpe: So I think these architectures — the issue is the switches you buy today have the correct amount of bandwidth and the incorrect architecture. Not incorrect, but less adapted architecture. I think the data center switches of the future at some point will look more like that. Potentially you could do it with MEMS.
--:-- Raghu Raghuram: To bring ATM back into the data center.
--:-- Mike Volpe: But even ATM had stat-muxed stochastic cells. Straight up circuits.
--:-- So now, networking vendors, including my former employer, this is bad news. Because when things don't change, you can't charge a lot of money for them. The whole point of a switch is that it's a complicated animal to have a statistically multiplexed traffic flow. It's a hard thing to have — one packet came in here, I have to throw it into memory, I have to do a lookup, do the lookup, I got to send it out of this gate. That's hard to do. So you make the thing expensive. But in a world where it's relatively static, you don't need that much intelligence. Don't need buffers anymore. Bam, send it right through. So I think that there's an opportunity for somebody to sort of rethink network architecture given this workload and make it way cheaper. It's classic innovator's dilemma because no matter what vendor you are today, this is not a good trend. You're not going to want to do that. But if a newcomer says, "Forget all that stat muxing shit. The world's different. I'm going to go with my thing," you might be able to build something interesting. It could be optical. It could be electrical. It could be a combination of the two. I think we're going to see what will emerge, and you can see clues of that from how Google does its network.
--:-- Raghu Raghuram: I think Amin has been the pioneer of specifically optical circuit switching. They were the first ones. That's for their toroidal thingamajig.
--:-- Mike Volpe: I don't understand why it's so slow, by the way. It takes them like a minute to switch.
--:-- Raghu Raghuram: I think to some degree it's because of the use case. Right now you use it to set up fairly static. When it is static, then you can...
--:-- Mike Volpe: Fair. For availability, if you don't want to...
--:-- Raghu Raghuram: Give me a few milliseconds, not a second.
--:-- Mike Volpe: But you probably could, honestly. The technology, I think, is there. It's just the use case is not there.
--:-- Raghu Raghuram: So I've been looking at a bunch of these, whether it's MEMS or Max Zander thingies, the MZIs and whatnot, to see if there's something in there. But somebody has to emerge with an architecture. And then the issue is going to be who embraces it, because you can't just build a switch without the rest of the system coming together.
--:-- Mike Volpe: The universe dilemma will extend to the user's inertia as well.
--:-- Raghu Raghuram: Exactly. At this point, the network, coming back to the "ship me a row" concept of infrastructure today, the network has become an integral component of your architecture. So this innovation to some degree almost has to come out of the GPU vendors.
--:-- Mike Volpe: But it's interesting, right? Because there's only NVIDIA, maybe AMD will get there at some point, but only NVIDIA and Google right now have the ability to ship you a rack system.
--:-- Raghu Raghuram: That's right, yeah.
--:-- Mike Volpe: So today they would have to adopt it. It'd be interesting to see, as more chip vendors evolve, is there an alternative architecture that they can all agree on? Sort of the shit that we did back in the days.
--:-- Raghu Raghuram: Back in the day, right? So you have two dominant vendors. The ITF 2.0.
--:-- Mike Volpe: Well, they have two vendors. They're dominant. They have their own approach of doing things. They're not likely to shift off of that.
--:-- Raghu Raghuram: And can you have a collection of everybody else that says, "No, actually, this is a standardized network architecture that we're all going to agree upon, and that's what we're going to build"?
--:-- Mike Volpe: The Rebel Alliance. I like it.
--:-- Raghu Raghuram: The buyers have to agree. That's the case.
--:-- Guido Appenzeller: So, Mike, you've seen a lot of these cycles now, some bigger, some larger. You've seen the last really big one. How does this compare?
--:-- Mike Volpe: As you probably know, I've started a new venture fund called Anabi Capital. Technically speaking, I retired in 2023 and I was supposed to get a yacht and play golf. That didn't work out very well, did it?
--:-- Guido Appenzeller: Why?
--:-- Mike Volpe: I actually think this is the most fascinating time to be doing our job and to be in our industry. Not just because the change is big and we all know how transformative it is, but because it requires more intellectual exertion to understand what next. You don't have a crystal ball. It's not obvious what the next move is. It's hard. So you have to keep thinking, and you have to think at a systems level. I don't think I've ever been in an industry before that I had to think about how light gets concentrated on a Zeiss mirror to figure out whether my chatbot works well. It's this amazing interconnection of things where there's bottlenecks and this changes and that changes.
--:-- I find it just the most intellectually fascinating time. And because the end prize is so significant, by figuring out how these different parts unlock, you can create a company here and a company there. This one can become worth a trillion dollars, and this one can become 500 billion, and so on and so forth. Honestly, I did not expect to encounter another time like this in my career. By far, I find this the most dynamic and interesting time to be doing what we're doing. I think we should all feel quite fortunate about it.
--:-- Guido Appenzeller: This tops 99?
--:-- Mike Volpe: Oh, yeah. Easily.
--:-- Raghu Raghuram: Totally, totally agree. And we are all in the same business for very much the same reason. This has been a fascinating hour. Thank you so much. We could be continuing this conversation endlessly. Perhaps we'll do it another time. This was super fun. Thank you guys.
--:-- Mike Volpe: Thanks for coming, Mike. We appreciate it.
--:-- Narrator: Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode.
