We live in a world where demand for AI capacity tokens is near infinite and supply is incredibly constrained. So another way to put it is we all just live in Jensen's [music] world. >> You were doing back then M&A for the most valuable company on the planet. >> The retro on the M&A process that I think worked well for Cisco was we were not afraid to fail and we did it with high enough frequency that we got a few outliers. You bought this thing for 500
million and like it's doing a billion in earnings 2 years later. >> When you see the AI buildout, how do you look back on your previous experiences? >> If you look at the revenue ramp of Anthropic, OpenAI are definitely experiencing growth rates that were far superior to what we saw at the time. Do I think the buildout that's happening now is useful and will be fully utilized, if not more, 100%. Will it be owned by the same people that own it today? Great to have you
here on the podcast, Mike. And is a very very opportune time to have you because you were one of the architects of the great internet buildout when you were at Cisco and uh now of course we've got the great AI buildout happening. So uh Vito and I and the audience would love to get your thoughts on then and now uh from all different dimensions from the actual buildout the capitalization of it the technology differences etc etc. Oh, so let's start. For those that don't uh know
or remember, uh Mike was a senior exec at Cisco, overseeing a lot of the M&A and strategic moves that Cisco made in the late 90s. [snorts] Um and of course that was the time the internet got built out. So Mike was in the middle of all of it. Um so clearly that let's start from there. So you were there when the internet started taking off. And then you guys rushed into to do a lot of great things and the industry rushed to build a lot of
bandwidth etc etc which for some time was not useful and then it turned out it was actually too little right >> and there are some loose similarities to what's going on now. So when you see the AI buildout how do you look back on your previous u experiences? Well, first of all, it's an absolute privilege to be here with two legends here, both you, Ragu, and and uh in the case of uh Gido and I, we got the chance to work together at least once. So,
so um it's fantastic to be here and uh as you said, an interesting time because you know, history doesn't repeat itself, but it definitely rhymes. >> Yes. >> So, so um so yeah, there are some similarities. um maybe useful to give some context on some of the observations of the late '90s buildout and what some of those things might apply in today's world uh versus not. Some of the kind of similar themes was that we we were building out this technology called the internet and the
web and all the things that wrote on the internet. We had expectations that this would be massively transformative to the world and we imagined a wide range of applications that would ultimately massively benefit from this buildout. Applications were different media types, right? So, first we just had pictures and text and then we had audio and we had video uh and then we had like high quality video applications like commerce, applications like social networking. All of these things were imagined in the '90s uh but didn't exist.
Uh in parallel with that there was a very big surge in demand. At a minimum there was a surge in demand for connectivity. >> Yeah. >> And on top of that a reasonable surge in demand for uh for bandwidth. Um and uh in parallel with that there was a great influx of capital of two shapes and forms equity and debt. Um so uh if you look back at sort of what happened first of all in in the bandwidth world uh and in particular in fiber optic
bandwidth the elements the increments of increase of the amount of bandwidth are quite different than compute >> that's because you know we invented this technology called WDM that essentially takes a piece of fiber and said if you used to put you know one gigabit on there or 10 gigabits on there by putting in more colors of light in in that same piece of fiber, you can now have 40 times or 80 times that capacity. So essentially capacity increased quite dramatically with the advent of technological change.
That's actually different because compute ultimately is bound by something that resembles Moore's law. So the pace of improvement uh uh is is more mo more moderated hard to believe but it is actually more moderated in compute than it was at the time. Uh and the other component that is I think different is that the practical applicability of AI I believe today is higher than the practical applicability at the time of network bandwidth. >> Yep. >> Uh and that represents itself by the revenue ramp of the
providers. So if you look at the revenue ramp of uh anthropic open AAI I don't know what the numbers look like within within Google uh even the providers of the open source models like fireworks together modal etc etc are uh definitely experiencing growth rates maybe not quite at the same margins but certainly growth rates that were far superior to what we saw at the time. >> Yeah. Now uh then what happens uh uh at some point there's a shock that some of these businesses that are
built on the internet are not good enough uh and then that ripples through the industry and ultimately the players that were most saddled with debt went bankrupt. Um now it's interesting to note that that debt was buying assets was fiber optic cables and those assets actually turned out to be quite useful in the future. >> Exactly. >> Except they were operated by somebody else because the guys that went bankrupt didn't have them anymore. >> Yeah. >> So uh the first thing to recognize that's the underlying
assets that were built at the time of bandwidth and I suspect this you know compute this time >> cycles. Yeah. >> It'll be useful. There's no question that it'll be useful. Whether it will be owned by the same companies or not is an interesting economic question. So, you know, that's a long-winded answer to say, do I think the buildout that's happening now is useful and will be fully utilized, if not more, 100%. Will it be owned by the same people that own it today? lot of
interesting dependencies and a lot of things that in fact are outside of the control because they're they're macroeconomic and geopolitical and so forth. it's really outside of the control of the builders themselves. And in particular, when I talk about the builders, >> uh, you know, some of the balance sheets are pretty healthy, right? It last count when I was talking to the folks at Nvidia, there are 200 NeoClouds and those balance sheets are not good. >> Yes. So, so I'm going to guess that there's uh
I I'll predict that there will be some of course >> yeah some of them better than others but broadly I expect that there will be an economic correction uh in the market uh but the underlying assets will be quite useful for a long period of time >> isn't there one other big difference from how was back then which is that everything is now much much larger I mean back then we were investing a small fraction of a percent in in infrastructure for for the internet today
we're investing 2% or so of our GDP, right, in in AI infrastructure. >> Yeah. Uh well, it is much bigger, but it's an interesting concept, right? Because in the 100,000 ft abstraction, what is it that is being built, right? Back then it was communication that was being built. Today, we're building intelligence. And the economic value of intelligence is probably higher than the economic value of communication. probably. Yep. >> Right. >> Um so, uh so, so I think it's it's unsurprising. I also think that that believe
it or not, I mean, it's been nearly 30 years, the markets are more efficient now, right? Which is why so much more risk capital is willing to make its way uh into AI, which you know uh even back then uh was not able to. So uh net is I'm actually more optimistic about this cycle but I also believe that there will be some correction and some cleaning up of the industry structure. >> The role of it has also massively increased right back then I mean in
the late 90s PCs were were somewhat common but but not ubiquitous. Today >> so to speak software has eaten the world and it's sort of for any company out there. It's a very central part of what they do when >> cost of distribution of intelligence and >> Oh for sure. And not just any company but like you know seven billion people have a supercomputer in their pocket. >> Yeah. Exactly. >> So so you know there's there's you know there's m much more evenly spread >> uh
consumption of of the uh the underlying uh benefit that's being produced. >> Yeah. Uh don't fully recall all the puts and takes in the 90s in the early 2000s [sighs and gasps] on the financial side here. Obviously, one of the recent shocks of the system was the rising interest rates >> and the scale of capital causing people to say, "Hey, this is going to crowd out everything else and we may not even have enough capital during the bandwidth buildout and the internet buildout." That was not
the case. Correct. There was no sudden increase in interest rates, was there? >> No, there wasn't a massive increase in interest rate. There was a couple of big differences, right? One one is that more I would say on a proportional basis I believe more of the capital came from the balance sheet of the equipment vendors >> Mhm. >> than today. Today it's a more uh capital markets driven uh buildout. So it you know it's private debt, it's public equities, etc., etc. There is Nvidia vendor I
don't think you're supposed to call it vendor financing but it looks a lot like it. [laughter] Uh, so there's a there's a little bit of that, but and there's some corporate financing, but it's largely equity. >> Uh, and it's not tied to specific equipment purchases and so forth. So, I'd say the capital structures are a little cleaner now. Um, the the one thing that I think interestingly though favors the '9s is a lot of the money went into the ground in the form of a fiber
optic. And when you put a fiber optic in the ground, it's good for a long long time. It's like it's a usable asset for 20 30 years. The I I think one of the deterministic factors is the compute you put in the ground. What is its actual life? And I I this is a fascinating phenomenon, right? Because the dominant part of that compute is is is chips. It's it's it's GPUs. That's 80 plus%. Yeah. >> Yeah. It's the big chunk of it. All right. So, I
remember back when people were very excited about Hopper H100's, right? You were you're they were like, "Okay, well, maybe it has a 5year useful life and you're renting it out at a, you know, buck 80 $2 per GPU hour." Okay. And that was four years ago. So, it the chip should be defunct now, like near defunct. Not at all defunct. >> Nope. >> Not only that, it rents at a higher rate. >> Yeah. It's it's 250 or three. So the the the kind of the interesting
part is we thought this asset that have a 5-year life partly because of AI itself is a useful asset for quite a bit longer and in that cont if you're in the business of financing an asset and that asset ends up having a much more a longer useful life then you're the beneficiary of that because your business models were constructed on a short so in some sense I I actually think this the the fact an unintended consequence of AI being good at software is that the
underlying asset life is getting longer thereby helping it to be more financable and that's an interesting phenomenon that I didn't quite expect >> when this thing gone underway >> although the investors are not the dead investors have not broadly accepted the extended life of a GPU but the new cloud's fairly >> I'm not sure I'm not sure I have either I mean like right now it seems what's mostly happening is that we a massive scarcity of an asset. So even previous generations are still worth a
lot, right? If we ever get to a point where we have uh you know an abundance of of GPUs that might completely change. You might go you might go back to those numbers. >> So I think the key metric is is is uh uh how much token revenue can you extract per watt. Yep. >> Right. And >> maybe yes. So the the hopper is going to be a lot worse than black well is going to be a lot worse than Vera Rubin, right? They're all just
more efficient at getting dollars out of a watt. So in theory at some point when you have sufficient Vera Rubin, it doesn't really make sense to run H100s anymore because like it's just the worst way of converting electricity into dollars. >> Correct. Right. But the problem is right now that the demand is so high that even if the watt to dollar conversion is low, you're going to still use the asset. So uh and right now the scarcity is so significant and ultimately you know at some
point in the conversation maybe we get there but ultimately the scarcity is in the manufacturing of the semiconductor itself >> and that doesn't look like it's changing anytime soon. >> Correct. Yeah. >> And so then scarcity persists then the life of the asset continues to be long. Yeah. >> Yeah. >> Yeah. I think it's correct. Um, >> yeah. And the other big difference, there was a period of time where there was unused bandwidth. Today, there's no such thing as an unused GPU, >> right? >> There
there's a bit of unused energy and like you could probably, you know, I don't know if we're going to talk about networking, but with better networking, you could get more flops out of a given GPU. Yeah, I was just going to say the MFUs were a factor in certain use cases and uh how the people have built it, but uh but certainly generally not. >> Yeah. >> Back when you were at Cisco, were you still there when Cisco was the most valuable company on the planet?
>> I Yes, indeed. I was there. >> So, you were doing back then M&A for the most valuable company on the planet and it worked pretty well, I would say, looking back, right? >> I was very happy with it. Yeah. Um, what would be your words of advice to Nvidia in uh in 2026? Like that whoever runs M&A there isn't exactly the same spot as you are. Correct. As you are. >> Yeah. Van's in the same spot I am. Um, look I the the retro on
the M&A process that I I think worked well for Cisco was that fundamentally uh we were okay with making enough acquisitions that we got a statistically significant outcome on those acquisitions. Meaning, some worked really well, some work so so, some didn't work at all. And you know if you kind of drew it out one one in each of those three sectors, a third of them worked really well, a third of them were whatever and a third of them failed. Um the benefit that we did is
we were not afraid to fail and we did it with high enough frequency that we got a few outliers. uh Andybetoshime's company, uh uh Mario Masa's company, uh a bunch of these things that were sort of, you know, even for us were venture-like returns. So, you bought this thing for 500 million and like it's doing a a billion in earnings two years later. [laughter] >> That's amazing. So a and so it ended up that during the period that I was there uh by I joined in
'94 and I was like the one bisdev hire in 2000 uh in 2000 roughly uh 50% of Cisco's revenue came from acquired sources. So the that was incredibly useful and we actually never really spent much more on a given acquisition than four or 5% of our market cap. >> Mhm. Most of the acquisitions were actually quote cheap relative to our market cap stand. >> How much would they mean for Nvidia? I don't even want to think about that. >> Yeah. Well, my my rationality at the
time was like if your stock trades more than the pay price you paid for the acquisition in one day, then you can do the acquisition as many as you want. Nobody [laughter] nobody will care. >> Both of them done with equity or was >> all equity. >> All equity. >> Almost everything was equity. A handful of the small ones were cash but >> and Cisco I mean they became major product lines. So it was not like this was a little tuck in that added a feature
somewhere but this was of a new product line >> that in some cases became the leading product line in the industry. >> Now you you're a networking guy so you'll appreciate this. The one thing that's a little different about a networking acquisition versus other ones is that networking operates on open protocols, which means that I could buy a switch company made by somebody else and it immediately talks to my switches or routers whatever freely because all the protocols are open. So, you know, at the Cisco,
we had a couple of proprietary protocols that we enjoyed, but by and large, all the OPF, B, BGP, ISIS, Ethernet switching, 802, whatever, they all worked. So, you could immediately integrate the product into your sales team and have your sales team like literally just change the badge on the thing. And so, that allowed us to be like super high velocity. Okay. How does that contrast to to uh Jensen's world today? one, they've made some really clever acquisitions, you know, like Melanox is the world's largest. Yes.
>> Right. It's like genius. >> Now, they're making other acquisitions that sometimes I scratch my head. I'm not entirely sure what Hugging Face is all about, for example. Like maybe there's a rationale. Hard to me to see it. I don't think it matters. I think it's fine for them to do some of these and take risk and some of them will work. And if they work as well as Melanox, holy cow. And some of them are not going to work. And I think the market has
granted them a moment of incredible optionality, right? Ba basically, they get to do whatever they want, which is the optimal moment for them to take these risks. So I think they're being very clever and you can always critique any given acquisition. Trust me, I got critiqued endlessly at Cisco cuz I bought Strataccom and not Cascade and I, you know, bought an optical networking company from Venode that went nowhere and whatever. But in the end, you're like, "All right, well, I I did I did 70 I
think our team did 70 acquisitions. I was there and those turned into half the company's revenue and we were doing 30 billion in revenue." So, it's 15 billion in revenue. uh the strategy worked and I think that I think that um uh the team and Nvidia is is is doing that um I think that's interesting and wise the degree to which they are financing the industry we'll see >> yeah yeah >> because that's different right that that in that sense Nvidia is basically I think they're
basically trying to shape what an industry looks like >> yes and I think industries are powerful and they find their own shape. It's hard to reshape them. >> Let me let me let me defend Nvidia a little bit. Having been at Intel and and having seen how the big hyperscalers essentially disintermediated Intel from the end customers, right? And at the end of the day, they could build better chips than Intel just because Intel had no idea anymore what the customer was doing with the chips, right?
I I I have some sympathy for wanting to have more than three customers to sell to or four. Maybe as do I. Yeah. So, and you could argue what is it efficient capital allocation strategy is it given it all [gasps] back to investors or could it be at this time which is probably once in a time in history put it all to work on future innovation right so >> well I think both of you make very valid points um the thing I would say Greo is
that an industry wants to look a certain way >> because when it does it creates maximum utility maximum economic value. Why are there three hyperscalers? Cuz it's a business where scale matters. >> Software. Yeah. >> Right. It's a business where like scale matters and therefore the industry wants to concentrate the the no matter how much you finance them, the little ones are always weaklings. So, and obviously if you're invida, you don't like that. You don't want three customers. You want 50 customers. So, you fund them,
but at some point they're going to die. it just because the industry whenever it's a scale business wants to consolidate. You know, we could fund a hundred airlines in America, but in the end, you're going to have three. So, I I my point is not that the um not that it's a bad idea, it's just that it's futile. >> Yeah. [laughter] So, so, so noble intentions, but uh but uh >> now to your point, like uh it could be a very good return on I don't
know that they I don't know if they actually care about the return on capital. They they they they might um and if they get out at the right time. >> Yeah, it's not even about the return on capital. It's >> I agree. It is it is to some degree shaping industries uh I mean shaping the deciding the shape of the industry. But on the other hand, it is uh enabling a lot of innovation as well because our startups and lots of other startups do get access
to that capacity now. >> Yeah. Right. >> Yeah. Yeah. >> Um >> no no look we're overall I get why they're doing it and and you know many of our companies are beneficiaries of it. So thank you Jensen. >> Exactly. [laughter] So um I mean at VMware we did probably not as many as you did but maybe 15 plus acquisitions over the full lifetime of the company. And uh the observation was whether it didn't matter whether you were acquiring a 20% company or a 2,000 person
company. The amount of postacquisition work in terms of integration culture business model sales team etc. etc. was all always a lot of work. How did you guys pull it off at that time at Cisco? >> Um, okay. Structurally the way we did it was that we had a a very small when I say small less than 10 people uh deal team that was job was to acquire and we had a somewhat larger uh integration project management team that had sort of rules and guidelines about how
we did integrations and uh uh they they had colleagues those people that project managed it had colleagues in the various business units and divisions and functional units within Cisco that collaborated with them. So the important thing to recognize there is that uh comp businesses companies are good at processes. Uh they're very bad at one-time events. And so essentially if we were going to do a large number of acquisitions, we needed to have repeatable processes that were uh performed over and over again. And we had a
centralized group that was the administrator of those processes. Um, more often than not, and in the beginning at Cisco, we did this a lot. There was nobody that was formally assigned to this. So, it' be like, Ragu, it's it's it's your job. You you go have fun with that integration. Let me know when it's done. >> Right? And you're like, where where do I even start? You know, that was like the modality. And we're like, okay, this is bad. So, we we created this process and
it and it worked pretty well. Um where it got challenging is we liked to buy small things a and small things became too small to become significant to the overall business because the business got so big >> that like you know it just didn't make a difference anymore. And then when you tried to this was a little bit after my time because I I stopped doing the M&A stuff in 2000 and switched over to running a business unit. But uh after my time they started buying
bigger things and then you get into the problem that you're talking about. big things are very very hard to integrate. They have they're they've been around a long time. They have a long culture. They like, you know, we bought like Scientific Atlanta, which was a disaster. We bought like a bunch of things that >> didn't work very well. We also got into the mode of like, oh, we're not fully integrating them. We're like partially integrating them. Yeah. >> It's terrible. >> Yeah, it it doesn't work
very well. So um so so in other words the mitigation is to have a process but then when you stop respecting the process you do have big problems like you talked about. >> Yeah. And what about from the portfolio company side? I mean you got portfolio companies we've got portfolio companies that uh at least in recent memory we have had some exits into being other large companies buying them. Two questions there. One is if you're a portfolio company CEO, what would you watch out for? Right?
And then sitting on the boards of these companies, how do you advise your portfolio companies these days >> about exits? About exits in general? Um, look you when you're when you're in our business of being on boards and you're facing a situation where you need to sell a, you know, there's an opportunity to sell a business, it's one of three outcomes. One, your company's doing great and it's a really bad idea to try to sell it. So then you try to convince the CEO, no, don't
do it. Two, the company's struggling. We don't have a lot of finances. It's a nice way to get out. Everybody's happy. then you just help the deal happen and you just sort of manage a process and the then you have the tweeners uh where you're not totally sure and uh I think that there are a couple of important indicators that help you determine what to do in that moment. The first is do does the CEO, the founder, the founding team have the fire like do they
want to do this? Do do they do do they believe they can get through this sort of intermediary period and uh uh bust open into the next phase? Do they do they want it? Because if you have a CEO or a leader that's kind of like, man, you know, maybe I should consider this. I'm a little tired. It's been a long haul. >> Yeah. >> Then let's just sell this thing and and kind of move on. Um and then the second factor is you have to
be pretty cold and objective about like is this going to be a interesting enough business. Not like will it double from here because at the end of the day if it's double I'll take the money today but will it 10x from here and that I would say gets to be a really hard decision because it's a little bit emotional also for you like you're you know you're not a founder but you know you've been working with this company for four, five, six years your heart's in
it. You you've told your good friends to go work there and and yet you have to step step away from it and go like objectively this is not 10x from here and then you help steer the ship. But I think the most important parameter is where's the founders's head at? >> Yeah. >> Yeah. and most so there is a certain element of mystery or unknown for a lot of the portfolio company founders on how do these things actually happen right people think one day somebody calls
another company and say hey I want to buy you versus and then there's other thing where people build relationships for a long time how did you approach it at Cisco and how would you advise your portfolio companies even if they're expecting a 10x. >> Yeah. >> Do you just call them up, say, "Hi, I want to buy you." Is that >> No. Well, obviously you do as as good a job as you can building a network and knowing what's going on. I I mean, it's basically
cataloging the inventory is the way we used to think about it, but every company in the market is a potential acquisition. So you catalog them and you know where where are they in their growth rate, what kind of founders they have, who are the backers, what are their aspirations, who are their customers, just like be informed. So and the and uh you know uh the business the corporate development group had to do that but I kind of considered it everybody's job at the company to be
constantly informed about what's happening around them. So uh and then essentially uh we at the company we would try to weave at Cisco we would try to weave in the inventory against a strategy statement that was there for example we're a router company but there are going to be more switches than routers therefore we need to have a switch what are the switching companies that are available right or optical networking is becoming important because of bandwidth blah blah blah what are the companies that are available.
You often have the the other way where companies come to you and says, "Hi, I'd like to get bought." I think broadly a bad idea, right? Because there's either an auction going on uh or you should have thought about it earlier if you or were interested in it, but basically you almost never pay attention. We never tried to pay attention to inbound. >> Uh flip side, you're you're now a small company or a startup and h >> how do you think about that? I always encourage
always the founders of our companies to talk to big companies to talk to competitors and to always be in dialogue. Uh some people will say like well you're shopping yourself. Uh no it's an inverse intelligence gathering mechanism. I even encourage my CEOs to talk to their direct competitors go to coffee. All right. Like yeah you if if you're clueless you might leak some information towards them. [laughter] Hopefully, you're not clueless and they might tell something to you that's interesting or they might shape your view on
an industry or something. So, >> I encourage people to be out there always talking and seeing what's happening in the market. And um usually you kind of have a pretty good idea of like, okay, my company's struggling a little bit. I think I'd like to sell. I kind of already know then because I've been in conversations with who's where. So I I would say uh you know don't a CEO of a startup shouldn't spend their entire day talking but going on one or two coffees a
week with you know the powers that be at Nvidia the powers that be at Amazon and open AI etc etc when the right people are talking do it it's fine you the information exchange you don't lose that much in that they're already thinking about it uh but you gain a lot by having that relationship >> I think this is really good advice I I keep telling my portfolio CEOs the same thing if If you don't have the cell phone number of the CEOs of each of
your competitors, that's a problem. There will be a day when you need it, right? Uh >> absolutely. >> So you it's it's a network you need to build. >> Yeah. Yeah. Now enough of this M&A stuff. Let's get back to actual technology. You are spent 30 plus years in infrastructure. >> Very few people know more about infrastructure than what you do what you've done in the past. Summarize the landscape for us right now. when you think about infrastructure opportunities versus where is the technology gaps that
you see and um what founders should be really looking at >> well super interesting topic you know I would say you you want to think about I think the current in the shape of this industry in two dimensions right one is the vertical stack that starts with you know EUV equipment and goes all the way up to AI applied AI and applications and everything that sits in between that sandwich >> and then you want to think about particularly the infrastructure side what's that what what's sort
of the the va the value chain of it um the first thing you would say is right now we live in a world where demand for AI capacity tokens whatever is near infinite and supply is incredibly constrained so another way to put it We all just live in Jensen's world. [laughter] >> He's he's king of the hill. There's no question. >> Um um I I think uh the shape of the industry will begin to change when either the de the characteristics of demand change or the
characteristics of supply change. And I think that my my view is right now the the supply side because so much of the limitation well there's there's some there are some power limitations for sure but the dominant limitation is silicon and um I would say that unless there's more fab capacity that doesn't change which is means it's not likely to really change in the next two or three years maybe 5 years out you could imagine that there would be more fab capacity you could put more wafers
through >> the same machines or you can make more ASML machines and so forth, but that's probably a governing factor of our industry, which means we live in a reasonably scarce world and those who have fab capacity are actually kind of kingmakers of of the infrastructure universe. >> Uh on the demand side, I do think that the world is going to get smarter about the type of tokens they consume. I think we've been in a kind of an era over the last couple years that everybody
just wanted the super high-end Frontier token. And I think the topic of conversation today is model routers and how do I move stuff to, you know, the economic difference between a top-end Anthropic token and a top-end Kimmy token is like 5 to one or 7 to1 or whatever. It's a massive economic difference. So people are going to get start smarter about consuming what uh and I I do think that that will begin to at least put some price dependency on demand and that's more likely to
happen in the in the in the near term. So uh those are first of all I think the dynamics that are going to govern our industry and everybody that sits in between you know if you're a neo cloud or you're an inference cloud or you're a model builder you leave between those two poles of supply and demand. Now for the infra itself um I I think right now we still live in a world where you need to think of the entire piece of infra as a
single system >> right so uh it's quite different I think the networking days for example of old where you would assume a very variable amount of traffic going from one to b types of traffic, times of day, lots and lots of variability, >> much easier to statistically multiplex that workload. Um, today basically you have uh lots of GPUs that are all working all the time, always connected with much more constant flows and you you uh because the interreationship between the various components are largely still proprietary,
you know, you're on NVLink or whatever, it's very hard to like swap one thing out for the other. and and the uh in fact today I would argue that probably the best architecture is the Google TPU architecture because it is literally thought of as a single system >> and I I think an Nvidia style architecture is migrating in the direction of where the TP TPU architecture is but it still has a few pieces that you can kind of swap out. >> Yeah. Um, and then you
have everybody else who's like, you know, I'm going to take a Cerebbrus chip and throw an Arisa switch on top of it and like all that good stuff. And at least in today's world, those systems have inefficiencies. Uh, because they're not constructed as a system. They're more of a patchwork. Um, I think that industry will evolve, but today people that can think of the whole thing as a single system and optimize the architecture of it through that have a competitive advantage. Yeah, I mean it seems
we're moving from from a from a card as the the minimum unit to a rack as the minimum unit or even even a few racks in some cases. >> Yeah, I would that's >> a part that they announced recently >> that is the minimum unit possibly. >> Yeah. Well, but if you look at how Google does its thing with these like tooidal pass through optical mem switches and like it's literally like the whole data center >> is is the minimum unit. >> So the row is
the minimum unit. [laughter] >> Exactly. Yeah. >> Yeah. But I I my sense is that's where Nvidia wants to go also. >> Clearly. >> So the two do two dominant providers are all going to that systems approach versus >> I mean even even the smaller startups are are heading that direction that jumping on that. How do you see how do you see AI chips evolving and right now Jensen King of the Hill incredibly dominant right? Google TPU very successful. Um currently mostly sold by Google. It's
expected to change. Not enough TPUs. >> Uh but not enough TPUs. Exactly. Uh how how will the how's the future looking like? Will we see more more competitive silicon? >> I think for sure. >> Um >> we all want that. >> Yeah, we all we all want it. Um and we all want to invest in it. But I I would say I would say first of all what I find interesting is that model architecture is becoming reasonably diverse. >> Right? So you have uh people some
people using FP4 FB8 uh some people are using mixture of experts with lots of lots of experts uh some have fewer experts some are jamming up the parameter count some not so much and I actually think that there is a inter relationship between your chip design and the model that's optimized around it >> right so >> how do software code design >> in some sense to your point like is there going to be more diversity in chips and I think Yes, because there is going to
be more diversity in model designs and depending on what kind of model you want to do, uh what kind of things you want that model to do, you can certainly select a different chip architecture for that. Now, uh reasoning intensity has dominated to date uh what chip architecture should be because uh reasoning intensity equals big parameter counts and big KV cache. >> Yep. Uh and that means that you need to talk a lot between memory and chip. >> Y >> uh and you know when the
only way you can talk between chips are the edges of it, it's just less connections than if you can use the top and bottom of a chip. So uh I'm a big fan of uh 3D RAM architectures. Um and uh some of the new stuff that's coming up with hybrid copper bonding is even more interesting because you can have even higher density. Uh so uh I think that architecture has huge promise so long as we continue to see these sort of KVach in particular KVach intensive
use cases and it seems like the most value added labor that occurs labor or AI that occurs is reasoning driven labor right so and and therefore that should favor architectures that have very good memory bandwidth. >> Yep. Um I we're investors in Cerebras. Cerebras did very well to go fast. Um but if the model doesn't fit in the SRAMM, >> you know, you need a lot of them. >> Yeah. Ex Well, yeah, you need a lot of Well, because once you get off off board of
cerebrra is not going to go super fast. >> Yep. >> Right. And and also because the chip the wafer is so large, the actual you're actually at a significant disadvantage in terms of the memory bandwidth once you go off chip. >> Yep. So, um, now they they run soul on on the Cerebrush chip. Uh, so that's a big model. So, they've figured out how to fit it in, but my guess is OpenAI had to do some hard work to get that model to fit in the
chip. >> Yeah. No, we are fans of 3D architectures as well, but the manufacturability of it is a lot more complex with all the effects. >> Yeah. >> So on and so forth. >> No, I'm a big fan. I'm a mechanical engineer that studied thermodynamics. So there you go. >> Um, she is my friend. [laughter] >> That makes you one of the very few people who would say that. >> Yeah. So, >> so we'll see how that >> the packaging divisions of TSMC and everybody else
are calling you right now. [laughter] >> Uh, yeah. Yeah. No, I agree. The packaging of those chips are going to be complex and especially obviously the thermal attributes of it. And then when we get to a world where you have both a logic chip and a memory chip in the same package, the different um uh coefficients of well they have the same coefficient of thermal expansion, but logic chips get a lot harder. So you know the ability to put that all into the same package will
be non-trivial, shall we say. >> Chips are hot again and they're getting hotter if >> the other possibility of course is to keep them still separate but have optical interconnectivity between them. Are you a believer in that? >> Yeah. Um I think yes. Uh optics though are very finicky things um that need to be uh experimented with a lot. So you know he when you heat up optics they they don't they don't like it. They don't you know photons don't like getting hot. And um I
mean in theory you would like have a substrate that have wave guides in it or something, but the minute the logic chip starts getting hotter and you get sort of outside the comfortable temperature range, you're going to have a variety of connectivity issues and so forth. So I I think optics are uh interesting. Um if you can do it with copper, I'd do it with copper is is my view. But if you if you if you >> Yeah. Even for scale up racks, not just interconnect
now, but even for >> No, no. For I mean, obviously for scale up racks, that's a different story. That's just simplifies the cabling and so forth. I'm talking about at the inside the server. Yeah. >> Yeah. No, for for when you start going top of rack and all that, you need it. >> Definitely. No, no. You just can't you don't have enough bandwidth. Um so >> so Cisco back then most valuable company on the planet early early hardware leader in networking which sort of power powered
all of the internet at some point wasn't most valuable company on the planet anymore. What what what triggered that? What can we learn from that? >> What tends to happen when you have a bubble in demand in perceived demand is that uh customers over orderer because they know they're going to get cut back. Y >> and so we every part of our product line w was out of capacity. We were we couldn't we didn't have we couldn't make enough switches. We couldn't make we had orders
on top of orders on top of orders on top of orders. >> And then what uh we were of course putting orders on all of our supply chain and because the demand felt so robust, we were taking the financial risk on those orders. In other words, we were like here's the money, >> keep the money and just get get us this stuff as soon as possible. possibly go wrong with that. Yeah. >> Yeah. So then >> when Deayan dries up, it's not like people stop buying.
They cancel all their orders. They're like, "Yeah, I never I ordered twice as much as I wanted to anyway, so I I need half an half of half of that." So the orders went to like one quarter of what they what they were >> and then you you a you've taken the risk on the supply chain and b your demand falls through the roof. And so I think we ended up in the December quarter and it particular this is kind of memorable because it happened in
December quarter. So we were one month off. So uh November, December, January. And usually what tends to happen with businesses that are a month off, you you know this, right? Like December's soft and then the first week of January you start seeing coming back and then by the end of January you've made it up. So we go through December it's soft and it's like no big deal. December is always soft and then week one January nothing happens. Week two January nothing happens. We genuin like oh
[laughter] >> So the quarter came in I don't know maybe we missed by 35 40%. >> Wow. >> Yeah. >> Massive for a company by the way that publicly traded for a decade never missed a quarter. >> Yeah. >> Uh and uh yeah and then that's how it all corrected. And it was it was difficult. Stock went from 80ome dollars a share to $9 a share I want to say. So it's like literally onetenth the value over a period of 6 months. Very tough. >> Yeah.
So so look at every time when the workload changes enough at the top layer the the underlying infrastructure software and hardware needs to change with it right with AI we've seen this we talked about compute right what's going to happen in networking I we've seen big changes networking we no longer have one network we now have suddenly the second backend network. Where did this come from? >> Did you just say Ethernet is dead? >> There we go. Ethernet is dead. Perfect. Ethernet is a framing mechanism
for >> Is it It's coming back. It was dead, right? [laughter] >> Here. Here. I think I think the the core tenants the world of telecom started with circuit switching, >> right? Like we made the phone and the idea with the phone is that you had starting point A, starting point B, constant bandwidth between the two, you connect the line and off it goes. And then we found out well this is like uh reasonably unreliable because one line cut and you lose it. Uh and obviously
it doesn't scale very well in a world where you have uneven traffic patterns. So then we decided let's go to packets and packets are very convenient because they allow statoxing. Yep. >> Uh which is the big big benefit of it. >> David and Andy back then right they cheerleading it. Yeah. >> Yep. So, so uh that and uh you know variable length packets and all of these you know IP packets essentially. So essentially the world grew up building networking equipment assuming that you had uneven arrivals
and departures into a device that spread out the workload and you would underprovision the device to connect more bandwidth because not everybody all communicates at the same time through the thing and the outputs of that inbound are highly variable. Okay, that that was that's the internet that's the workloads that we live with. >> It was almost dogma, right, for for a long time. Okay, so fast forward today. What does an AI data center look like? It does not look like that. You don't have random arrival.
Um, you links are consistently fully utilized. GPUs want to be fully utilized. Traffic patterns are actually much more predictable than they ever were before. Particularly in training, they're incredibly predictable. They're less predictable in inference, but you can see the traffic p especially if you do like good batching. >> But it's still very predictable, right? >> Yeah. Especially if you do like big batching like you you can definitely make it more predictable. >> So the fundamental tenant of the the the workload has changed >> yet if
you look at the dominant networking architecture today which is the Broadcom uh uh what's it not Trident tomahawk >> tomahawk >> uh which is the Dune acquisition that they did is a stat packet switching architecture. >> Yes. So I think that at some point for data center networking we're going to have a uh a new kind of architecture that assumes a high degree of predictability and a full utilization as opposed to an unpredictable a stochastic statox approach. Um so essentially back to circuits is what you're
saying and optical circuits. >> Yeah. So >> or I mean time division multiplexing works as well, right? or combinations of those. You know, you can have you can have many modalities of multiplexing, but they're they're not stochastic, right? they're they're uh uh constantly timed somehow like you can do TDM, you can do >> uh whatever else you want but basically so and I think these architectures the the the issue is the switches you buy today have the correct amount of bandwidth and the incorrect architecture not
incorrect but uh less adapted architecture uh and I think the data center switches of the future at some point will look more like that will look more uh like you know potentially you could do with MEMS >> to ring ATM back into the data center >> but even ATM had statmucks stocastic cells right straight up circuits >> um so now networking vendors including my former employer this is bad news >> because when things don't change you can't charge a lot of money for them right like
the whole point of a switch is that it's a complicated animal to have a a stochastically statistically multiplexed traffic flow it's a hard thing to have. Ah, one packet came in here. I have to throw it into memory. I have to do a lookup. Do the lookup. I got to send it out of this. That's like hard to do. So, you make the thing expensive. But in a world where it's relatively static, you don't need that much intelligence buffers anymore. >> Bam. Send it right through.
Right. So, I think that there's an opportunity for the to to sort of rethink network architecture given this workload and make it way cheaper. It's like classic innovators dilemma because no matter what vendor you are today, this is not a good trend. You're not going to want to do that. But if a newcomer and you say, you know what, forget all that stat ming like the world's different. I'm going to go with my thing. Actually, you might be able to build something interesting. Um, now it
could be optical, it could be electrical, it could be a combination of the two. Um but I think we're going to we'll emerge and I think you can see clues of that from how Google does its network. >> Yeah. >> Yep. >> I think Amin has has been the pioneer of specifically optical circuit switching right is they were the first ones >> and >> that's for like the tooidal theme. >> Yeah. Yeah. >> I don't understand why it's so slow by the way. It takes them
like a minute to switch. So that one I don't totally understand. I think to some degree it's because of the use case that right now you use it to set up fairly static why do you care >> fair I mean for availability you want to >> give me a few milliseconds not a second [laughter] >> yeah but you probably could honestly right I mean the technology I think is there just the use case is not there >> yeah so yeah know I've been looking at a
bunch of these whether it's mems or uh these maxander thingies the mzis and whatnot to see if there's something in there but you know somebody has to emerge with an architecture and then the issue is going to be >> who embraces it because you can't just build a switch without the rest of the system coming together. >> The I mean that's the other thing >> the dilemma extends to the user's inertia as well. >> Exactly. >> Right. Well, in this way, the network to some degree,
coming back to the, you know, ship me a row concept of infrastructure today, the network has become an integral component of of your architecture. So, this innovation to some degree almost has to come out of the component >> the GPU vendors. Yeah. >> Yeah. But it's interesting, right, because there's only only Nvidia, maybe AMD will get there at some point, but only Nvidia and Google right now have the ability to ship you an a rack system. >> That's right. So today they would have to adopt
it. Uh it'd be interesting to see like you know as more chip vendors evolve. I is there uh you know is there an alternative architecture that they can all agree on? Um sort of the that you know we did >> back in the days >> back in the day right. So like if the thing is like you have two dominant vendors >> the ITF 2.0. >> Yeah. [laughter] Well they they you have two vendors they're dominant. uh they they have their own approach of doing things.
They're not likely to shift off of that. And can you have a a collection of everybody else that says, "No, actually this is a standardized network architecture that we're all going to agree upon and that's what we're going to build." >> The Rebel Alliance. I like it. >> Yeah. The buyers have to agree. That's the key. >> Yeah. >> Yeah. So, Mike, you've seen a lot of these cycles now. Some bigger, some larger. You've seen the last really big one. H how does this compare? As
you probably know, I I've started a new venture fund called Habi Capital. Technically speaking, I retired in 2023, and I was supposed to get a yacht. And >> that didn't work out very well, did it? >> Why? I I actually think this is the most fascinating time to be doing our job and to be in our industry. Not just because the change is big and we all know that you know how transformative the change is but because it requires more uh intellectual exertion to understand how
what next. It's like you don't have a crystal ball. It's not obvious what the next move is. >> And so you have to keep thinking and you have to think at a systems level. Like I don't think I've ever been in an industry before that I had to think about how light gets concentrated on a Zeiss mirror to figure out whether my chatbot works well and like but it's all it's this amazing >> interconnection of things where there's bottlenecks and this changes and this changes that
change. So I find it just the most intellectually fascinating time. And because the enterprise is so significant by figuring out how these different parts unlock, you can create a company here and a company there and this one can become worth a trillion dollars and this one can become 500 billion and so on and so forth. And I honestly I did not expect to encounter another time like this in my career. And by far I f I find this the most dynamic and interesting time to be
doing what we're doing. And I think we should all feel quite fortunate about it. >> Amazing. This this tops 99. >> Oh yeah, easily. >> Totally totally agree. And we are all in the same this business for very much the same reason. This has been a fascinating hour. Thank you so much. And we could be continuing this conversation endlessly. Perhaps we'll do it another time. >> This was super fun. Thank you guys. Thanks for coming, Mike. We appreciate it.