Next in Tech | Ep. 221: Datacenter Slowdown?

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, an S&P Global Market Intelligence Podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry and Company Data at S&P Global Market Intelligence. And today, we're going to be exploring the question of whether or not there's a slowdown in data center build activity that's going on up there. There have been a few headlines that have popped up, a few questions that have been asked. And our team has been fielding a lot of inbound inquiry about what's really happening in the data center world. And with me to discuss it today is a returning guest, Dan Thompson. Dan, welcome back to the podcast. 
 

Dan Thompson: Thanks, Eric. It's great to be back. It's been too long, man. We've been missing each other apparently. 
 

Eric Hanselman: It has indeed. and you're getting up there in terms of the number of times in the podcast where maybe we keep thinking there has to be a returning guest pin or like some sort of chat that we got to get. 
 

Dan Thompson: I'm all about that. 
 

Eric Hanselman: Dan Kennedy is pushing for a jacket. 
 

Eric Hanselman: I don't know if I got a budget for that, but we'll see. Time will tell. But you're on the list. 
 

Dan Thompson: All right. Fantastic. 
 

Eric Hanselman: So, the question is -- I think the challenge that we wrestle with a lot of times is a lot of the misalignment that exists in time scales between IT activity and data center activity. And we've talked about it a bunch of times. But at this point, we do seem to be getting into a period where there are a lot of questions about whether or not there are economic impacts. We've had the deep-seek moment in which everybody sort of said, "Oh, we're not going to need GPUs anymore, which clearly wasn't real. And a lot of different things. And I'm curious on your take. Well, let's maybe frame the question to begin with and then where do we go with this? 
 

Dan Thompson: Yes, sure. I mean, boy, these are fun headlines, aren't they? I mean they get a lot of clicks. They get a lot of reads and then subsequently generate a lot of calls for us. I think just at the onset, I mean, like what I would caution is sort of knee-jerk reactions to things that you're reading in the media as ever. I probably don't need to tell anybody that. But just as a reminder, there's -- and there's oftentimes a reasonable explanation for what we're seeing. And so, I hope together today, we'll unearth some of these things. But if it's okay with you, the way I would like to kind of approach this is sort of talk through what it is that we know for sure, what it is that we've been hearing in the marketplace and then what we, i.e., myself and you as well, what we think about all this? Like if we lay all those cards out on the table, what is this saying to us? -- and how can our listeners perhaps think about this as well and kind of where do we go from here? So, I would love to kind of take it from that approach if we could. That way, we're kind of clearly delineating between what's rumor, what's fact and what is just kind of our thoughts. 
 

Eric Hanselman: Sounds like a good way to move forward. 
 

Dan Thompson: All right. So, I guess let's start with what is it? What do we know? So, I'm just going to kind of throw out some facts here, feel free to interject or interrupt. But -- so we know that we've just come through a season of tremendous build-out. Many people, probably I at some point in time have referred to this as an arms race. Essentially, the major hyperscalers of the world, including Microsoft, Amazon, Meta, Facebook and so on are essentially locked in a race to create new and interesting use cases for artificial intelligence, including generative AI and others and really just see this as sort of the next frontier, if you will, of profitability of innovation. And so, in turn, have been doing lots of maneuvering to make sure that they have the headway to meet their goals there. We also have seen that there have been some slowdowns and pullbacks from Microsoft and Amazon, potentially others on future data center builds. So, we'll talk about that in just a minute, kind of what we're hearing and what we're seeing. We also know that OpenAI, which is a key customer of Microsoft, announced a new partnership with SoftBank and sort of have they expressed an interest in diversifying its infrastructure provider. This seems to be a bit fluid. I'm sure you, as I did, saw the headline earlier this week, seems that OpenAI has come back to the font of Microsoft and are attempting to drink again, perhaps taking more funding from Microsoft. And so perhaps this will repair the bromance if you'll humor me that. And maybe this means that we're back on again with Microsoft. But in any case, we know that that has been a bit of a pain point. Sacha Nadella had some pretty strong comments in the weeks following that announcement in terms of his view of AI and kind of where we need to go. For its part, OpenAI also have not seemed to be able to make up their mind whether they have enough compute or don't. They've kind of gone back and forth on we're not compute constrained or yes, we are compute constrained. So, there's that. Given all that, it's very reasonable that Microsoft would have some hesitancy about really long pre-leasing, meaning I'm signing a lease for something that's not going to -- meaning a data center that's not going to exist for 3, 4, 5, maybe even 6 years. Of course, they would be hesitant to do something like that. So, I think that is a bit of a backdrop. We've also seem to kind of throw in a couple of other ideas here. Microsoft has just announced a series of layoffs to the tune of 3%. This is largely middle management. I'm sure you'll have some thoughts on why that is. I have my own. We'll talk about that later. On top of that, though, or on the back of that, the most recent earnings calls, I think, give us some very nice insights here. Our colleague, Jan was wrote a report about this, very nicely summarizes that since 2020, Alphabet, Amazon and Microsoft have all been increasing capital outlays on annual growth rates exceeding 20%. So that's 5 years. 
 

Eric Hanselman: Yes. We were startled when it hit 15% and now to see it continue to grow. 
 

Dan Thompson: Right. It's like... 
 

Eric Hanselman: Kind of eye-watering. 
 

Dan Thompson: A lot of stuff happening here. So, Alphabet's CapEx is more than doubled and Microsoft has more than tripled in the last 5 years. Very interesting stuff. All 3 of those providers, Alphabet, Amazon and Microsoft noted in their earnings calls, backlogs for capacity, meaning that they have customers who want stuff that they can't give them just quite yet. And we can take from that -- from the fact that they're not slowing down their capital expenditures, they have customer backlogs, there's no reason for them to be pausing if we think about it just holistically. So maybe there are sort of micro things playing out on a market-by-market basis or even a country-by-country basis, which will be causing some of this consternation. And the core technologies behind all this. If you look at the fact that we've now spawned a new industry and granted it was repurposing old with the GPU as a service providers now often referred to as the NEO clouds, -- they're doing significant business. The availability of GPU instances seems to be fueling their whole business. So, the Volters and the core weaves of the world. A lot of things that seem to be pointing to the fact that there is still resource scarcity, as you're pointing out. Yes, absolutely, absolutely. And that's something that we hear from the manufacturers, the equipment manufacturers, the GPU manufacturers that you just talked about -- I'll talk about that in just a minute and when we get into what we're hearing. Also, we know power requirements continue to increase exponentially for the latest GPUs, right? So, this is just kind of throwing the facts out on the table here. So, when you consider the world was sort of shocked by 45-kilowatt racks, and then we were shocked again by 150-kilowatt racks. I think we're all still reeling from the announcement of a 600-kilowatt rack. And then the show last week displayed a 1-megawatt rack. There's a lot of change Eric. 
 

Eric Hanselman: Well, especially when you figure that at a megawatt, this is -- back in the day, a megawatt was a tidy little data center in and of itself. 
 

Dan Thompson: Oh, yes, yes. Or more recently, like a megawatt was what you would power like a 10,000 square foot data hall with. So, like hundreds of racks. Now you're talking about a single rack that consumes that amount of power. I do want to say like with all those facts, I want to add one last fact, and that is that building has not stopped. I just got back from Phoenix, where I was doing tours there. I've done tours in different parts of the country in different countries already this year. Cranes are still everywhere. Building is still in motion. Like we have not reached a point where Microsoft, Amazon, Google or Bing or anyone for that matter, are bailing on buildings that are in progress. That would be lawsuit City. And it was -- it's basically -- that would be very, very bad, but we're not there. I just want to be clear. So, when you're reading in headlines about Microsoft or even just hearing me talk about Microsoft pulling back or slowing down or pressing calls or whatever, this is on future buildings. These are on buildings that don't exist yet. Perhaps the dirt hasn't even moved yet. So, this is not like a project that was underway. This was one that maybe the company got financing for, maybe they were starting some of the activities with the utilities, things like that, maybe not. But these are future plans. not current building. So that is worth knowing. But as I was saying, definitely in Phoenix, Arizona, cranes are still moving, concrete still being bored, walls are still going up, all of the above. 
 

Eric Hanselman: Well, and this is one of the things I spoke to in the intro, but I think it's worth stepping back into, which is the time scale that a data center operation operates under. These are investments that have got 10- to 20-year investment life cycles. The build process is very long ramp. The supply chain is a very long ramp. These are things that if you're even considering building, you've got to start planning for years and years in advance -- and so in a lot of cases, you've got major consumers, whoever the operator happens to be, who are going to place bets that they're going to have to shift over time. And at that time scale, they're going to lay a marker for something to continue the analogy, for something that they may or may not actually pick up down the road, which, again, at least appears to be the kind of thing that was grabbing some of those headlines. 
 

Dan Thompson: Yes. Yes. And then like consider also that we have come from decades where, let's say, you and I wanted to build a data center, we would go meet with our friendly local utility and say, "Hey, I need 100 megawatts. utility would say, okay, cool. Yes, no problem. We'll get that rolled out to you. When do you need it to where today, -- the reality is in many markets across the United States and across the globe is that you and I get together, we say we want to build a data center. We meet with our friendly local utility. The friendly local utility says that's great. I need to see your balance sheets. I need to know what your credit rating is. I need to know who your customer is because what we're about to do is kick off a 5-year planning period in which you will be one of many included in that planning period. We'll get you power in about 10 years. That's radically different world. 
 

Eric Hanselman: A little harder to scrape those electrons and pop them up onto the wires. It's a very different world. 
 

Dan Thompson: That's exactly right. So, let's talk now if we can, about what it is we're hearing. So, viewers or listeners consider this the rumor mill. So don't consider any of what I'm about to say is back, but just consider it as -- these are things that we hear in conversations. One of the things, I guess, first and foremost is we have heard that there is a slight, and I want to say slight again, hesitancy on the part of the hyperscalers for significant pre-leasing. The next part of this, though, is we're hearing mixed things. So, we're hearing mixed signals from folks on what exactly that time frame is. Some providers have said that if you can get a building built in 2 empowered, key point, if you can get a building built empowered in 2 years, you'll have a very captive audience from the hyperscalers. Some of the providers we've spoken to said, no, it's more like 4 years. They'll still be very excited that you can get a building built empowered in the next 4 years. It seems across the board, though, that if it's going to take more than 4 years, you may run into a little bit of hesitancy or perhaps the provider may look at you and say, let's talk about that later. What can you get me in the next 4 years or 2 years or whatever the actual time horizon is. Again, like considering the facts that we talked about a minute ago with the fact that maybe in some markets, we have serious questions about whether you can even get me power. Maybe I have my own questions about whether I'll be able to get GPUs in time to fill up this building that you're going to build me or whether the power and cooling equipment is going to show on time. I mean all of this to me makes very good business sense of why I might have worries. Also, like not least of these is that my key customer that I'm building for is now sending me mixed signals, right? I mean, like all this is very reasonable that a company would then say, I don't know, maybe I'm interested, yes, but maybe let's keep talking. 
 

Eric Hanselman: Not leaping in with both feet straight away. Well, I mean -- and again, to your point, there are so many market dependencies, just where power is available, where you actually want to go site capacity. I mean, that it throws a bunch of variables into these conversations. 
 

Dan Thompson: Yes. Exactly, exactly. And this, too, I mean, like what you were just saying, this is something else that we are hearing is that utilities, which have made prior commitments are now walking back on some of those commitments. So, say, for example, they said we can get you power in 2 years. If the utility comes to you now and says, actually, it's going to be 4 years, I can imagine that's going to create some problems or some challenging conversations that you're about to have to have with Microsoft or Amazon or Google. to push a time horizon by 2 years or even a year beyond what the contract signed is, I imagine that's going to create some challenging conversations. This is all very reasonable. None of this is like -- I think we were talking about at the beginning of the call when you and I were just chitchatting that like we don't need any broad conspiracy theories like this is just good business, right? I mean, like, of course, you're going to be a little bit hesitant given those things. Another thing that we've been hearing is that there has been some amount of pressure inside Microsoft to cut costs and preserve cash flow. Again, this is rumor mill. I want to stress this. However, that seems to align with these recent layoffs that we've just heard about. There could be many reasons for this. as any financial analyst would know, there could be 1 million reasons for that, right? And one of which included was outlined in the earnings call, Microsoft took a very slight hit in their margin power. And perhaps they're just doing this to recover margins for investor performance. Maybe that's what they're doing. It could be that they want more cash flow because they want to build buildings now instead of leasing buildings. It could be they want more cash flow for -- I mean, buying GPUs. I mean there's a million reasons why they might want more cash flow. 
 

Eric Hanselman: And good financial management, there always has its own benefits, but definitely, a set of reasons that could be driving this, some obvious and some clearly that we won't be able to know. 
 

Dan Thompson: Right, right. But that could be playing into this. And then finally, we continue to hear about supply chain challenges. I mentioned this just a minute ago from power and cooling manufacturers, so being able to get my backup generators, being able to get the cooling units, like those continue to be challenges. So again, like if I'm Microsoft or I'm Amazon and I have signed a contract with data center provider, Eric and Dan, and Eric and Dan come to me and say, "Hey, you know what, so we thought we had the backup generators. It turns out they're going to be about 6 months late. I imagine that's going to be a very painful conversation. 
 

Eric Hanselman: You could figure that, that would be, again, a delay of operating capability, especially when there are concerns about performance of all of this capital investment because the markets are, in fact, looking at when the returns from all this capital spend are actually going to start to see the light of day. Now that says that you've got a data center that you can't flip the switch on until you get all the backups put together, chillers in place, all the rest of it. Now you've got a stranded asset that's going to sit on your books for half a year. And that's not going to be a happy thing from a capital performance standpoint. 
 

Dan Thompson: Yes, that's exactly right. That's exactly right. And I think the utilities not being able to do it in a timely manner, that perhaps being a reason for a pause or a slowdown on a very specific project, supply chain issues, that potentially being a reason for a pause or a slowdown on very specific projects. Again, like what this is telling me is that speed to market still matters. I think that's not going out on some great limb to say that. I think speed to market and impediments to speed to market are going to make a difference in these conversations, right? If we consider the arms race language that we talked about at the beginning of the program, any slowdown or any impediment to my progress is going to be something I'm going to think about as one of the hyperscalers. It's going to be something I'm going to try to mitigate and work around. 
 

Eric Hanselman: Well, and it's one of these situations where if you look at a few years ago, the capital investment environment was also very different in that you've now actually got to pay interest on when you borrow it as opposed to our 0% interest days back -- way back when. And it's one more thing that adds additional pressure in terms of making sure that, that capital is really performing. 
 

Dan Thompson: Yes. Yes. No, excellent point. Excellent point. So, what do we think about this, Eric? So, if we think about -- we talked about sort of the facts that we know, we've talked about things that we're hearing. What's the rumor mill saying? What are we seeing in markets, things like that? I have to say you just kind of right off the bat, it seems very reasonable. that Microsoft would be hesitant to go way out on a limb for a customer that we were just describing who is sending the mixed signals. That just makes good business sense to me. Especially if there are other supply chain issues and things that might get in the way, if I'm making a commitment for you for a building that's 4 years from now and you're sending me mixed signals that you may or may not want that building, I'm going to get out of that lease as quick as I possibly can because I'm not going to go out on a limb for you if you're not going to commit to me, right? I mean like that's -- again, that's just good business sense. 
 

Eric Hanselman: Yes, it absolutely makes complete and utter sense in terms of how we got to think about it. This is -- these are just good basic business decisions. The other thing that I would comment on, we talked about this just a little bit a minute ago, but you have to imagine that it's getting extremely hard for the hyperscalers to envision or imagine even what they are going to need from an infrastructure perspective 2 years from now, let alone 4 years from now, let alone 6 years from now. Again, like we talked about this a minute ago, like the 45-kilowatt rack grew to 150 kilowatts, now 600 kilowatts, maybe even a megawatt. Like when you're trying to imagine what cooling infrastructure would be required to support a 1-megawatt rack when none of us had really even seen but one, and that was on a display at somebody's booth at a show, you know what I mean, but like -- and you're trying to like rack your head around what the planning or what the infrastructure for that looks like I imagine this is creating a challenge. And so maybe what we're seeing is a bit of a redesign period like we saw at the end of 2020, beginning of '21. We've come through a similar thing like this in the past, where all the hyperscalers paused for just a minute. They all blamed on different things at the time because everybody was still reeling from COVID and trying to make their books look as good as possible. They all blamed on different things at different times. But then the foot went on the gas again, we were off to the races. It's been build, build, build since then. And what we saw was the design radically changed. And maybe what we're seeing right now is folks saying, "Hey, you know what, we just built a ton of data centers. We need to fill all those up. We got a major change coming again from an infrastructure perspective. Maybe we should take a minute and think about this. 
 

Dan Thompson: Seems very reasonable to me. 
 

Eric Hanselman: Well, and if you think about it, under the hood, we are seeing a major change. We talk about the power increase. But what also comes along with that, in order to make sure that these things don't go up in a puff a smoke, there is the shift to liquid cooling. And the liquid cooling shift potentially comes with not only just a change in the way in which you're plumbing your data center, but maybe some of the design standards around this. How cold does that water have to be? Are your chillers actually providing sufficiently cool water? Now this is as we've gone through processes in the past with changes in air cooling, we had a shift -- the organization set standards for how data centers are belt ash rate. It actually raised the acceptable temperature for air within the data center. And in order to accommodate systems to allow systems that ran hotter to be able to actually survive hotter to be able to take some of that cooling load off to allow the data center to run a little more efficiently. We haven't yet seen any of those ashray temperature shifts for chilled water, but you got to kind of wonder, are we heading there as well because it's -- again, maybe there is this point at which there's a rethink about what you do around cooling this much higher density environment. I mean back in the -- we had all these conversations about AI data centers and whether or not they were real, right? 
 

Eric Hanselman: Well, now I guess they're kind of real. 
 

Dan Thompson: Different day. And to the point you were just making around liquid cooling, will that liquidly cooling system that we designed for 150-kilowatt rack be sufficient for a 1-megawatt rack when I've got a room full. Like I have to imagine these are the questions that the engineers are asking. And they're all saying, I need to -- I mean, because this is the reality of the data center business is all the data center providers love to tout how flexible they are. That's a reality. They want to be able to accommodate whatever design that Microsoft, Amazon, Google or whomever else would want to do. But at the end of the day, I'm going to order my cooling system, and it's going to take maybe as long as 2 years to get here. So, I'm going to have flexibility right up until the time that you sign the dotted line. And then I kind of need to know what you want me to build because we're going to be sitting around waiting on equipment for 2 years. So, you know what I mean, so like there's good reason here, I think, for the hyperscalers to be taking a step back and say, okay, look, again, all the things we've been talking about, I don't know what a megawatt rack is going to look like when you put 100 of them right next to each other. Like how is that even going to work from a cooling perspective? What other things need to change about the data center to accommodate that? And then to turn around to a data center provider and say, "Hey, can you build this thing that I don't really know what I want yet? And we all just sort of heard about it at the exact same conference. You know what I like that has to be incredibly challenging. So yes, I mean, again, like this all seems very reasonable. The sense though that we're getting, just so that it's said is everybody -- and hopefully, this is not just industry optimism, but everybody seems to sense that if there is a pause going on here that it will be short. just like the last one was short, and we are back off to the races again. So whatever handwringing is going on over that is on, we continue to hear that it will be resolved in short order. The other thing I think that's worth kind of chatting about, and I'll choose my language wisely here, but we have had a change of administration here in the United States, which has a different emphasis than past administrations have perhaps. Some of the pullbacks that we've heard in the media have been reducing capacity overseas, increasing capacity here in the United States. Given what we know, it seems reasonable that companies may say, "You know what, I was going to do this thing overseas, but now maybe I'll do it here in the United States. Maybe that saves me money. Maybe that gives me favor with the current administration because I'm putting America first. Maybe that's just an internal philosophy. It has nothing to do with the administration. It could be a number of things, but you could imagine at least a world where companies are thinking in this way. And again, we have seen this. This is a thing like we have seen some data center pauses overseas and then some -- we've heard conversations of -- they came in, they said, can we skip out on that lease, and can I transfer that capacity here to the United States? Like same capacity, same numbers, but just don't do it overseas, do it here in the United States instead. So, you could see a world where the current administration has some influence on that or at least the thinking on that. So that could, again, this could be part of it. Again, as everything in politics in the United States tends to be short-lived. The one constant in the United States has changed. So, I mean, I think I don't I have no worries that overseas development will go away altogether. In fact, if you watch the news just this week, there's a huge announcement in Saudi Arabia, huge announcement that the Kingdom is going to be investing here in the United States. So, like it doesn't seem like any of this is really like a hard and fast rule. It just seems like companies may be just rethinking things. 
 

Eric Hanselman: Because there's all -- the Gulf and Middle East, you see a ton of build-out, lots more power generation capacity getting built, lots of things that are happening there. So yes. 
 

Dan Thompson: And also, like on the shifting capacity idea, you could also see this as a reasonable conversation. Microsoft sits down with you, you say, "Hey, Microsoft, I'm sorry, but the local utility has said it's going to be another 12 months before I can get that power, I promise to you. And then Microsoft turning around to you and saying, okay, cool. Well, if we can't do it here, is there some other market that you could satisfy this demand for? And then if you're the type of provider that says, "Hey, Ford just making up markets here, but if you're -- yes, you know what, I can't do it here in Arizona, but I could do it in Dallas. How about that? And Microsoft said, cool, yes, great. Let's move the lease from Arizona to Texas. You still meet my timelines. I'm still happy. I'll make it work. You know what I mean, you could see this type of thing playing out. So again, like these are things that all seem reasonable to us. Like this isn't like any alarm bells or anything like that going off, like when I see construction crews leaving an existing site, then I'll be worried. When I hear data center providers saying, we have no pipeline. then I'll be worried. Frankly, when I hear the equipment manufacturers saying, we have plenty of stock, then I might also be a little worried. 
 

Eric Hanselman: And just to be clear, none of those things are happening today. 
 

Dan Thompson: That's exactly right. So yes, I mean, like all of these signals, this to me, we've seen far worse over the years in terms of pullback slowdowns. We've seen companies bailing on contracts on existing data centers, like we have lived through that over the past several decades. That is not at all what's happening now. So just to kind of hopefully offer a little bit of calm and perhaps a couple of reasonable things to think about in terms of like what could be happening behind the scenes. Just so it's clear, like -- and I wish it were this easy, but I don't think it's all necessarily related to the OpenAI, Microsoft consternation. Amazon is not in that realm, and they've been involved in some of this as well. I think it's more complicated to that. But as is always, sometimes the truth is more boring than diction. And I just think there's probably a lot of different dynamics playing into this, and this is just normal business. 
 

Eric Hanselman: Just the normal noise is in here, right? 
 

Dan Thompson: That's right. That's right. 
 

Eric Hanselman: Well, thanks, Dan. This has been great. Always interesting to dig into a lot of the complexities in what is such a fascinating market. So, thanks for being back. And I guess we'll have to see where this all goes. 
 

Dan Thompson: Absolutely, absolutely. Well, I'm sure in 3 months, there will be something new. So, I'm just going to invite myself back. 
 

Eric Hanselman: Well, and it will be one more notch on the belt in the returning guest category, right? 
 

Dan Thompson: Yes, that's right. That's right. I got to chase down and make sure that my rankings still good. 
 

Eric Hanselman: Sounds like a plan. Well, that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun, Kyra Smith, and Adam Kovalsky on our marketing and events teams. I hope you'll join us for our next episode where we are going to be talking about FinOps, the effort to get one's arms around costs of IT, mostly focusing on Cloud and a lot of the things that are taking place there, it is really something taking off as organizations have been looking to manage what is costs on the IT side. I hope you'll join us then because there is always something Next in Tech.