Next in Tech | Ep. 266: Managing data center risk

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, and S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry Research at S&P Global. And today, we've got a preview of an upcoming webinar that's going to be looking at managing risk in digital infrastructure. And with me to discuss it are 2 of the folks who will be on the webinar, Kelly Morgan and Tony Lenoir. Welcome to the podcast to you both. 
 

Kelly Morgan: Thank you, Eric. 
 

Tony Lenoir: Thank you, Eric. 
 

Eric Hanselman: So we're going to be looking at really what are some of the ways in which what has been this serious exuberance to build data centers, and everything has been driven by AI infrastructure demands and all of the various buzzy topics that are out there. The challenge, of course, though, is when you start looking at actually funding these environments and a lot of the associated financial structures that have to be built around these, especially when you start looking at how you ensure and manage the risk of building out data centers and everything that's necessary to actually start ensuring that they're producing effectively and really achieving what is the promise that we seem to be all heading towards in all of this. There are a bunch of different aspects in terms of how we're -- we really are getting towards enhanced capacity that I think are probably worth digging into. And maybe, Kelly, just to start off with, can you give us a little bit of background in terms of data center builds? I think there's this idea that all you have to do is find enough land and you simply stand it up and fill it with GPUs and bang, you've got AI, but that's a little more complicated than that. 
 

Kelly Morgan: Right? If only that were true, yes, a lot of people would be very excited to just build these data centers as easily as they used to. So there are a lot more challenges. And when it comes to risks, one of the things we've been discussing with people in the industry are the greater number of risks now in getting the data center up and running. So in the old days, you would say, identify a need for a data center, you'd identify a site for it. You would go and ask the power company for that power. You would get your permits for your data center, a couple of years would go by, and the data center would be built. No problem. Now of course, these data centers are much bigger. The power required is much greater. It's becoming tougher to get all the ducks in a row that you need to have these things built when you want them. And so there's an increasing amount of risk that the data center buildings themselves will be delayed. This can come from a bunch of different factors. We're seeing a lot more local opposition to data centers. So the permitting stage and the zoning stage that used to go pretty quickly in a lot of locations, not everywhere, but that's becoming more complicated. That can be derailed. It can be delayed. Sometimes these data centers are completely refused in certain locations due to just opposition from the local population. So that's a factor that is becoming more of an issue and wasn't such a problem before. And then, of course, a separate factor is the ability to get power. That is probably the biggest one on people's minds at the moment. And so I think Tony is here to talk about that part. 
 

Eric Hanselman: Tony, go ahead. What are your thoughts? I think Kelly teed up some of this in which we were living in a world in the past in which the total power consumption of data centers and what they're really looking for was a relatively small chunk of what the grid was able to provide, a little different than what we're seeing today, yes? 
 

Tony Lenoir: Yes, absolutely. And maybe to backtrack it a little bit, I would say that today, electricity is probably one of the most valuable commodities out there aside from maybe time itself. Certainly, it's the most essential to our modern society. When you think about it, there is no technology sector, there's no data center industry without electricity. When you buy a new laptop, a new iPhone, when you get that out of the box, the first thing you do is you plug that into a wall outlet to power that device without that, no device. And what am I trying to get to here? When you think about ensuring a data center, it goes beyond the scope of the risk associated with physical assets themselves. It also includes business interruptions and all the risks that go with that, including bridging service level agreements, for instance, the penalties that may involve, but also the opportunity costs and revenue losses and so on and so forth. Most data centers these days guarantee the 59 type of availability. So that amounts to being down about 5 minutes a year, I believe, Kelly, if I'm right. So you can imagine a major -- the impact that the major power outage or lack of power may have on the industry as a whole. 
 

Eric Hanselman: That's an interesting point because if you think about what we've typically looked at for insurance and managing risk, yes, there's business continuity. But in data centers, that now expands into a much broader role. There are so many different pieces that start to come into play beyond just simply the kinds of things that you look at for an average enterprise. Now you're talking about much more complex systems with so many different elements that are playing together well, and Tony as you're pointing out, with significant financial penalties that insurance would be managing against for any downtime that occurs. 
 

Kelly Morgan: Yes. So you have that combination of the downtime issue and data centers have typically tried to offset that with backup power, but there are growing questions whether that will work at the gigawatt scale, right, at the kind of scale that these new data centers are requiring backing up -- having that backup in case the grid goes down, gets to be a lot more complex. And there's additional risk when talking at these great scales and requiring the power of a whole city for a data center campus now. There are issues with whether the grid can even provide that at all or fast enough, right? And so where does insurance come into play there in terms of delays to the data center because the power is not available or when the data center has to build a whole separate power plant, how do you go about ensuring that? What are the risks involved there? How does that backup work? And Tony, I toss it back to you again to talk about more of those risks. 
 

Tony Lenoir: Sure. You touched on that already, history of data centers. It used to be that what the facilities were much smaller. The required much less power. And I suppose data center developers didn't think too much about it. They wanted to develop a data center. They went to their -- they looked for areas with a lot of fiber connectivity. They looked for a substation that maybe had extra capacity. They went to the local utility and just planned around that. There was not too much to it. And the pace of developing those data centers was much slower. And there have been 2 major milestones in the last 5 to 6 years actually. First, there was the COVID. There were the COVID interruptions. Society stopped, everybody started working from home and so on and so forth. So data centers scrambled to meet that new demand for their services, lots of video conferencing calls and so on and so forth. People streamed a lot more from home. And so the pace of building accelerated, and the power demand increased at the same time. And then you had, of course, the launch of ChatGPT in the fall of 2022, which kicked off the AI revolution. And I suppose that's really what we're talking about here today, right? And the thing is that really turned the power sector upside down. Well, it really took everything by storm. They were not used to that pace. In fact, electricity demand in the U.S. from the early 2000s to about the launch of ChatGPT was essentially flat. Now because of these developments, we are looking at electricity demand to increase at a compound annual growth rate of 3% to 3.5% from now to 2030, 2035. The big difficulty here is the mismatch between the pace at which big tech and data center operators and developers want to build out that data center footprint and the pace at which the power sector can accommodate that. 
 

Eric Hanselman: Fundamentally, you're talking about a power sector that wants to be able to plan decades in advance. Their asset lifetimes are much longer, whereas in a data center, hey, if you're waiting for a year or 18 months, that's a heck of a long time. You got working capital that you're trying to go put in the ground and put up to the rest of the Internet, that's a challenge. 
 

Tony Lenoir: It is. And in fact, I know it's an analogy that's been used in some of our reports. We've compared the data center industry and big tech in general to a sprinter at the Olympics or whatever. And the power sector really are like marathons. They plan for the long rates. 
 

Eric Hanselman: And I guess that's one of those challenges you both talked about, the scale of the power demands that are starting to what I was going to say starting, but actually are already here. We've moved from data centers that we're talking about hundreds of megawatts to now looking at gigawatt capacity. And we're now looking at strategies that are starting to get a bit more risky because of the demand that's out there and now, going from what was simply all work with the local grid operator, they'll provide the power that I need to now looking at various alternative energy supplies. We have data center operators looking at restarting nuclear power plants. We've got all sorts of alternative means of being able to generate the capacity that's out there. all sorts of things that are starting to be explored with these partnerships. People are talking somewhere in the future of small modulars, although I think small modular nuclear reactors, which, of course, but the challenge there is, can they have the capacity that's necessary. All the while, well, Kelly, you had made what I think is one of the interesting underlying points around this. You have grid operators that are starting to push back on the -- on how the data center itself is even going to operate. Historically, we've had backup power that was there as a backup. Now you've got things like regulations from Texas with ERCOT, now actually starting to float the idea that the data center actually would need to run exclusively on backup power when the grid itself was stressed. So we've turned around what was that initial equation of the grid really being the reliable supply to now the data center having to be the end of this equation in which reliability has to be built into the operational capabilities of the data center itself. And that it seems starts to put us in a very different environment when we start thinking about the risks, the operational risks of the data center. 
 

Kelly Morgan: Yes. It's a different risk profile when you have the backup power becoming the primary power and then do you need a backup power to your backup power system, right? Does the grid become the backup power? It definitely becomes more challenging. I think another key area for the risk analysis is the cost of these data centers. So it's particularly the equipment going into the data centers now is so much more expensive and more powerful requires -- these GPU chips require a lot more power per chip. So there's less tolerance for error in the sense that you have so much power, so much heat being generated that if something goes wrong with either your power system or your cooling system, you're potentially burning up $1 billion or more worth of equipment. So there's -- from an insurance and risk perspective, there's a lot more to cover or there's a lot more value in each of these buildings now as well. 
 

Eric Hanselman: And I guess the thing around this that you're pointing out is that margin for error is that much lower. And the reality, of course, is that historically in data centers, you had sufficient cooling capacity. And if you ran into problems, if you had cool loss, if you had your cooling problems lost in some form or fashion, you had a little bit more running time to get systems shut down to be able to actually start to have some level of controlled shutdown for the environment. But when you have GPUs that are, one, hard to come by; two, expensive; and three, start to dissipate very large amounts of heat. That cooling failure -- and now, of course, we're actually using different types of cooling. They're now starting to use liquid cooling, whereas in the past, we were simply moving chilled air through an environment. That meant that, in fact, we had a certain amount of capacity within the facility itself. As long as we could keep the air moving, maybe we still had some options. But if you lose the supply of chilled water for a GPU, the time between when that supply fails and the GPU actually starts to in some cases, literally melt is just that much shorter. You're now talking minutes in terms of where that failure is just simply because of the amount of heat that those individual devices generate. You've got to keep them cooled, you've got to keep that cooling water flowing and you don't have the thermal ride through that you might have in the data center with air cooling, where all you had to do was keep the vans going and you at least had some hope of keeping things okay. That starts to put us in a very different environment as well. 
 

Kelly Morgan: And given all of that, then climate risk also becomes a key issue here, right? Because you've calculated your cooling to operate under certain circumstances and perhaps those circumstances are changing. That's another key risk to take into account what is the risk that maybe there's a water stress problem in that area and what you were calculating on using in the way of water as part of your cooling might no longer be available. There are certainly ways to do cooling without using any water, but you have to have set your data center up that way. And then if you are dealing with a difference in temperature that's outside of what you were originally planning, which has happened in certain cases with older data centers that were not planned for, say, heat events that we now experience, there can be a lot of challenges there as well. So there is increasing risk on that side, too. 
 

Eric Hanselman: Extreme weather management becomes a big part of this because as you said, it's not just what's going on within the environment itself. If you've got flooding, if you've got wildfires, those are going to, of course, be just as impactful to a data center at the same time. 
 

Kelly Morgan: Exactly. Not to mention, I'm sorry, I feel like the bad guy coming in here. But then the other big area of risk which we're seeing now is security risk, right? So we've seen this with the Middle East and some of the data centers there actually being targeted. And as you end up with these larger and larger campuses with potentially whole power plants attached to them, perhaps they also become larger security risks. So that's another area to now take into consideration. 
 

Eric Hanselman: The physical security risks and the cybersecurity risks again. I guess we're doubling down on the bad news here, but... 
 

Tony Lenoir: Also the supply chain security risks. He mentioned the conflict in the Middle East. This obviously, everybody knows that's had an impact on supply chains, the fossil fuels around the world and some regions of the world depend on this flow of fossil fuels more than others to produce their own electricity. I suppose here in the U.S., we are fortunate enough to have all kinds of sources under and above ground, right? We have a lot of oil, we have a lot of natural gas, go to the desert Southeast, you got a lot of sun. You go to the Prairies; you got a lot of wind. We can produce electricity using all kind of primary sources. Some regions are not that fortunate. And I'm thinking about Southeast Asia here, even Europe. You can probably hear and you hear it from my accent, I'm not from here originally, I'm from France. If you think about it, those security risks tied to supply chains and access to fossil fuels were the primary reason why France moved to producing its electricity primarily from nuclear in the 1970s. And so it is interesting to see the domino effects of some of those developments around the world. 
 

Kelly Morgan: And that's an important point, too, the backup power for, I'd say, 95% of the data centers out there is based on diesel generators, right? So they could be impacted by diesel shortages, too. A lot of them have been converting to biodiesel but even the backup power runs using some of these supply chain constrained systems, I guess. 
 

Eric Hanselman: Well, and biodiesel also is constrained because we've got limitations on the feedstock to go create it. We haven't yet worked out some of those details. But certainly, at least in North America and Europe, that's a particular challenge. Some of those feedstocks in Asia, may be a little more plentiful. But again, there still is that balance. And the question is, if you do switch cut over to diesel from whatever source, how much running time do you have? When do you have to start bringing in tank trucks full of diesel to keep the data center running? And will you actually be able to find a tank truck full of diesel when that actually happens. We had that interesting set of highly dependent systems we saw in Superstorm Sandy when it hit New York. We had data centers that had alternative supplies of fuel, but needed electric power to run the pumps to get the fuel up to the generators to be able to actually run. Once again, a lot of different dependencies in terms of where this fits. Tony, do you have a point on this? 
 

Tony Lenoir: I was just going to say to get back to something that Kelly was saying, I wanted to get back to maybe the different data center profile. Some of them are more at risk than others, obviously, right? When it comes to energy, one of the many ways that the stakeholders have adopted to address the discrepancy in pace is to allow data centers to connect to the grid faster if they agree to some demand response programs, basically curtailing their workloads during periods of stress. So obviously, when you have the type of workload that is batchable in here, I'm thinking about cryptocurrency mining, you can maybe plan your runs around identified periods of grid stress. For some data centers, it's not possible, right? Of course, AI training technically is batchable, right? Correct me if I'm wrong, Kelly. But there are may be more risks associated with disruptions there as well. And in fact, probably more costly because an AI training run cannot be checkpointed, meaning if it stops, you have to start all over again. So that entails many costs. There's the cost of the initial run. There's the cost of the rerun and the -- and there might be the opportunity costs associated with delaying your model or versions of your model and falling behind in the race to achieve AI competitiveness because as you guys know, we are very much in a race to achieve AI dominance. It is very much the sprint that you were describing before, we've got our track shoes on, and we're trying to get this thing going as fast as possible. The point that you're making, though about being able to get to data center environments that are more flexible in their power consumption has been one of those things that I don't know, I describe it as maybe a holy grail that we've thought about for ages because historically, data centers have been set up for very high availability. And there is the idea that you could potentially shift workloads in the event of problems with power availability, but that requires work from the data center clients, the folks who are actually running workloads in data centers to ensure that's there. You'd mentioned cryptocurrency mining. That's a really interesting thing because we do have a world in which cryptocurrency mining can actually take advantage of energy in lots of different places. I was driving through gas fields 1.5 months ago. And at every so often at one of these major junctions in natural gas lines, there would be this great, big, huge barn with a set of turbines in the back and that we're there for crypto mining because cryptocurrency mining, you can turn on, you can turn off. Tony, as you were saying, it is something that you can stop and restart without penalty. And AI is not like that. That the -- as you were saying, if you get a training run, stopping that in the middle has a whole bunch of costs associated with that. And maybe there are SLA penalties, if in fact, you can't complete the run. That hasn't typically been the way in which a lot of the providers are handling it to date, but it is one more concern in terms of being able to ensure that you actually have the kind of availability you need. And we're still a ways from having data centers and computing environments that you can easily migrate to wherever power or availability happens to be for the computational work. who knows a future that maybe we get towards, but we'll see where that goes. 
 

Kelly Morgan: Exactly. 
 

Eric Hanselman: I'm curious in terms of managing some of these risks. We've talked a little bit about some of the front end of these. Do you have thoughts about what you can do to actually mitigate some of these risks and maybe manage it? In some cases, these are hard problems to solve. 
 

Kelly Morgan: They are. I think so far, we've seen supply chain risk on the data center side being managed by trying to order a lot of the equipment in bulk ahead of demand, for example. So a lot of the ways that you mitigate risk are actually influenced by scale. So we may end up seeing this impact the industry in terms of who the top providers are. The ones that survive and do well may be the ones that have the scale to offset some of these risks, both supply chain, but also a lot of these other types of risk. So we'll see how this falls out, I guess. 
 

Eric Hanselman: And Tony, on the power side, do we have options? 
 

Tony Lenoir: Kelly already alluded to the trend towards on-site generation. And that is definitely something that is picking up momentum. Ultimately, I think the data center operators, developers want a connection to the grid because despite everything we've said, it's been a very resilient system. It really is a marvel when you think about it. But since it's been difficult to get interconnection to the grid because of what we've discussed that they're turning to on-site generation and primarily fire -- sorry, gas-fired power generation. And so that in and of itself, I must say battery energy storage. So when you think about gas-fired power and battery energy storage, so gas, lithium ion. So when you -- in terms of insurance, you think fire, right? So those are additional risk factors I suppose to take into consideration for insurers. But yes, so those are some of the workarounds that are being quickly adopted at the moment. Of course, there's the emergence of the return toward nuclear energy, nuclear energy was passive for almost a couple of decades. People wanted to move away from that. Even I alluded to the fact that I'm already from France earlier, even France was moving away from nuclear energy, and they're doing a complete 180 at the moment saying, "Oh, wait a minute, now we need all of this power. And on top of that, we'd like this power to be carbon-free if possible. So nuclear energy, when it comes to that is, it really is the ideal fit for data centers. It's high-density energy. When you think about it with one nuclear plant, you can with the average size of the nuclear plant delivering about 1 gigawatt of capacity, you can power about 1 million average homes here in the U.S. You can power a large data center campus. And for data center company, for instance, striking a power purchase agreement with this kind of power plant, it's great because you can secure all of your power needs through one deal instead of multiple PPAs with -- yes, renewable power plants, solar and wind power plants. So really, nuclear offers many advantages. And of course, at the moment, we have the rise of many small modular reactor start-ups, including Energy that went public in a very successful IPO about a week ago. So this -- but I suppose when you talk about nuclear, you pull on one thread and everything many other threads Raul. And so you start talking about nuclear and insurance rates and so on and so forth. So then you get into the issues of waste disposal and so on and so forth. So additional considerations for insurers here. 
 

Eric Hanselman: As you said, you pull on one string, and it pulls you out one side and back over the other. So many different pieces around this. I guess we'll save any further discussion for the webinar itself, but this has been great. Thank you both for giving us a little preview into some of the thoughts around this, and I will point our listeners to the webinar coming up soon. 
 

Kelly Morgan: Thank you, Eric. 
 

Tony Lenoir: Thank you, Eric. Thank you, Kelly. 
 

Kelly Morgan: Thanks, Tony. 
 

Eric Hanselman: And that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun, and Kyra smith on the marketing and events teams. If you like this episode, please subscribe or like us. I hope you'll join us next time because there is always something Next in Tech!