Next in Tech | Ep. 288: Data Center Energy

View Full Transcript

[00:00:00] Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, chief analyst for industry research at S&P Global. And today we're going to explore the complicated interactions of AI and energy. With me to discuss this relationship are Gina Teltsyk, Rich Karpinski, Brian Partridge, and Melanie Posey. We have a gang on today. Welcome back to you all.

[00:00:26] Brian Partridge: Great to chat, Eric.

[00:00:28] Melanie Posey: Thanks, it's always great to be here.

[00:00:31] Jean Atelsek: Yeah. Thank you.

[00:00:33] Eric Hanselman: And well, Jean and Melanie, it seems like the two of you have been on just about every other episode recently as we've been talking about everything that's going on in data centers, energy, some of the impacts, and some of the cost associations. This is something, though, today we're going to look at the other side of that and really look at what's taking place with AI's impacts on energy in various forms, because you have all been at various different events.

[00:01:03] Eric Hanselman: Melanie, Rich, and Brian, you were all at AWS's Energy Symposium, and Jean, you were at the Linux Foundation Energy Summit, all digging into some of what's going on around AI and its various aspects. I guess maybe to start off with, what do you see that has really changed, in this conversation? Energy has typically been an industry that has moved relatively slowly, both given the speed of infrastructure and some of the risk tolerance. We've talked to a number of CEOs on the energy side using AI, but they've typically been looking at outside monitoring, maybe predictions, some of that, but now you're starting to get into discussions about control. I guess, what did you see out there? Brian, why don't you kick us off?

[00:01:52] Brian Partridge: Yeah. So, it's great to be with you guys, I'm in New York for Climate Week and kicking off our integration of datacenterHawk into the 451 Research and the S&P Global Energy team, which is really exciting and we really think brings together sort of the best data center intelligence asset in the industry. But to first sort of address your question, Eric, about sort of what we're seeing around AI and energy, I think the first thing that we need to talk about is that, you know, really energy compute and capital for that matter are no longer the only constraints that we're, we're dealing with in sort of AI and AI diffusion.

[00:02:30] Brian Partridge: And community sentiment has entered that chat, and this is really about, and we've really felt this in the United States, this is about the overwhelming concern, I would say, across the United States around the impact of AI and data centers around energy affordability, environmental concerns, around things like noise and water, and then generally about AI impacts on things like jobs and just society in general. So, we're all sort of dealing with that.

[00:03:03] Eric Hanselman: Oh, well, in fact, you're echoing the concerns from the previous episode with Tony Lenoir and Matt Richardson, who got a chance to dig into it in great detail.

[00:03:11] Brian Partridge: Oh, that's wonderful. And those folks do some great work and have some excellent data. I just wanted to sort of put that out there because clearly that was a sentiment in the rooms we were in last week. On, you know, on the energy side specifically, I think we're seeing some great innovation and investment on both sides of this equation, you know, AI for energy and energy for AI. On the AI for energy side, we're seeing, you know, very interesting innovation about thinking about hey, how AI could be applied to the challenge of producing enough energy to support AI. And you can see that from everything from sort of grid orchestration to helping to shorten the overall workflow around doing things like getting new transmission lines. And then, you know, you heard very loud and clear that the energy industry, the power producers themselves, are doing what they can to sort of step up to the plate on the production side to meet this demand. A lot of different messages in there, but essentially the bottom line is we've got some questions we need to address, concerns in this country, but the people that are building AI and building power for AI are moving forward apace.

[00:04:27] Eric Hanselman: And so this is most of what you were hearing in the AWS side. Jean, I'm curious on the LF Energy side, what were you tackling there?

[00:04:35] Jean Atelsek: So, LF Energy is a utility-driven open source foundation that is working with major system operators, primarily in Europe, but also, you know, others around the world to develop a virtualized and software-defined grid to enable better use of the energy infrastructure that they already have. And that includes renewables, lots of wind and solar power in Europe. So, this was much more, the LF Energy was much more on the energy provider side and how they are working to meet AI that is arriving years ahead of, you know, where they can build new facilities.

[00:05:23] Eric Hanselman: With this level of acceleration, they've got to figure out how to actually optimize what they've got. But interesting to see that that open-source approach in terms of how they're actually trying to address that.

[00:05:33] Jean Atelsek: Yeah. I mean, the providers there have said…and it's very early days for this organization. They had their first graduated project. It's a substation virtualization operating system. All of the parties there, which were major transmission and distribution system operators were delighted by the progress that they've made so far, how it's helping them save costs, and how it's helping them innovate faster. So yeah, it was interesting.

[00:06:10] Eric Hanselman: Well, getting all that data out there and opening up capabilities starts to get toward, everybody's favorite agentic AI topic. Rich, what did you see in terms of agentic approaches? Is this still a no-go zone? I mean, agentic tends to identify autonomy, but there also are agentic approaches to be able to look at, you know, operations and optimization. What were you seeing?

[00:06:37] Rich Karpinski: Yeah, I mean, it's a great question as you framed it earlier, Eric. It's a very conservative industry, and it's very mission-critical infrastructure. If we're worried about turning our desktops to agents for what might happen at that level, think about loosing agents on the electrical grid.

[00:06:54] Rich Karpinski: It can be potentially scary, right? At the same time, moving beyond co-pilots and very simple chatbots is where all the power and opportunity is in these sort of industrial environments like a grid, right? We saw a couple things that sort of address that, right? One is, as they work with agents, as they deploy agents, they're extraordinarily harnessed and constrained, right? They're not letting agents loose on this infrastructure and on these applications. They’re grounded in the standard operating procedures, the SOPs, and the other frameworks and guidance, that's part and parcel of the industry, right?

[00:07:29] Rich Karpinski: So, they're not set loose. They're grounded in how the grid has traditionally worked. They’re grounded in models, physical models, rather than the LLM sort of big frontier models. Very specific models around the physics of electricity, right? And what all those constraints do is let those models work, let them do the things like looping workflows, et cetera. and so, there's a lot of power in that, but they're happening within a very constrained environment. And the big application that AWS showed was not in the control and the execution side of the grid, it was in the planning side.

[00:08:03] Rich Karpinski: So to help operators work through all of the grid interconnection, applications and requests that they have, which number in the hundreds and are now spaced out to be deployed across the years, right? It's a big challenge to get those evaluated. Taking all the documentation, all the work they need to do with that and turning weeks and month-long processes into hours and days, right? To get speedier evaluations of that, right? And so, the agents can come up with forty, fifty different recommendations and then the human in the loop can look through those and again, use an agent to sort even through those fifty and get to a much quicker resolution of this really important topic of how to get people, data centers especially, onto the grid in a timely fashion and in a way that they can manage.

[00:08:45] Rich Karpinski: Those sorts of applications are what we saw out of companies like Duke Energy at the event. So, a large operator really leveraging this technology to great impact.

[00:08:55] Eric Hanselman: Well, I mean, Melanie, we've been talking a lot about agentic approaches to a number of different problems. It sounds like this is a fundamentally different way, or am I, you know, overemphasizing what's changing on the energy side?

[00:09:10] Melanie Posey: I think the way to look at this is, as we all know, AI drastically increases demand for electricity. So, if you look at where we are in terms of, as Rich said, getting these data centers interconnected to the grid, almost as if AI is being marshaled into service to help solve problem that all the AI demand ended up in the first place.

[00:09:45] Eric Hanselman: Fixing the problem it creates.

[00:09:48] Melanie Posey: Yeah, delicious irony of AI for energy and energy for AI, right? But I think the way to look at this is AI can, and agentic AI in particular, can bring all kinds of benefits to any vertical, as far as the workflows and automating some of the steps in the process of getting from, you know, here's the problem, here's the solution. So, we saw that at play in a big way at the AWS Symposium, where one particular use case that is the bottleneck for all of AI is the fact that an AI data center needs this much power, and they need it tomorrow, and the electrical utility can produce this much power, but way beyond a tomorrow timeframe. So how do we compress that timeframe to get the power to the AI then get the AI to the people? And I think the hyperscalers and the data center operators in general are going to play a huge role in all of this because of things that need to come together inside of the data center before AI can work its magic on workflows and productivity and everything else. So, what AWS can bring to the table with regard to this whole process is, it's not just about the models, it's about practical deployment of agentic workflows. And to get that right, you need a whole bunch of other stuff along the way. You need integration, permissions, governance, and just the ability to connect all the bits of the enterprise to the agent that runs in the cloud to orchestrate all of those workflows. So, it's much bigger than public cloud. It's much bigger than AI models. It's AI execution, I would say, and something that's sorely needed in the energy space, given AI for energy and energy for AI.

[00:12:21] Eric Hanselman: Well, talking about the energy for AI, and it, it seems like a lot of these discussions are really reinforcing, you know, the thing that we keep talking about, which is that, power provisioning is that strategic constraint. I thought it was interesting to see that Atul Arya was there, and talking about, you know, powering the AI revolution, multidimensional global energy pathways. I guess Brian, you know, this is something that we've been talking about, how do we actually move this forward from just a power availability perspective? What's your take on that?

[00:12:56] Brian Partridge: Yeah, no, it's always great to watch Atul work. He is an incredible asset to S&P Global, leading our overall research business for energy. Multi Pathways is all about the global transition, if you will, but also expansion of energy sources across the world. Of course, how this happens very much depends on the region of the world that you, that you're talking about and what their sort of domestic resources are. Of course, we've had all sorts of geopolitical shocks to, you know, those energy pathways. But what Atul was talking about, you know, coming back to the AI discussion, just talking about how much energy security has come back to the forefront discussion for many, many governments, probably all governments around the world because of the importance that's being placed on AI and AI sovereignty.

[00:13:58] Brian Partridge: You hear things like compute sovereignty. These are becoming sort of national security concerns. So, you know, Atul was sort of laying out what that could look like and clearly what AI has done. It's created a lot of near-term demand for firm power sources like LNG. It's also created, by the way, a ton of demand for renewable sources, as we can see by the amount of, you know, renewable PPAs that the hyperscalers procure, but it's still not enough.

[00:14:33] Brian Partridge: There's also, you know, a lot of expectation and excitement and quite frankly, investment from the hyperscalers into firm power options like nuclear fusion and fission. So what AI has done is taken an energy market that has been, you know, fairly steady state, in The United States anyway, in terms of growth, and sort of shocked it. And then you combine that with what's happened in terms of geopolitical conflict, and COVID even in that mix, and it's just, it's just created an incredibly complex sort of dynamic.

[00:15:11] Eric Hanselman: Well, I, I guess, so how do utilities and hyperscalers actually start to address these problems?

[00:15:18] Brian Partridge: How can they continue to build, without putting disproportionate either costs or reliability risks on the grid itself? Well, there's a number of answers to that. And the first thing to say about a hyperscaler is the default position in terms of where they want to source power is the grid for a number of reasons, which I mentioned, right? Because you can get you know, a good amount of the pie chart of that power can be renewable when you go to the grid. But of course, the other option that has reared its head is to co-locate power with these new data centers, in particular the AI data centers, very large load campus environments where there's just no grid capacity. And so then, you know, you've got a large industrial project that brings both data center and compute capacity and power online at the same time. Those can stand alone as islands of power, if you want to look at it that way. But, but ideally and optimally over time, what we'll see is that even those islands of power, which are stood up as quickly as they possibly can, and of course, the constraints on supply chain and permitting affect those projects as well. Ultimately, you know, the idea is to get all of those assets grid connected, over time.

[00:16:36] Rich Karpinski: Yeah. And Eric, that's an equal challenge for the grid operators as well, that while that relieves some of the capacity challenges placed on the grid, they really want to be in a role of helping to orchestrate and coordinate that behind the meter capacity against what they already have.

[00:16:52] Rich Karpinski: As Brian said, large consumers, be they data centers or industrial sites, they want to be connected to the grid. So, they don't want to be all on, they don't want to be all off. They want to be a little bit of both, right? And what the grid operator needs to do, and this is a great application of AI and analytics and real time, understanding of where the data flows are and the energy flows are, they want to be able to orchestrate when to turn on that off the grid, when to have a battery system need to be activated, when to know when they should be filling up a battery rather than draining the battery system, knowing where there are virtual power plants or micro grids in certain areas or regions that can be accessed.

[00:17:28] Rich Karpinski: And that makes the grid much more of a two-way enterprise, which is not how it's obviously been, delivered or operated traditionally. And again, it places a lot of that responsibility on software, on things like AI, on the ability to pull data from these systems and analyze it in real time. So that's a big part of the, let's see, the AI for energy part of the story.

[00:17:50] Eric Hanselman: Well, Jean, I mean, that sounds like that was exactly what was being addressed at LF Energy

[00:17:55] Jean Atelsek: Yeah, precisely. I would say a lot of the projects that LF Energy is and that the system operators are supporting have to do with analytics. Getting reliable short-term forecasting of grid capacity and basically, you know, having the visibility. And it's a theme that we've seen among enterprises too, in terms of their use of AI, is that, a lot of these different systems have different data formats.

[00:18:29] Jean Atelsek: And so normalizing or having standard protocols for the data, is sort of a foundational problem that has to be addressed before these multi-directional flows can be managed. It's not unlike, you know, load balancing on a computer network, right?

[00:18:52] Eric Hanselman: To making sure, yeah, where you need it at the right point in time with, you know, the various sets of supplies that you've got. Yeah.

[00:18:59] Jean Atelsek: Exactly. Yeah.

[00:19:01] Melanie Posey: Eric, if I could add there to Jean's point, one of the problems in being able to create these systems that are responsive to demand requirements is having the data in place. Because as we all know, AI ends up giving you garbage out if you put garbage in, right? So, one of the things we heard from some of the utility providers at the AWS Energy Symposium is that a whole lot of the work in setting up these agentic workflows for interconnection to the grid has to do with the data. Basically, getting the data cleaned up, getting the data sorted and structured the right way. And here's where technology companies like AWS and the other hyperscalers can come into play, because they can basically help you with that data pipeline and be able to help you set up systems that are reliable and resilient, which can respond to those signals the market.

[00:20:23] Melanie Posey: And to a large extent, one thing that the utilities also  have to do is make sure that all of this power that's going to the AI data centers somehow benefits the community at some point, that there are some broad system benefits that come from these power build-outs that are needed to power AI center build-outs too. It can't simply be a case of the risk of all of this gets transferred to the ratepayers, that a benefit that is shared by everybody else at the end of the day. So, one way to look at this is the hyperscalers and the AI data centers all need to have with the energy companies, and the energy companies need to automate and agenticize, if that's even a word, their workflows so that cost-benefit picture, that equation, stays in equilibrium for everybody.

[00:21:36] Eric Hanselman: Agentification, I think, is where we're trying to or, or some other good made-up word somewhere around there. But, well, but access to data, you're pointing out, just that really key part of, you know, Gene, you're talking about how things get opened up, but, is this just sort of slapping some MCP servers in here?

[00:21:54] Eric Hanselman: I mean, it seems like one of the things that, from the AWS side, that we're talking about was being able to open some of those doors to data, but doing that in ways that had agentic functionality, actually leveraging Quick, which I thought was an interesting take. It gets back to that question of do you have a solid enough base and enough context? How do you put that together? I guess, how do you see that in terms of the approach and is it just sort of plugging into, you know, an MCP backend of some of the utility data going to be enough for that?

[00:22:30] Rich Karpinski: Yeah, I'd be happy to take that. It was definitely a lot of slapping MCPs in front of data sources as a solution at the AWS event. And the energy operators we talk to can make that work in some ways, right? aAgain when you get into live control of a grid, maybe you want something a little more robust.

[00:22:48] Rich Karpinski: You need a data lake, you need a fully mapped data environment, data fabric. But for some of the things that they want to do, so like again, this planning application where you're just evaluating incoming applications and you're looking at all of these SOPs and you're looking at all of this massive amount of data, you can put an MCP server in front of that and you can let the agent and the model do a lot of work that would take a team or an individual engineer much more time. So there is going to be, and there is in the market, some different approaches to solving this problem. And this is an ecosystem from the frontier model providers who all are working and all were at the AWS event, and I'm sure they were all talking in Jean's world as well. And then to the infrastructure providers like the AWSs and their competitors, the industrial software providers under it. And then there's a data layer fabric in between, the Cognites and who was just recently acquired by Schneider. So, like this, this is very much the right approach or the best approach for the right application of the right use of data within these kinds of agentic frameworks is very much in play, and I think it's one of really one of the more interesting questions.

[00:24:01] Rich Karpinski: And I'm sure that's some, I know that's something that Melanie's team is looking at in a broad way, and my team, which looks mostly at industrial sorts of applications, is looking at in that industrial software ecosystem, how they're trying to figure out this data layer challenge.

[00:24:15] Eric Hanselman: If you think about some of what's going on, this is definitely the place where I think most of those concerns lie, which is ensuring that you've got the right data and that you got access to it in ways that are actually going to make sense and be able to be actionable and available to the right agents at the right times.

[00:24:32] Melanie Posey: And Eric, I think one thing we're going to see come out of all of this is, the importance of having the technology piece work together with the specific requirements and the nuances of all the different workflows that organizations across different industries have. So, to the earlier point about slapping an MCP server in front of everything, just table stakes.

[00:25:03] Melanie Posey: That's just the underlying plumbing, if you will, of making a whole agentic actually operate properly. So, to the extent that nobody can go it alone, this is more true than ever when we talk about industry workflows. It's that you need both pieces. You need that technology piece, you need the orchestration across all the different pieces, and you need that domain knowledge that comes from the industry, in this case, that comes from the utility industries to really keep the AI flywheel going.

[00:25:50] Brian Partridge: Yeah, and Melanie, I would just jump in here, right? And what you just described, I would say this is true for every industry. That's very hard to do. But it's also interestingly where the real scale opportunity of ROI for AI sits. But when you think about what you just said there, it really requires a complete deconstruction of existing workloads and, you know, to make it simple, make them AI ready. So how do I think about deconstructing my workflow and then inserting the right checkpoints and human in the loop interventions across that workflow, which would be completely, you know, different than how I executed the workflow two years ago. I think that's a journey that, you know, most organizations are on now. Some are making good progress there, but, you know, there's also cultural inertia. Sometimes the tools aren't quite ready to do that complex of a job. And so, I think that's where the part of the AI diffusion that we're really sort of in the early stages of lies.

[00:26:54] Eric Hanselman: It's an ecosystem that's changing so rapidly. I mean, this is something that even just keeping up, you got to be moving relatively fast. And that means for, especially in industry specific capabilities, there's that big risk that you fall behind, and that's a challenge.

[00:27:12] Well, I want to sort of get us wrapped up here. We have covered a lot of different subjects, but I want to task each of you for a quick summarization. Can you do a five-word summary, really what your takeaways are and what this expectation is for AI in energy for the future? And, I don't know, who wants to jump in first?

[00:27:37] Jean Atelsek: I'll start. So, with only five words, it's hard to fit all that.

[00:27:42] Eric Hanselman: But that's the point, Jean

[00:27:46] Jean Atelsek: But I'd say, the LF Energy approach that they're taking is that openness can be a superpower.

[00:27:54] Eric Hanselman: Ah, all right. Brian, you want to take the next one?

[00:27:58] Brian Partridge: Yeah, I would say AI and energy must be single system.

[00:28:06] Eric Hanselman: Critically important. All right, Rich.

[00:28:10] Rich Karpinski: Yeah, I think AI, that's not one of my words, but, expands demand, but AI also is necessary for optimizing and orchestrating supply. So expands demand, optimizes, orchestrates supply,

[00:28:23] Eric Hanselman: Oh, a good summary. All right, Melanie, you've, you've had the most time to think, so.

[00:28:31] Melanie Posey: Oh, it's-- that's so unfair. So much pressure. I would say simply, from standalone assistants to governed agentic orchestration. So that's where we're headed right now. It's not about assistants, it's not about chatbots, it's agentic orchestration to enable workflows. And I know that's way more than five words, but there you are.

[00:28:58] Rich Karpinski: We were counting, yeah.

[00:29:00] Eric Hanselman: All right. Well, Brian, you brought us in. Why don't you take us out with a summary of what we've been really putting together and what you think that our listeners should really take away as the real serious highlight?

[00:29:13] Brian Partridge: Look, I want to come back to the datacenterHawk acquisition. I think that's really important. If anybody out there wants to learn more about what that's going to mean for you or for 451 and S&P Global, just let us know. And hey, this is a great time and very interesting and dynamic time in our industry, and we look forward to continuing to bring the insights that matter, to our stakeholders. It was great to have this discussion today,

[00:29:40] Eric Hanselman: Well, and many more to come, and we'll point our listeners to a lot of what we've been talking about in the show notes here. But that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Dylan Scheible, Sophie Carr, and Feranmi Adeoshun on the marketing and events teams. If you liked this episode, please subscribe or like us. I hope you'll join us for our next episode, because there is always something next in tech.