Next in Tech | Ep. 187: Datacenter and Energy Innovation Summit

View Full Transcript

Welcome to Next in Tech, an S&P Global Market Intelligence podcast with a world of emerging tech lives. I'm your host, Eric Hanselman, Chief  Analyst for Data and Insights at  S&P Global Market Intelligence.

And today we're going to be continuing 

a discussion that we've had about data 

centers and energy, so AI on energy 

consumption, and we're going to do it 

around an upcoming conference that we're 

putting together, the Data Center and 

Energy Innovation Summit that's going 

to be taking place in Washington, D.C.

on October 30th.

And to discuss that with me and a 

lot of the topics we'll be digging 

into at the summit, I've got three 

members of the analyst team, Kelly 

Morgan, Steve Piper, and Dan Thompson.

Welcome to the podcast to all of you.

Hi, Eric.

Thank you.

Happy to be here.

Thank you, Eric.

Yeah.

Thanks, Eric.

Glad to be back on.

And Dan, actually, of the three of 

you, you've actually been recently 

on the podcast talking about some 

of the concerns that are out there.

The thing that I think catalyzed a lot 

of this were the projections of Just 

simply the volumes of energy that we were 

seeing in terms of what's going to be 

created as we look at this wave of GPU 

driven consumption that's taking place.

This has really been something that 

we've been working to be able to 

make reasonable predictions about.

There are a lot of sort of wild eyed, uh, 

statements that have been made, but this 

is really getting back into an area there 

where we've been spending a lot of time.

But I guess, can you start us off 

just with a little bit of background, 

just to catch those of our listeners 

that don't have the background 

haven't maybe caught some of these.

Yeah, sure.

So yeah, I mean, I would encourage 

anybody listening to this to go back 

and listen to some of the great work 

you've done on this podcast already 

with some of the other members.

A lot has been uncovered, but at 

a very high level, essentially the 

conversation that we have been having and 

intend to continue to have in Washington, 

DC is really taking a deep dive into data 

center growth across the United States.

We'll actually talk on some level 

globally as well, but mostly what we'll be 

discussing is here in the United States.

But what we've seen early since COVID 

is that the hyperscale cloud providers 

have been building out more and more 

space in terms of data center capacity 

across the globe and also here in the 

U S and during COVID it accelerated.

And then in the last couple of years, 

since the advent of chat GPT and the 

world was introduced to artificial 

intelligence, specifically generative AI.

It seems like that pace has yet again, accelerated.

This has of course gotten folks excited.

It has gotten folks excited.

Worried, probably every range of 

emotion that you can think of in terms 

of reactions to this, simply because 

the amount of power that a very small 

handful of companies are requesting 

of our, our nation's power grids is 

quite frankly, it's, it's outrageous.

And so, what we want to do with this 

particular data center event is really 

dig into it and say, what's driving you?

Where do we see this going?

Is it a bubble?

As some folks have suggested or 

questioned, and what do we see?

You know, like what, what are 

the utilities going to do?

What are investors doing?

What, what can we expect for the next 

12 to 24 months in this whole realm?

And so, I hope we're able to answer 

a lot of those questions and kind of 

unearth some new insights for that.

But, you know, again, for diving at here, 

what we're trying to get at is artificial 

intelligence can be very power hungry.

And we have some guesses as to where 

it's going to be deployed here in 

the United States and elsewhere.

We have some of our own estimates as to 

how much power that was going to require.

And as you mentioned, Eric, 

everybody has an estimate.

And of course we have ours as well.

And we want to surface that with 

our audience attendees and talk about 

where we believe it will be built, 

how, and is it all AI or is it.

Cloud stuff and AI and what does that mean 

for, for the rest of the audience, whether 

it's the investment community or the 

data center community, things like this.

Well, I mean, talking about investment, 

I mean, this is something where there's 

already been reasonably healthy growth 

and Kelly, that's what you and your team 

track with the data center knowledge base.

And this is an environment in which there 

had already been a reasonably strong 

growth in these markets already, right?

Yes, yeah, and that's one of the 

challenges and we've talked about 

this with many of our customers that 

even before the launch of chat GPT 

in many areas, it seemed as though 

data center growth was not enough, 

the cloud providers and other folks 

were constantly wanting more capacity 

or seeming to need more capacity.

There was a real race to try 

to get enough capacity built.

So that's definitely continuing.

And then I think on the investment 

side, investors have certainly 

become more interested in this 

sector over the past few years.

I think partly just because so much 

capital is needed or has been needed 

to build these and certainly will 

be going forward, but also firms 

are getting a lot more creative 

about how to finance these builds.

And a lot of investors are just realizing 

that this is a sector that could be 

interesting as an investment sector.

So, we've already been seeing a 

lot of change and I would say 

innovation really, and in what was.

Kind of a conservative traditional 

sector up until a couple of years ago, 

there was definite innovation, but 

maybe at a kind of slow, steady pace, 

and now all of a sudden that's really 

been pushed to the limit, both in 

terms of financing innovation and the 

technical innovation required for some 

of this AI equipment in data centers.

Well, it's something where, to your 

point, this was data center buildouts 

were compared to a lot of things 

in technology were relatively long.

timeline.

You know, we're talking about 

years of planning and such.

And yet now we've got some significant 

pressures also being exerted 

on another area that has really 

long comparatively timelines for 

planning, which is the energy sect.

And Steve, I mean, this is This is 

a level of disruption that really is 

swinging the needles in terms of a lot 

of long-term planning, the inputs for 

planning, in ways that the energy industry 

really hasn't been used to, right?

That's right, Eric, and 

especially power, electricity.

We are rapidly sort of 

approaching the point.

I think this year has been 

a watershed year in energy.

Coming to this understanding that the 

requests of the tech sector, the, the 

data center and kind of AI developer 

sector, the requests to access the 

grid are outpacing the grid's ability 

to easily accommodate those requests.

And especially in those parts of 

the country where data centers have 

traditionally sort of expanded and 

grown and where they're comfortable.

So, we see ourselves confronting a very intense competition 

for access to the grid, to the electricity supplies, to the 

capacity of the transmission system.

All of those elements of the electricity 

supply chain now are a matter of keen 

focus for data center developers as 

they try to figure out how to line up.

The power they need to launch 

the applications that they want 

to build in this new AI world.

Well, and we've got a, an example 

recently of some of the links 

to which they're willing to go.

The constellation energy deal with 

Microsoft is just the latest exemplar 

of what some might see is a pretty, 

I was going to use the word extreme, but certainly dramatic 

shift to be able to, to get capacity.

It's a very new paradigm, right?

And it comes on the heels of another 

deal between a nuclear plant operator 

in Pennsylvania also, and Amazon 

earlier this year, this deal with 

Microsoft to take power from the restarted 

Three Mile Island reactor puts down a 

marker for just how important access 

to the electricity is from the 

perspective of both kind of, okay, nuclear power provides large amounts 

of stable zero carbon electricity.

That's still a zero carbon, still a 

very high priority for the tech sector.

And also, if you look at the deal that 

Amazon signed; it essentially bypasses the 

wholesale grid by providing a dedicated 

connection and this is getting to some 

of the technical details, but a dedicated 

connection to the nuclear power plant.

I don't know about the Microsoft 

specifications of their deal with Three 

Mile Island as yet it was recently 

announced, but there's both this component of finding the generation.

Like a nuclear plant, but also dealing 

with the grid infrastructure to 

get that power to the data centers.

These are both examples where 

these large tech companies are 

very focused on both aspects.

Well, that's the direct line to this.

One of those things, you know, from 

our power station directly to you.

What's interesting about that, 

Derek is like, I think if you listen 

to the conversations happening at 

data center conferences this year.

There was a whole lot of 

emphasis put on the small 

modular nuclear reactors or SMRs.

I mean, it was like all anybody 

wanted to talk about this year.

And what's fascinating when 

you dig in with Steve and his 

team, there's a definite nod 

towards, yes, that would be cool.

But the development 

timeline on that is huge.

And admittedly, like, if you read 

the news announcements for this Three 

Mile Island deal, I mean, the timeline 

on this is going to be long as well.

It's going to take them a while 

to get this thing restarted.

2028, I believe, is the timeframe that I think that's right. 

Refueling and restarting a nuclear reactor 

is not something you do over a weekend.

Right, right.

But where I was going with that is that 

the interesting thing about this is 

there was so much focus and emphasis 

putting on SMRs that it seemed like 

everybody forgot about what about 

existing nuclear facilities that have 

been turned off for whatever reason.

Could we restart those and would that 

be quicker than deploying SMRs at scale?

And it's just fascinating.

I hate to call a trend, something a trend 

when we just have two examples, but I 

mean, these are very expensive examples 

and very large examples, but in any case, 

it's just fascinating turn of events.

For me anyway, I perceive it 

may just be like, Oh yeah, this 

is just another Tuesday, but 

I suspect that's not the case.

The play with respect to the grid 

infrastructure is pretty interesting 

in the sense that back when we built 

out the nuclear fleet in the seventies 

and eighties, the infrastructure 

was sized for many more nuclear.

Generators then were 

ultimately constructed.

So that puts some infrastructure 

in place now that is in some sense, 

accessible to the tech sector.

And as Dan pointed out, we've also, as 

a country, been shutting down nuclear 

plants, and that creates the capacity 

to maybe recommission those plants.

Or to use that infrastructure, you 

know, another example out there might 

be the Palisades plant in Michigan.

That's looking at perhaps reopening.

It creates the possibility of in 

a fairly short time with minimal 

additional infrastructure investment, 

you're getting access to the 

large amounts of power that the 

companies want, it probably does 

point to a trend where people are 

going to look at that, where 

that infrastructure is available.

The question is, yeah, how 

much headroom does it have?

How many opportunities like that exist today?

And that will kind of form the upper limit 

of the activity we see in this space.

Well, I guess that question is exactly how 

much more capacity can you bring online?

How quickly, but the other side of 

this coin, of course, is managing 

that demand growth and looking at 

moderating that in some form or fashion.

And I know one of the other things that 

the summit's going to be talking about is.

You know, potential efficiency gains 

from being able to improve the operation 

of the data centers themselves.

And you've talked about power 

utilization, efficiency, water 

utilization, efficiency, Kelly.

There are a lot of things that 

we're looking at from an operational 

perspective that could potentially help.

Both existing in new builds and some 

possibilities there, at least in 

terms of some level of savings, right?

Certainly, yes.

And I think it's a combination.

I know in our team, we've done a 

lot of work on going down to the 

very basic level of the chips.

And of course, one of the key 

questions in the industry right 

now is how efficient the chips will 

become, especially for AI uses.

So, there could be some 

improvements there.

I think everyone's kind of hoping 

there will be some improvements there.

And then all the way up through the 

types of cooling and the way data 

centers are structured for AI, that 

needs to be quite different in terms of 

how the data center itself is organized.

But of course, the cooling 

needs to be very different is something 

we'll be discussing, of course, but yeah, 

there's room for more innovation, I think, 

and a lot of firms are working on this, 

how to make the data center more efficient 

while also maintaining the reliability 

that you need, especially for AI.

Where you have very little room for 

error and uptime has always been a 

particular concern, but I guess the 

thing that that we've talked a little 

bit about this on previous episodes, 

which is that we talk about efficiency 

gains, where we're talking about tens 

of percentage points improvement, as 

opposed to having energy use altogether.

So, I guess there are some limits 

on how far we can go, even 

looking at technologies like 

liquid cooling, of course, which.

In some cases, now with high 

performance GPU clusters, you were 

talking about how data centers 

themselves need to be reconfigured.

I mean, now GPU clusters consume so 

much more energy in such a smaller space 

that you now got to look at alternative 

cooling technologies to move beyond air 

cooling to technologies like liquid.

Definitely.

We'll be talking about that.

Certainly.

Yes.

And we think about some of the 

other pieces that are in play.

And a lot of this, again, we 

touched a little bit about what the 

concerns are for the investment community.

This is a big piece that starts to 

drive where investment dollars go, 

which is that placement available land 

available power, this starts to open 

up potentially new geographies of if 

we've got the energy capacity to be able 

to meet it, I'm guessing you're going 

to be touching on those areas as well.

Yeah, no, that's right.

So, we've, we've got a, an 

entire panel that looks at.

Just where our data center is being 

built, where could data centers be built?

Uh, and really hoping to 

dive into the particular.

So, so one of the companies that, 

that is already signed on to 

be on that panel is DC Blocks.

So, they'll be able to speak directly 

into why they chose pieces of property.

Over other pieces of property.

Cause this is a constant question that we 

get from our customers as well as why are 

the data centers choosing to go there?

What was it about that specific piece of 

property that was interesting to them?

In some cases, you know, in 

some cases we know, but it's great.

It's going to be great to hear 

from the data center company.

to kind of give some insight into 

what it is that they're looking for.

Uh, but to your point, I mean, 

we've seen some interesting examples 

already just this year, right? I think it was either Microsoft 

or Amazon for getting off the top 

of my head, announced that they 

were building in Mississippi.

There has been lots of stuff going 

on in the media recently about XAI, 

which is now deploying in Memphis.

So, this is one of Elon Musk's 

companies that is working 

on artificial intelligence.

They chose a former manufacturing site.

In Memphis, it just so happened 

to have power available.

And then they've also X has XAI have 

brought in their own generators.

They have been running off the 

grid or in addition to the grid.

It's a bit unclear exactly what they're 

doing, uh, but they have a lot of power 

capacity that's there sitting on site, 

and it's going to be powering this 

supercomputer, which Elon claims to be the 

largest in the world yet to be decided.

But in any case, we've also seen 

developments in North Dakota.

So, in some in Cheyenne, Wyoming, you know, 

these are not places that we on the data 

center teams are typically talking about 

in terms of data center development, 

but it's now thrust into the limelight.

Because the area happened to have 

available capacity, like Steve was 

talking about a minute ago, there 

was infrastructure sitting there 

already, or there was some willing 

party to get it there quickly.

And so, it is definitely changing the conversation.

I think we all would agree that data 

center capacity in metros, in major 

metros, will continue to be important.

But this crazy new world that we're living 

in seems to be opening up opportunities 

for basically you tell me where I 

have power or where you have power 

available and we'll figure the rest out.

Seems to be the current conversation, 

but how long that will go on into 

the future and can we expect this 

to continue will definitely be 

a conversation at the program.

In real estate, the joke is 

location, location, location.

When it comes to these data center 

expansions, it is electricity, access, 

electricity, access, electricity, access, 

as these examples make abundantly clear.

And then to, to Kelly's point, if 

you do get 20 percent efficiency on 

the energy side, that, that means 

you have a less grid footprint.

print, less power footprint that 

you need to serve, that can be an 

access advantage to, you just don't 

have to acquire as much necessarily 

to move your project forward so that 

that becomes a business imperative.

Well, it's one of those things we keep 

kicking around, uh, from the broader 

team and really looking at the logistics 

of workload placement to manage demand.

To get to a point at which we can now, 

I mean, right now, most of that, the 

heavy-duty training activity is what 

consumes so much of the power, just 

because there's so much work that 

has to get done to train the models.

The models that are being trained 

right now, are these multi billion 

parameter, large language models.

And, and what happens when we start 

to shift to a world in which models 

get a little more compact, where they 

get sized more appropriately, right 

now, we don't have a lot of tools 

to really understand what are the 

limitations of models if quality versus 

size for a particular application.

So, everybody goes big and maybe 

there's a point at which we start 

to size models a little better.

Maybe there's a point at which 

we can do the training in remote 

locations that make more sense 

where there's energy availability.

So, a lot of different pieces that 

are in play to really start to 

make sense of what that does.

Demand profile looks like where it 

gets put together and where it exists.

I mean, we've existed so long in 

an environment where data center, 

physical proximity was an advantage.

We'll look at Northern Virginia, right?

Dan, I think what you phrased was the, 

the power paradox in Northern Virginia.

with Dominion Energy having 

to restart capacity there.

Not everybody's got that opportunity, 

but maybe this is something that 

starts to drive data center builds 

in much more far-flung locations.

Yeah, absolutely.

And I think we've got some early evidence 

in our voice of the enterprise surveys 

that suggest where companies are deploying 

AI infrastructure today versus where 

they hope to in the next two years.

Uh, and that will be, of course, 

on display at the data center.

Uh, and so we'll be 

able to talk through that.

And, and I have to say like some of the 

data is a bit surprising where folks are.

are hoping to deploy this.

I think co-location providers are 

in for potentially a good win as 

are just enterprise data centers.

It seems like up until this point, most 

folks have been doing it in the cloud.

We believe they will continue to 

do this in the cloud, but also.

Uh, in these other venues that I mentioned.

So, we'll definitely cover that.

Uh, and yes, I, I, I 

wholly agree with you.

It seems like the training aspects of 

it, of AI can happen almost anywhere.

The question that needs to be answered 

is when we get to the inference or 

the interaction with the AI models.

Where can that happen exactly?

And does that need to be 

proximate or can it be in some 

cases far away from the users?

This is, I think that's what we're going 

to be talking about in 2025, I believe.

We'll see.

Well, it's that question 

of, can you do training?

You need to do training where you have all the data.

Do you have to do the training 

where the data is being generated?

Do you have the interconnection capacity 

to be able to move the data someplace 

else so you could do the training in a 

place that is more energy appropriate?

A lot of different things 

to get one's hands around.

And of course, how do you 

actually, start to manage it?

How do you pay for all this?

We already are in an environment where 

the resources that drive a lot of 

this, the GPUs that are behind it are now 

actually, being leveraged as part of the 

financial structures of what's going on.

Kelly, one of the things we've 

talked about previously is the idea 

that you've now got some of these 

alternative GPU providers that are 

actually, using their GPU infrastructure 

as the collateral behind some of 

these deals, which is a little mind 

boggling on some level as well.

Yes.

And the amount of money that goes 

into these pieces of equipment from 

the chips up to the servers and 

on into the data center itself is 

pretty dramatic, not to mention.

The capital that will be required 

to bring new transmission lines to 

some of these places and then on 

out to the electrical infrastructure 

and generation even beyond that.

So yes, lots to talk about in 

terms of the funding of all this.

Well, so many different topics to cover.

I will refer our listeners 

to the show notes.

We'll have a link to the summit and 

some information on the agenda, but I 

will point them in that direction 

in terms of deeper conversations.

But this has been a great overview.

A thank you to all three of you for 

being on the podcast and digging 

into what is a very complex subject 

with so many different repercussions.

Great.

Thank you, Eric.

Thank you, Eric.

Thanks for having us on, Eric.

Look forward to seeing 

you in Washington, D.

C., sir.

And hopefully we'll see a lot 

of our listeners there as well.

But that is it for this 

episode of Next in Tech.

Thanks to our audience for staying with 

us and thanks to our production team, 

including Sophie Carr, Feranmi Adeoshun, and Kyra Smith
on the marketing and events teams and our agency partner, The 199.

I hope you'll join us for our next 

episode where we're going to be 

talking about cloud native application 

security, looking at some of the 

nuances about what's different.

In cloud environments and in some of 

the new patterns that are being used 

for application development, I hope 

you'll join us then because there 

is always something next in tech.