Listen: Next in Tech | Episode 123: Generative AI markets

View Full Transcript

Eric Hanselman

Welcome to Next in Tech, an S&P Global Market Intelligence podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Technology, Media and Telecom at S&P Global Market Intelligence. And today, we're going to be discussing artificial intelligence and the markets surrounding it with returning guest, Nick Patience, who heads up our research in AI, ML and all things in that end.

Nick, welcome back to the podcast.

Nicholas Patience

Thanks, Eric. It's good to be back again.

Eric Hanselman

And great to have you on, especially with all of the craziness that is generative AI right now. There's a lot that we've been hard at work on in terms of really looking to assess this market and the dynamics, the players, market projections and the rest of it. We've been through some crazy times of late.

Nicholas Patience

We certainly have. If you think of -- I guess, at the time of recording, we saw 6 and a bit months after the introduction of ChatGPT on November 30, 2022, and I've been reflecting a bit on that. So since that time, what's happened? So governments have been forced to rethink their technology strategies, in some cases industrial strategies, their strategic alliances. It's made AI pioneers reconsider their entire life's work and predict the end of humanity. It's accelerated the rush...

Eric Hanselman

Other minor things like that, yes.

Nicholas Patience

Yes, things like that. It's also accelerated the rush to regulate AI, which was going to happen, anyway, but it's forced a lot of jurisdictions to speed that up. And it's created a run on GPUs, increasing in that process, NVIDIA's market capitalization by -- I calculated, by 137% from November 30 to May 30 and then up 154% up until today. So that's a pretty good effect for what some people just call a chatbot. It's had some pretty seismic effects.

Eric Hanselman

You would think it is seismic in so many different dimensions, but there's a lot that's in play around this. And given this very short run-up, we've seen some really dramatic evolution in just the way in which really people are thinking not only about AI broadly, about the specifics of generative AI, about its applications, really all of the different dynamics that are taking place within that market. There's a lot that is in play, and it's an environment that is evolving very rapidly. If we think about what those early days of ChatGPT, of just the text prompt had started off with, we've come a long way since then in what those implementations look like, what the applications look like and really some of the conceptual thinking around what really generative AI is all about and really where it fits.

Nicholas Patience

Yes. Exactly. On that last point, I guess, on the kind of conceptual level, I mean, we've gone from patent classification in what we now call somewhat or strangely traditional AI or traditional machine learning, so from patent classification to content generation. We've gone from narrowly focused machine learning to potentially extremely broadly applicable generative AI. We've gone from deterministic to a certain extent and then predictive algorithms into purely probabilistic things. So there's lots of different ways this shift has happened.

I now see vendors and to a lesser extent, customers but also investors thinking about traditional AI versus generative AI. And that's not to say traditional AI and its kind of -- all its kind of ability to make predictions has gone anywhere and gone away at all. And it's totally useful for so many different use cases. It's just that this is -- this looks and feels like something completely different.

As you know, I've been looking at this space for over 23 years now, and I've never seen anything like this in terms of the level of interest from our customers and from segments I have never had any interaction with. And it's swamped us. And it's been really good and we've been really busy, but it's more than hype. There's something pretty fundamental going on here, but that's not to say that everything that went before doesn't have any value. It does. I think it's a question of putting all that into context, really.

Eric Hanselman

In 6 months, suddenly we've got an old-school version of AI.

Nicholas Patience

Exactly. It's pretty wild.

Eric Hanselman

Well -- but even within generative AI, the ideas and what we've been thinking about is what it constitutes and how you work with it have changed. Because we've started out with text prompts, and that's rapidly evolved into image manipulation and image creation. And the integration of all of these different pieces and even the way in which we're working with the various implementations of generative AI are evolving very rapidly. We're now starting to see some of that melding of the different forms that it takes. The prompts' structure or the prompts' mechanisms, that's evolving rapidly as well.

Nicholas Patience

Yes, it is. You're right. It was very much text first and let's see what happens. And then sort of image was a separate thing. And the other elements like code or video were separate, but they are starting to merge. But a lot of -- I guess a lot of the emerging is using text prompts to generate something else other than just text. But text is also obviously vitally important. But using text to generate images, to generate code, to generate video, to generate audio, that's happening and it's happening incredibly quickly.

There are so many start-ups in this space and there are so many open-source models out there as well. There's potential for confusion as to where to place bets or to make investments, whether you're actually investing money in the companies themselves or you're choosing to buy or build these things in house. And there's a lot of questions a lot of organizations have to ask themselves.

For instance, fundamentally among them, do I need a foundation model of my own? Not everybody does, and these things can be, as we know, extremely expensive to run. There's a whole -- I have a whole kind of flow I can walk clients through as to whether you need these or not. And a lot of it is to do with obviously infrastructure in house or the ability to pay for cloud-based infrastructure and skills in house or the ability to pay for systems integrators to come in and help you.

They'll need some clarity. And that's why, I mean, the thing we're going to talk about today a bit is why we've decided to do this generative AI market monitor and try and put some stakes in the ground really.

Eric Hanselman

And speaking about that specific research, this is something where you've got an AI, the word for it was a market monitor which basically is a market assessment that's taking a look at this market and trying to at least scope and scale what the market looks like. Can you give our listeners a little background into what that is and how it goes about evaluating the market?

Nicholas Patience

Sure. So we looked at 7 segments essentially of the generative AI software market. And I mean, just to clarify on the software stuff, we're really focused on that. We're not focused at the moment in a market forecast on the infrastructure underneath it. But the 7 segments are foundation models themselves. And then we looked at text generators, image generators, code, video, audio and structured data generators which are often used to create synthetic data. So there's kind of 6 types of data and then the foundation models underneath them.

And then we do -- here at S&P Global Market Intelligence, inside the TMT group, we've been doing these market forecasts which, as we say, we call market monitors. Well, we've been doing these market forecasts for a long, long time using a kind of rigorous bottoms-up analysis. And my colleagues and I, we went through and we identified a total of 262 vendors in the space overall and figured out various -- the various growth rates of the subsegments and overall.

So just to give you the kind of headline number, we calculated from 2023 to 2028 a compound annual growth rate of 58% from a fairly low base at the moment of something like $3.7 billion rising up to $36.4 billion in 2028. And then within that, there are various parts of the market that grow faster than others. And obviously there's various parts of the market that have many more vendors than others.

So just one little number on that last point, of the 262 vendors we assessed in this forecast, 117 of them alone were text generators and only 19 were foundation model companies or providers of various kinds. So there's a lot of disparity in terms of where people are playing in this market.

Eric Hanselman

And in the various participants. Well, I think it's worth also identifying the difference between a bottom-up and a top-down market assessment. And you've touched on that. But I think providing a little bit of clarity, what we're doing is we're actually looking at the various companies in the market and valuing their various contributions as opposed to the top-down which is creating a model of what that full market could potentially look like and looking at all of the potential use cases, an approach that's at least sort of our thinking, not as grounded as the bottom-up model and not -- or less likely to give you a real feel for who's in the market and who those market participants actually happen to be and their individual contributions.

Nicholas Patience

Yes. That's right. And we give -- we're pretty -- I would say we have a very rigorous process. And we're pretty confident that this results in an accurate -- as we stand here in 2023 anyway, obviously you're forecasting the future. And as we know, predictions are hard, especially about the future as somebody once said.

Eric Hanselman

Yes. It's the old Dutch saying. Yes, indeed.

Nicholas Patience

Exactly. So we look at -- so the advantage of doing it this way, we think, gives you an accurate picture of that forecast. And we also look at the concentration level of the market. So there's always different ways of how markets can be concentrated together. So for instance here, we find the top 20 vendors accounting for 55% of the total revenue in 2023, with the next 50 contributing 29% and then a long tail, 17%.

And so that's a pretty competitive market at the moment. It's not too concentrated in terms of how the vendors are clustered together, but that's likely to change. I mean when I was talking earlier about the 117 tech generators, essentially, I guess, to be blunt, there's a lot of companies there doing something quite similar to one another. And we know what happens when that occurs, there will be some consolidation in the market. And as we go back to this forecast which we will do in the future, we expect to see some of these various sectors change in terms of the structure and the size.

Eric Hanselman

And that's the interesting thing about doing the bottom-up analysis, is that you are actually identifying who those market players are. So as that consolidation happens as we start to see that shift in the market, especially in the market that is moving as enthusiastically, shall we say, as this one is, something that's certainly complex to get one's hands around. What are your major takeaways of the key aspects of what the analysis really shows?

Nicholas Patience

Well, one of the questions we get asked a lot is where do you think value will accrue in this market? And as we look out essentially to the end point of the market into 2028, at that point, if what we are looking at turns out to be the case, then the foundation model segment would be the largest, with the text generators coming next and then the various clusters around images and videos and cogeneration. But that only tells you one part of the overall story.

Within these various sectors, we think there are some interesting things to look at. So we've mentioned obviously the text generators. There's a lot of vendors in that space and we expect some shakeout. If we look at the image generation stuff, which is a lot of the fun stuff to play with, I think, is one way of putting that stuff. There's a lot of interesting images you can come up with...

Eric Hanselman

Astronauts on horses, right? It's all of those good things.

Nicholas Patience

Exactly. It does always involve -- it usually does involve astronauts or something in space at some point that shouldn't probably be in space. That tends to be the kind of driving theme for this. But we see, yes, there's a lot of -- there are some well-known start-ups in this space, but then there are obviously large companies like Adobe and some others here coming in -- coming to the fore.

And a lot of the use cases now, not surprisingly, are very much of -- for the -- of the creative professional. So not only artists but marketing professionals, design, fashion, areas like that. And there's a lot of pickup there quite early, we think.

There's obviously some concern around intellectual property rights and there's concern around viral fake images. Those 2 things are big issues, but we think that some of these things, they'll be overcome. Whether they'll be overcome through self-regulation through the technology companies themselves or there'll be a legislation and regulation of some kind, I think it'd be a bit of both, and I think there will be regulation around that. One of the key challenges actually for this market as a whole is intellectual property rights and copyright law and how those things may have to evolve over time, but maybe that's for another podcast.

But one of the other ones I wanted to look at was the cogeneration market. So in theory, obviously anybody who can write a language, whether that language is English or French or Chinese, can now generate code. Now that's obviously, that's true, but that's not particularly the point. The point is there already have been cogenerators around before generative AI, but how can generative AI code generators augment the developers we have and the developers we want to have?

It's quite a small market at the moment. We think it's very fast growing. And there's a lot of input coming from the large cloud providers, notably Microsoft and Google but others as well. And we think this one has got real legs, partly because there's already well-established software testing methodologies out there. So if I wrote a piece of code for my company, they wouldn't just be chucked straight into production. And nor was yours, Eric, I'm sure, even though it would probably better, but it won't be put into production. It will go through all these testing methodologies...

Eric Hanselman

You overestimate my coding skills, but...

Nicholas Patience

But it will go through testing methodologies. Now exactly the same thing will happen with code that's generated via generative AI. And so I think kind of some of the -- again, some of the fear mongering that goes around, our code, proprietary code is going to get out there, it's going to get back into ChatGPT and all these kind of things. I don't really think that's so much the point. I'm sure there will be some more scare stories down the line. But I think it's -- because there's already that structure in place, I think this is a really interesting area. And it's very much augmenting what people already do and may enable them to do it faster.

And it's also resurrecting some -- not dead languages but languages for which, frankly, there are not a lot of young developers who want or are interested in learning, like COBOL being the classic example. We talked about all these many, many years about legacy COBOL developers and how you're going to deal with those people who are retiring. Well, I'd say, I mean, a lot of them are past retiring by quite a long way. And so it is kind of how do you actually deal with that code that exists in mainframes, especially in a lot of industries such as financial services. And this generative AI cogeneration area is a really good use case for that, I think.

Eric Hanselman

Well, I mean, we're dealing with language models. And not surprisingly, software languages actually fit pretty well. Now to your point, there are some cautionary tales. The Samsung testing script situation in which test scripts got dumped into a particular large language model and were able to be extracted is one of those. But I think what you're identifying is that key part of it, which is that we're still developing general guardrails for how you use these capabilities, how you apply them.

And with cogeneration, we've got that opportunity to be able to use some force multiplication for a resource that's already pretty constrained, which are coders. And being able to leverage and facilitate what they're able to do to be able to do a lot of the -- do the fundamental substructure work, things to be able to build the program scaffolding that's necessary to build larger frameworks and allow coders to then work on those things that are more complex and harder to replicate.

Because again, that model has to be built from something. In order to generate it, it has to know enough about what ought to be there in order to build something real and in a world in which so much of the code that's actually written is already being pulled from open-source repositories, things that we can use to be able to train large language models. Here's something where it actually can save developers that cost of going out, finding the code they need and building at least the base framework to be able to get applications up and running more quickly.

Nicholas Patience

Yes. Exactly. And we've been talking to a lot of vendors in the space of development tools in the broader sense of the word. And they see a lot of opportunity there, understandably. And they're obviously the best place to kind of implement that. But the open source point is actually an interesting one. And whether we're talking about code or talking about the -- just the effect of open source on this market, it's pretty huge. I mean there's the Hugging Face repository of open-source models. I don't have the numbers in front of me, but they -- I know they've been adding thousands of various language models or foundation models of various kinds over the last few months.

And again these are things that have to go through testing, but there's a lot of potential out there for organizations that do more than dabble and actually do some experimentations into whether they can effectively build their own model, whether they can just use fine-tuning which is taking an existing foundation model and fine-tuning it with some of their own data to make it appropriate for them. I think there's a lot of interest among corporations of mainly larger ones, let's be honest, than small ones in doing exactly that.

Because, again, like the previous wave of AI and ML explosion of interest, say -- I'd say 2014, '15, '16 time frame, there was a lot of that, I need to get -- make me some of that AI stuff, go get some of that AI stuff. And then once they figure out through the experiments, then they think, well, this is a bit too broad, then they narrow down to very specific use cases.

And that's what we have been rolled out over the last few years. So when we do our other surveys, our Voice of the Enterprise AI and ML surveys we do every year, a lot of that -- we do one on use cases, as the listeners probably know, and one on infrastructure. And on the use cases, they're quite specific in investment banking. It might be there's fraud detection, et cetera, et cetera, but there's -- and in health care, it's clinical workflow optimization.

These are very, very specific things which are not necessarily useful in other industries. Why? Because they're based on the data that those organizations have. And so the same thing goes here. The difference slightly is the existence of those foundation models for language in general. And that opens up a new opportunity.

And one of the major opportunities is the way that we interact with applications as humans. And the reason it opens up opportunities is because it can generate text, as opposed to just "understand it," which obviously it doesn't really understand it. This is probabilistic stuff. But it's -- it looks like it does. I mean but if you can also generate language out of that, you can have much more interactive chatbots that actually -- might actually work this time around.

And so there's a lot of other areas such as search and knowledge management areas that have been tried for years and have not really worked. And the fundamental thing that's happening is you're unlocking the value of all that unstructured data that every large organization and many medium-sized and smaller ones have but have never really been able to harness and generate value from.

It's always been something to manage, to archive, to maybe delete, to store but not to actually do anything with particularly constructive. And it's been the promise, I'd be honest, of machine learning for years to do that. And it's had some success, but it's quite limited. This, I think, has the potential to really unlock that value.

Eric Hanselman

Well -- and there's that question that you're raising, which is for a lot of these applications, maybe an existing foundational model will be enough to start with. And to get them going and the role that the Hugging Face class of organizations have in being able to provide those now gives a new aspect to the market that, hey, there is now the potential for trade in the foundational models, in the tuning of the foundational models with your own data to be able to actually leverage all of the data that you have to do something more meaningful.

Nicholas Patience

Yes, exactly. And it comes back to that point of whether you as an organization need a foundational model and have -- do you have the -- the alternatives will be essentially -- there's kind of 3 ways you can go about this. You can do prompt engineering, so figuring out what kind of prompts get the most effective predictions out of models. And then there's plenty of ways of doing that and there's a lot of -- it doesn't require a huge amount of skills.

And then you have the fine tuning of the model, which does require skills, but there are open-source alternatives. But eventually, you need -- there are skills that need to be -- have to be required. And then if you really think a foundational model is going to give you a unique competitive advantage and you have the resources to do it, then go build a custom model.

S&P Global has made it clear in public that S&P is building its own foundation model. We can do that and we're doing our own proprietary financial data. But not every organization can do that, so there's alternative routes to taking advantage of generative AI.

But I'd come back to all the kind of things that -- the stuff that it's really good at like content creation, cogeneration, summarization, semantic search. That's not to take away from traditional AI which is very good at things like classification and computer vision and process automation. And so we need to get to that point where, well, we will do eventually, where people figure out that generative AI can augment all sorts of use cases in traditional AI, but traditional AI also has its own uses.

The one thing I would add, though, is when we can get to the kind of chaining effect of chaining models together and handing over more sophisticated APIs with prompts themselves, you can then start to see automation happening, processes being driven by a single generative AI prompt which then generates something else, which generates something else, which generates something else, which executes code somewhere and has some sort of outcome.

Now we're only in the very, very early, I would say weeks of that. This is not a very fast-moving space, but that's where you start to feel some real value there in automating processes because that's essentially what software does, isn't it? And I think that's we'll get there eventually, but it's very, very early days.

Eric Hanselman

Slight iterative process. Yes. I've seen that interesting Kensho demo in which there's code generation based on a prompt that then pulls more data that then gets wrapped into an analysis of that data that gets delivered, yes. It's getting into that next stage of leveraging the foundational pieces and then moving beyond that.

But to your point, even classical, I guess I'll have to get used to identifying that, is what we had been doing, AI kind of applications are all still very relevant in that, if you're looking at a generative model with billions of parameters, the training costs alone are going to be very significant. And if you can target a specific use case that doesn't require billions of parameters, can look at the classical ways of doing that analysis, it's all a matter of what you're trying to achieve, what your budget is to make it happen and how you actually can approach solving what are a -- now a much more interesting set of problems.

Nicholas Patience

Yes. I think that brings us on nicely to a brief discussion about infrastructure. As I say, our model is a software market forecast. Our generative AI market monitor is focused on the software aspect. Now we know, as I'm sure the listeners do, that this does require -- as you just said, if you want to build your own foundation model from scratch, it requires a lot of compute resources, storage resources, networking and so on and so forth. And whether you rely on a hyperscale cloud vendor or you want to do this on premise or you want to do this in a hybrid environment, and if you want to do it at the edge, which could be extremely unlikely at the moment, but these are involved resources.

So what we've noticed in amongst some of our colleagues who are focused on infrastructure, perhaps more than I am, everyone is affected here, every one of our -- of the infrastructure vendors. And most of them are somewhat overwhelmed by the amount of interest they've had. And this spans NVIDIA and AMD and the other chip companies that can offer these kind of accelerators, are one area.

I've been talking to quite a few data center providers. I'm not a data center expert at all, but they've had companies coming at them from vertical market segments as they see it, that they've never dealt with at all, saying, we need to do this, and we're really interested in doing this. How many GPUs can we get ahold of? What are the systems and how quickly you can get them? And things like that. And whether you think those people are running too fast or just going after some sort of Holy Grail, I guess, is to be determined. But the interest is real.

And we've even seen areas such as vector databases. So vector databases have been around for a long time. But vector databases and the way they store data is exactly how large language models want that data to be stored, and that's how they want to interact with it. And so my colleague, Jim Curtis on my team, he's been talking to all the vector database companies, some of which are 25 years old.

And they've again had interest from vertical markets they've never even offered to try and sell into or tried to years ago and had no interest, and now everybody wants to know who they are. So it's having this spillover effect into all of these other areas, not just NVIDIA, all these other companies having real huge up levels of interest in what they do. And that is basically down to generative AI.

Eric Hanselman

Everything old is new again. Bell-bottoms are back. Man, we're just -- the pendulum swings, right? But hey, this is -- we know these cycles happen in tech. It's always interesting to see how these have all shaken out.

Nicholas Patience

Yes, and so we're going to have a lot of research coming out. We've created a little tiger team inside the analyst team across channels, across areas. And we're planning various bits of research. We've got our Voice of the Enterprise AI and ML infrastructure survey coming out soon. We're doing another one on -- for our DevOps channel on how generative AI is affecting code generation in more detail than we've obviously been talking about here.

We've got another that look at -- we're looking at vector databases with all of these kind of things I mentioned. And then data centers and how things like the liquid cooling market has -- heated up is the wrong expression, but you know what I mean. Probably it's -- that area is...

Eric Hanselman

Well, it hasn't cooled down, but...

Nicholas Patience

No, it hasn't cooled down, no. There is always a need for it. But that's, again, directly related to this. Because, if you've -- if you took a -- if you had a data center and it was -- this is very simplistic, obviously. If it's full of CPUs and you replace it with one that's full of GPUs, that costs a hell of a lot more. And it eats up a lot more energy and creates a lot more heat.

And so there's all these other knock-on effects that this is having. And it's all -- well, this is all down to the fact that, November 30, 2022, OpenAI decided to slap a free interface on top of its model. But -- it's a lot to do with that. And it was going to happen at some point, but just they did that at that point.

I was talking to -- before that happened, last year, I was talking to a lot of the vendors that have these large language models and they've been building them for years. And they were saying, well, we've got a really good one, and yes, ours is better than that one or it was more appropriate for this use case. But you had to take their word for it. There was no way of knowing.

And now you can play with these things to a certain extent and figure out which ones are best for the use cases for which you're trying to solve. So I think it's had to happen at some point, and it's happened. And although I don't think we can have a 6-month period quite as crazy as the last sort of 6.5 months have been, the cat is out of the bag. And I think we're going to see such knock-on effects throughout from the top-of-the-stack business applications right down to the chips and everything in-between.

Eric Hanselman

And hopefully, things will be at least not quite so crazy, but hey, we'll have to see where it goes.

Nicholas Patience

Yes.

Eric Hanselman

Well, especially, it seems like we've got a lot more to talk about. So Nick, I want to make sure that we take a marker to get you and Jim and other members of the team back on to talk about everything, from the infrastructure to support this and really where we go next, these next stages.

But this has been great. We are at time, so we're going to have to call it at this point, but thank you. I appreciate all the information. And we'll see where all this stuff shakes out.

Nicholas Patience

Yes. Thanks, Eric. Thanks for having me. It's been really interesting. So I look forward to coming back soon to explore more areas.

Eric Hanselman

Well, and so much more to look at. But that is it for this episode of Next in Tech. Thanks to our audience for staying with us. And thanks to our production team, including Caroline Wright and Ethan Zimman on the marketing and events teams; and our agency partner, the One Nine Nine. I hope you'll join us for our next episode, where we'll be digging into a whole range of different technology perspectives. I hope you'll join us then, as there is always something next in tech.