Next in Tech | Ep. 254: AI data

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry Research at S&P Global. And today, we're going to be talking about databases and a lot of what's happening and is the association with AI with James Curtis. Jim, welcome back to the podcast. 
 

James Curtis: Thanks, Eric. Thanks for having me, it's great to be back. 
 

Eric Hanselman: It's great to have you on. There are a whole set of different things. I guess, maybe start off with some of the things that you were seeing in at AWS re: Invent and some of the database perspectives there and some of how that's spinning out into the broader AI world as we start looking at the year ahead. 
 

James Curtis: Yes, that's great, absolutely. What a great event it was. It was in Las Vegas. It is the preeminent event in our tech space. I think it attracted like 60,000 attendees, that kind of thing. This year, AWS made a few actually very interesting announcements that I think are sort of worth digging deeper into. One was that they rolled out this thing called database savings plans for AWS databases. And it's not something we see a lot just in terms of other cloud platforms rolling out. But it was interesting in the sense that it covers discounted savings plans you could have for like 9 different databases, Aurora, DynamoDB, KeSpaces, there's a handful of these. And what's interesting about this is that it was geared towards sort of a mid-tier market, right? And that's because a lot of the enterprise clients of AWS, the large ones, probably have one-off deals or maybe very privately structured contracts, things like that. So, this is really for this middle-tier businesses, and it's really sort of to reduce some of the cloud spend that they might have but also has a sticky factor to it, Eric, in the sense that they want you to use their database services. They want you to stick around and run your businesses and operations and things on it. So, it's just kind of an interesting element to that. And as you know, Eric, from previously, cloud and running cloud workloads is kind of a shifting landscape right now. 
 

Eric Hanselman: It is indeed. And it's also one where you take a look at the whole FinOps movement has really gotten a significant amount of groundswell because there has been concern about the cost of cloud, how do you manage -- yes, you have indefinite scalability, but most organizations don't have infinite budget. And how are they actually going to manage that? It's interesting to see because there have already been overall compute utilization packages that are starting to optimize how you spend compute, you look at committed spend discounts, all these kinds of approaches to be able to help organizations manage, that's one of those things that when you then get to what is the core of really the delivered value, the data that -- or the data systems that are sitting behind all this, that sounds like that's maybe the next step certainly that AWS is heading towards. Is that something where you're seeing this as more optimization? Or is this sort of bundling discount? What approach are they taking? 
 

James Curtis: There's the discount element here, right? They're offering discounts on some of these things, but there's a strategic element here. And the strategy behind this is that AWS and others recognize that we're not living in sort of a binary world. Are you in the cloud or not in the cloud, right? It's very, very nuanced here. And the idea here is that embedded within these discounts applies to things like serverless, right? You're rewarded if you take on these discounts for these databases as a serverless option, right, and other things like that. And it's meant to be pretty short, it's like 1-year commitments and the ranges go from 12%, roughly right around 20% is probably the average, but up to 35%, depending on what you have. But yes, I think that's true. I think it's just the general recognition that we are in a nuanced cloud computing world. There's everything from cost to flexibility to what makes sense to run where. It's more complicated now. 
 

Eric Hanselman: Well, it was interesting, as we've talked about, this was probably the first reinvent at which the word multi-cloud actually seemed to be embraced. And it seems like that's actually creating some thoughts about ensuring that there is stickiness that it does get seen as ways to be able to manage expenses, be able to manage this. It seems like this is the -- maybe to your point, the next stage of really that cloud maturity from both the cloud provider side as well as from the cloud consumer side. 
 

James Curtis: Yes. And certainly, it's not, hey, engage with us in a very complicated contract-driven way that may not favor every tier of the market, right? And in this case, I think there's recognition there says, hey, this is very favorable actually to these mid companies, right? And it's taking services that AWS and sort of nuancing those to the market because obviously, we want everyone to be on our platform, right? And so, some of this is just better recognition to say we have a nuanced audience. They have certain needs, they have specific needs, and pricing tends to be a big trigger point for people. Let's -- there's a way to sort of let them consume services in a manner that they want to consume them, or it makes sense for them and their industry. 
 

Eric Hanselman: Well, it's something that is getting much more competitive and also in a world in which the switching costs in the past, you were pretty locked into whatever database capability you're putting together. And we're getting into a world in which it's not insignificant, but the switching costs maybe are a little lower and maybe there are options. I guess something that they've got to keep an eye on market forces there. 
 

James Curtis: Yes. No, it's good stuff. I mean some of the other announcements they made, I think, might be interesting as well is that they have a -- I might give some context here because a lot of what AWS does is they have a very big database portfolio. Like it covers everything from relational databases to analytics to a lot of the SQL vendors, depending on what application you have running or sort of your workload. Back in 2021, AWS used Elasticsearch, and they forked it, and they called it OpenSearch. It has since been donated to the Linux Foundation, but they still have a service around effectively OpenSearch right now. Getting back to the other point of catering to sort of nuances in the market and particularly you mentioned AI at the beginning, this is a really interesting update that they did is that they did a couple of things that are worth noting. One is they're trying to solve 2 problems. One problem is that from an AI standpoint, particularly if you want to get sentiment or semantics, you're going to have to vectorize data, okay, and then store that some way, right? And it allows you to sort of hit that. And that can take a lot of time. It's particularly if you want to index it. And so, AWS recognized that. The other question -- sorry, the other one is that sometimes, particularly when you're working with some of these databases is that there's a lot of manual labor involved. And do you want a quick result? Or do you want an accurate result? 
 

Eric Hanselman: It's the classic, you got 3 choices. You get any cheaper, faster, smaller... 
 

James Curtis: Pick any 2, right? And really all -- and so some of this is recognition. And so, they rolled out some pretty interesting updates around open search and sort of build out the capability in the portfolio. I actually think you are kind of worth mentioning you might find interesting. One is that let's say you've got 2 vector embeddings and then you load those or ingest those into open search, which then you can perform searches on and semantic searches and even combine that into a hybrid search. But you now have the ability to sort of run that index or build that index on GPUs, and it will spin that up very quickly. And then the second element, yes, I mean, it's -- and there's some really interesting elements to that. So, it sounds like you had -- you wanted to. 
 

Eric Hanselman: Yes, I was just wondering, is -- so that's something where you can select acceleration? Or is that something where you're actually targeting where Open Search is actually running? How does that actually work? 
 

James Curtis: My understanding of the way it works is that, like I said, it -- I mean, to take data and build and vectorize it takes some time. But then you also have to index that like anything else, right? Because you want to search those vectors, right? And you want to do that very, very quickly. But that can actually take anywhere from hours today to run that index. And so, they're giving you the option to say, do you want to run these on GPUs? I guess you don't have to. And there's probably a pay-per-use fee associated with it, right? GPUs aren't free. At least not yet. We'll see whether or not we get there. But this says, hey, would you rather run this in days or would you rather run it in a few minutes. But it's an interesting trend that we see other places. I think, Eric, maybe you and I have had conversations previously in the fact that if you add a lot of compute to something and there's a slight cost to that compute, but it will be done extremely quick and faster, right, that could potentially -- that marginal cost on that compute -- and then the reduction in time is actually better than just running on maybe your standard over a longer period of time. And this is just a general trend we're seeing. I don't know if you're seeing something similar. So... 
 

Eric Hanselman: It all goes back to time is money, and there are some things that you need to get done in a timelier fashion, and that's got value. 
 

James Curtis: So that's the option. The other one is that they're back to what I said before, pick your options is they're including as part of the open search, they're including what they call auto optimizations, which really addresses sort of search latency and accuracy, right? And so, the way this works is that the users or where they were doing that. It's in their requirements. But we are around, hey, I want this fast, or I want higher quality and maybe they can allocate some memory requirements to sort of help in that. And maybe it's sort of a dial or knobs or things you do that. And you have that option, right? Because they -- again, the previous thing we said is that we don't want to just spread peanut butter by an entire audience with a singular feature. We want to be sort of nuanced in giving people what they want. 
 

Eric Hanselman: I mean, realistically, you're in an environment where this is a technology that's going to be used in so many different ways, in so many different applications that tunability winds up really being critical because again, you're not in a one-size-fits all, you've got resource constraints, you got time constraints, you've got cost constraints. And especially when you look at a technology that is as broad brush as open search, the ability to tune it is really important. 
 

James Curtis: Yes. And search is just in general; the search technology is having a bit of a moment here. You and I both have been around this space a long time. And enterprise search has always been a bit of a stayed market, I guess, if you will. It sort of had its place -- it was embedded in various applications to find things, but it hadn't really progressed all that much over the past decade, I guess, if you will. Well, long come vectors and search is having a bit of a moment right now because they're like, wow, if we combine what was traditionally enterprise search now with semantic search, we got something here. We have -- we just have incredibly rich search. 
 

Eric Hanselman: It seems like semantic capabilities really get beyond what had been that traditional keyword and keyword associations are fine. But as humans, what we're really looking for is that more detailed, that more associated capability that semantics actually delivers for you. 
 

James Curtis: Yes. I mean we all -- if you're a language guy or you like to sort of have fun with language, I mean, the English language is great for words that have multiple meanings, the same spelling. And it doesn't take very long to do a search and realize that it's not coming back with what you wanted to. And so that semantics are just golden here. I mean it's just -- it's -- I mean, search is now an incredibly popular service for AWS. The Linux Foundation has a lot of traction. So yes, interesting AI's impact on existing technologies, if you will, and how it's working as an enabler. 
 

Eric Hanselman: Good things. Well, especially now as we start to get towards what are a lot of the sort of the -- I don't know, not necessarily pixie dust kinds of capabilities, a lot of things that are transforming what had been widespread tasks like search that now are getting reenergized because they've got capabilities where you don't have to do what you typically did with search, which was having to optimize, having to build relatively complex search tables. Now you've got something that gives you substantially greater sophistication much more simply. 
 

James Curtis: Yes. There's a handful of other technologies that have sort of gotten new life in them, storage and all these other ones, right? Well, on that point, so last year at re: Invent, I believe was it last year -- I'll go back a little bit my notes, but they released this thing called S3 vectors. It may have been earlier, but it was generally available at about the same time. As we talk about technologies that are seeing the golden touch of AI, object storage is one of them, right? And Amazon is pushing immensely forward with their S3 storage, and now it can house vectors, which effectively means it could be a type of vector store, then you could query, and object storage is cost effective. It's massive scale, and it permeates in all sorts of different ways. 
 

Eric Hanselman: But this is something that you've been talking about in terms of what's been happening with vector capabilities? And is this something where you're really retrofitting vectors to an existing database or storage environment or going for lack of a better term, native vectorization. How do you see those trade-offs? And are there advantages to one side or the other? It seems like from an S3 perspective, now you've got the same storage environment that you've been working with except now with vector capabilities, you get to use the same kind of access mechanisms. You get to use the same sort of back-end capabilities and now you get vector functionality. Is that -- how does that stand up against some of the other vector options that are out there? 
 

James Curtis: Well, I think -- so there's -- it's funny. I mean I have to go back and look. I think when vectors first came on to saying, hey, we're going to take unstructured data. We're going to vectorize it, which is basically like a big number -- big string of numbers that go in a certain order and you can store those within the database. And I want to say it was it, I don't know, maybe 1.5 years ago, I don't know when some of this stuff was emerging, I may be off a little bit. But it's relatively soon, but since I covered the database market for the most part, give or take, 300, 325 vendors, 99% of those now have vector support. They -- and in addition to, that, I think the topic of a previous podcast we had was that there was also vector-specific database vendors, too. And now we're seeing object storage, right? So, I think the answer is yes. It's permeating throughout space. And it's interesting to kind of see what's happening there. 
 

Eric Hanselman: Well, and actually, just to take a step back, for those of our listeners who aren't familiar with vectorization and the important part, this is a step forward from the ways in which we've historically organized data, which was you had a bunch of tables, columns and rows, and they could refer to each other and that was the way to link things together. Vectors actually are a different way to be able to make those associations and substantially more efficient. 
 

James Curtis: Yes. No, that's great. So, thanks for clarifying that. But yes, we had not had a great way to deal with unstructured data, right? You could stick it in databases, but you had to wrap metadata around it to sort of search on it. And now you can basically take stuff sitting around the enterprise that's unstructured and vectorize the stuff and get the semantic meaning out of it and search for that. I mean it's just a huge game here. And it's like, hey, all of a sudden, the data we've been sitting on, we can now use that, right? And so, storage vendors are super excited about this stuff. It just opened up massive doors for us, right, in terms of how we can leverage it. 
 

Eric Hanselman: You've now got a much more complex way to build associations. And if I am here, what is similar to this, a lot of things that people are familiar with on recommender engines and things of that nature, but, of course, extend across all sorts of different data uses in terms of where they fit. And of course, critical for AI. And when we start thinking about other things that AWS was talking about, they're doing a set of things or there were a set of announcements associated with some of their AI foundational elements as well. 
 

James Curtis: It seems like every event I go to; you travel a fair amount too. You don't go to an event right now and AI isn't sort of the primary topic that sort of is interleaved with the existing technology that they had, right? It's kind of interesting because last year, they announced a pretty big rebrand of this product called SageMaker. I think you're familiar with SageMaker, right? It was their analytics platform that from a high-level standpoint, you could sort of create, manage, and deploy machine learning models. They had some feedback on that, that basically says, "Hey, customers really liked that, but they also noticed that in addition to doing that, they were stitching together a whole bunch of other services around that into sort of reference architecture. And a lot of that had to do with, to your point, is like I've got AI coming into my enterprise now and some of those were AI elements, right? Because they like the machine learning part, they like the analytics part, right? And so last year, they said, "Hey, now SageMaker is a platform versus -- I mean, you can still consume it, I think, as a single service or a series of services, but it's now a platform, right, that basically is both AI and Gen AI has other AI elements in that, right? And that's sort of a case study with let's take a bunch of services we have, let's wrap a platform around it. because it now serves AI integration, rollouts, all sorts of things. 
 

Eric Hanselman: It seems like it's got the benefit that we so often see in platform approaches, which is that you get a prescriptive way of building a set of capabilities. You get a whole set of tools that are bound together, especially now in the relatively early days of AI, it can be particularly useful to just simply have a set of guardrails and say, here's the tool set, here's how you leverage it. Here's functionally how you integrate all these capabilities, wrap them together in ways that make that process simpler and do it in a way that offers a little more guidance, a little better integration. 
 

James Curtis: Yes. No. And the result is that there were some manual efforts there, right? I mean there's no organization that has one single vendor that has one single product that does them all. Most environments are probably highly stitched and integrated together. So, we ran a survey question on some of the research I was doing to basically said, "Hey, do you generally prefer an environment that has sort of best-of-breed services or products or offerings? Or do you prefer sort of an integrated approach, your environment has maybe your top 3 or 4 workloads or things that you want, did you prefer that? So, you might guess what people favored, but... 
 

Eric Hanselman: This is actually -- it's one of those things because we see the same thing in security, which is do you like best-of-breed? Do you want a prescriptive approach? And the pendulum swings back and forth. I will make the guess that the platform approach was where people were headed. 
 

James Curtis: Yes. But what's interesting is that it wasn't a significant preferred, but it wasn't dominant, I guess, if you will, right? Like meaning like it wasn't like a 90-10 or something like that. And that's because I think there's a couple of reasons for that. One is that -- over one is, yes, generally, I think the points you made before, codify things and security works together in there, data movement works together, and I have maybe a front-end interface to it, and I can onboard people, right? All these things stay true until they don't. And when you want -- you may have maybe specific reasons or enter AI, and it gets a little more complicated, right? You need this thing over here that performs this very specific function. 
 

Eric Hanselman: Well, it's interesting to see where all of this has gone. I'm sure there will be a lot more to roll out as things get going forward. I will point out our listeners to a recent research report and get those in the show notes and get some of these study results out in people's hands. This has been great, Jim. I appreciate all the insight and the feedback. 
 

James Curtis: Thanks for having me, and I hope to be on again soon. 
 

Eric Hanselman: Very definitely. But we are on time for this episode of tech, but thanks to our audience for staying with us. And thanks to our production team, including Sophie Carr, Feranmi Adeoshun, and Kyra Smith on the marketing and events teams. I hope you'll join us for our next episode where we're going to be kicking off our CEO series. I hope you'll join us then because there is always something Next in Tech!