Next in Tech | Ep. 260: Putting Agents To Work

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, an S&P Global Podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry Research at S&P Global. And today, we're going to be looking at some of the next stage thinking around to join me to talk about that are returning guests, Emily Jasper and Sheryl Kingstone. Welcome back to the podcast to you both. 
 

Sheryl Kingstone: Thank you. Welcome back. And this is my first time with Emily in a long time. 
 

Emily Jasper: I know we've been doing research over the years, but this is our first time joining you together, Eric. 
 

Eric Hanselman: And talking about the research, this is something that we point to in the show notes, but this is something where you've actually both been working on a project. And I guess before we get too far down the road, -- can you give us a little bit of background in terms of what are you focusing on? And what are you trying to make sense of in all of the wonderful exuberance that exists around agents today. 
 

Sheryl Kingstone: Emily, why don't you start because this is your baby. 
 

Emily Jasper: Yes. I came to Sheryl since Sheryl and I have worked a lot on many projects over the years, and she is all about those workflows and the technology that's used to enable them. And I, myself would be considered a little bit of an AI skeptic only because I -- there is no silver bullet for everything. And as much as we wish there would be, I had a thinking suspicion that AI was not going to solve everything right away. However, I wanted to find a way to articulate that there are real actionable ways that agents can make a difference, especially in sales workflows. And so I was like, Sheryl, can we dig into this a little bit more? I know you're already thinking about the future of this, and I'm still working with folks who are trying to get their bankers off of like business cards of posted notes. So where do we meet in the middle? 
 

Sheryl Kingstone: Yes. And that's really a good point, Emily. I'm just going to jump in 2 seconds because while Emily said that you came to me for the orchestration, the CRM has been around for 3 decades. It's my -- I implemented a sales system in 1987 when the road map was to go from DOS to Windows, okay? And we still have the same problem of -- it's like a data entry system from hell. No one likes to use it, and we're always promising the next best new tech to make it better. So it was better task-based UIs, then it was better data, then it was better AI, then it was -- and now to be honest with you, I hadn't written about sales in so long because nothing was moving the needle. Now with the agents and the way we can actually pull it all together, and this is when you came back to me, the fact that we're able to finally maybe make CRM valuable, finally helping the sales reps sell, which I've been talking about for 3 years 3 years, 3 decades. And finally leveraging the data that's all over the place, with the whole change of the LLMs and the Agentic workforces. So that's why we're here, Eric, and you know this as much as we do. 
 

Eric Hanselman: So agents are the pixie dust. Emily was all down on the Pony angle to this. Here she was bursting all my balloons on this. All right. So what does this really boil down to? 
 

Sheryl Kingstone: Hopefully, revenue execution, hopefully paying attention to our data, and Emily is going to go into more about that and hopefully getting people to really value these systems. 
 

Emily Jasper: Yes. If you think about the expense of a CRM and Charles, like it's in most instances, the most painful part of someone's job. But firms spend a lot of money investing in it. It's a huge enterprise system. And there's finally a way to make it a little bit easier. It's not necessarily going to replace a CRM. It should make your sales team go from that 80-20 rule, you get more of those folks who are at that mediocre-ish 80% moving into that high-performing 20%. And I think then the investments that go into the platform, into the data start feeling more worth it because you could tie it to real revenue generation. 
 

Sheryl Kingstone: And we can more easily bring in third-party data. So we will go into that. We will go into the value, and I'm trying not to make this an infomercial, but I really mean that because think about the protocols that are coming out with MCP and we're going to talk about that a little bit later. But the fact that not all the data in CRM today is what we need to help salespeople sell. So that's where the value in the ecosystem comes into play. 
 

Eric Hanselman: So it seems like -- well, one identify one of the many elephants in the room, which is some of those projections that SaaS applications are going to be fully consumed by agents. All you need is for the CEO of a major company to sort of hang that out there. And of course, everybody suddenly is like, yes, jeez why do we need to invest in this stuff anymore? It sounds like, though, Joe, you're calling out what is one of the principal challenges, which is that these have been large SaaS systems or either SaaS or on-prem, CRM systems have been the repository for so much of that information and certainly so much of that data. The question is it sufficiently valuable information. What's that progression look like? And I think most organizations wrestle with ensuring that data quality is really where they need it. But even simple things like duplicate customer entry kinds of things, all those -- have we associated -- are we really associating all of the events with the right people, with the right contact? Are we spelling the contact names right? Are we -- all those kinds of things that the data quality piece has certainly been part of the limitation. And so what do we have to do in terms of really moving the needle on the data quality side? Is that what you're heading towards in terms of... 
 

Sheryl Kingstone: Actually, I want to back up a little bit before the data quality and go back to the eye-roll that you missed since we're not on video. So the eye-roll was the. 
 

Eric Hanselman: We need the audio eye-roll there, but... 
 

Sheryl Kingstone: We need the audio eye-roll because like about a few minutes ago, I did an role on the death assess. So let me just state that's not happening because an agent can't act without a control plane, without data that's driving it, without the ability to train it. And where is that going to go? So before we even get into data quality, where is the data sitting? Once we know what it's doing, right, yes, CRM has had a data quality problem, and no one will dispute that. It's one of the major things that Emily was working on a long time ago, too, was how are we going to automate and improve data quality? How are we going to append? How are we going to do all of that? And I want to shift the discussion over to Emily. How are we handling this data quality issue, even living in CRM, too? 
 

Emily Jasper: Yes. I would agree. The death of stats, I was at a panel last week, and it was the biggest topic. And literally, everyone from not only the SaaS providers, but from some of the AI providers all rolled their eyes as well because I think what -- if you're going to make a bold statement like that, obviously, it's an opportunity to talk. But what CRM still provides is begrudgingly as someone might agree is it's the framework of the data designations. So a visit versus an e-mail versus an opportunity versus a transaction versus a corporate hierarchy, et cetera, et cetera, et cetera. 
 

Sheryl Kingstone: And data model. 
 

Emily Jasper: Exactly. Let's be honest, AI world that we're living in, it's the time for the library scientists to come out and celebrate how amazing they are because this is exactly where everyone should be thriving. And so from a quality perspective, that data model is critical because if you're not going to be able to classify all those things correctly, the agent is only going to start squishing it all together without that. Now the value is that the agent then puts the appropriate things into the data model where it lives. And so maybe some of that UI changes over time. But that's actually going to be where we have the intersection. It's going to be less that one is replaced by the other. It's that we're now going to have it enabled better by these agents. And the models where these data items are going to sit are going to be more accurate because you're exactly right. It can read the name from the e-mail so that it's spelled correctly. It can associate to the corporate hierarchy because it has a reference point for where that business sits in a large conglomerate. That's going to be the value component, but it's only going to be as good as that referenceable data. So if you're still only reading from e-mails, and I'm a good example, Sheryl, you are probably a good example, too. We might just put S&P Global as part of our signature, but each of us works in different divisions. If someone didn't have the divisional structure of S&P Global as a hierarchy, they might still just keep tagging us to the corporate level, even though we're in wildly different parts of the organization. And if you were a commercial person, that could not -- that wouldn't help because you wouldn't be working with the right people who might be able to sign your contract, for example. 
 

Sheryl Kingstone: And we want the salespeople doing that in the system. 
 

Emily Jasper: Yes. They don't want to chase that down. But you're exactly right. And -- but that's the old way, right? We used to make the sales team's problems was to sort out where did this person sit in the organ, where is that in a division or a subsidiary or are they in a minority investment organization. Nowadays, these agents can reference all of that metadata and point people in the right direction. We're saving on that time. 
 

Eric Hanselman: So Emily, a lot of this is actually getting back to something you talked to a year ago, actually, probably maybe more than that now. We were talking about how do you leverage trusted data to be able to do this. that next stage, I guess we're to a point at which agents start to change this and that the data management, data quality pieces are agents the way to work through this? What's -- how does that progression actually happen? 
 

Emily Jasper: It's funny when we had those conversations, we were digging into bias a lot at the moment because we were trying to make sure that we're identifying that general models are not trained on identifying maybe what should be weighted more from like an information accuracy standpoint and thus, you might have bias in your model. Where CRM agents seem to have an advantage is that many of these platforms either have their own native models that are focus on those workflows, right? It's a CRM workflow model that knows if you see A, your response B, C or D are going to be related to sales tasks really at the core of it as opposed to maybe just a super general model that would be like, I don't know what you're going to do with this information, you could throw a party, you can make an Instagram post. -- whereas a normal salesperson, I'm not on Instagram doing that. The secondary advantage to that, though, is that the AI providers or data providers are also like hooking in the relevant business information for these use cases. We talk about corporate hierarchies, for example. Yes, there may be a general example of that could be scraped from the Internet based on public filings. but investment portfolio holdings could be a different story and knowing that structure is incredibly valuable when you're looking to cross-sell, upsell or essentially repeat business across a number of firms. And that's going to be the quality data that takes it from being, these firms might be associated to each other to oh, this firm is part of a 25% stake. And that was announced last year, there is a good chance they're going to replicate the same project that we've just sold over here. 
 

Sheryl Kingstone: And that goes into the most popular use case, right, -- and that whole lead gen pipeline management is like deal insights, opportunity prioritization, forecasting. That's where a lot of the value comes into play is really combining all of this data. 
 

Eric Hanselman: And so I guess in terms of some of how you actually accomplish that, it's still relatively early days in terms of the agent pieces. Is that something where we start looking because we talked about whether or not this is something where you get agent capabilities that get built into the platforms. Is this agentic overlays? How does that start to shake out from a practical perspective? 
 

Sheryl Kingstone: This is where Emily and I were doing a lot of the work around understanding the value of your traditional like CRM like Salesforce and what they bring to the table, what's changed with respect to the value of the LLM and how can we make that smarter with better data. And then on top of it, if you go beyond that, when you look at what I call the agent control plane, like we've been talking about this, right? So what is the agent control plane? It's around governance. It's around trust. It's around observability. It's around memory and state. It's around orchestration, right? A lot of that is also important with the combined entity of what your CRM brings versus what is in your data, like your CRM application process where it's trying to go. And then how is that going to play out and what is the role that the agent can help with and augment with. And that's where changes around the model context protocol has really changed because it's opened up again the ability to bring all the data together and action on it across a variety of sales systems. 
 

Eric Hanselman: So we -- I guess, a few years ago, we had the -- we were keeping that metric of the mean time to a mention of AI. And I think maybe we should start heading towards the mean time to the mention of MCP. 
 

Sheryl Kingstone: Sorry. the exchange... 
 

Eric Hanselman: But you bring up what is the important point, which is we've been through that sort of buzzy process of everybody says MCP, they said, but this is where the values that MCP starts to offer, which is the data exchange pieces start to make a real difference. 
 

Sheryl Kingstone: Everyone complains. Everyone complains. We don't have a generic way to communicate. We're giving you a generic way to compute them. And then we give them a generic way to communicate as someone going to be creating another buzzword. I you can't have it both ways. 
 

Eric Hanselman: Sorry. 
 

Emily Jasper: It's funny that you say that, though, because if I can add a little bit back to Eric, what you had asked around is it agents? Is it agent platform? How does this fit together? I think what we're seeing is that initial AI projects were off to the side because there was a concern around the agents touching especially proprietary data as well as disrupting workflows that had been established. It's hard enough to get a salesperson to write a visit note, but to then add in an element where they're asked to review if an agent captured it correctly, that's a whole other level of effort. nobody has time for that. But now we're seeing -- and it's the same issue everyone always has, you created a swivel where someone has to use the agent over here and it's not part of the native platform that you're also asking that team to work in every day. And I think that is where the change is starting to occur that the Agentic platform that is now being overlaid in these CRM systems. And this is across the board. Everybody has started to have these, and they're using a native data kind of cloud or zero copy workflow to expose underlying data to them so that it reduces that swivel and it democratizes access to some of that data natively that because of MCP makes it more accessible. The trick is, did you put the right data in? Or did you just throw data in because you could? -- which is essentially the same issue we had with cloud a couple of years ago. People just look to data sets in and then their compute costs went sky high. They had to crawl it back and be like, okay, is this actually valuable? And that's essentially the same scenario we're going to see in these agentic platforms overlaying on the CRM. You're going to have a lot of access to data. MCP is going to make it so much easier. Now you have to refine what is the valuable part of it. 
 

Eric Hanselman: And is this -- is MCP a pathway to actually understanding some of that data quality? Because a lot of times, the issue is how do we actually -- who has the scale to be able to get back into those repositories and really understand what that quality of data is. I guess... 
 

Sheryl Kingstone: Or if it's the MCP or if it's the training of the LLM and the orchestration that can pull it all together and point it out. The MCP is just like a communication protocol to integrate and combine, right? But it's not going to improve data quality if the data itself can't be pointed out, aggregated up or cleaned up. And that's where potentially the agent can come in and correlate some of the information or point out anomalies or point out concerns to be more proactive, that's where the agent is going to come into play because it's going to be trained with policy and guardrails. 
 

Eric Hanselman: Does that have to be integrated into the platform itself? I guess I bring up MCP because there's a question of how do you actually get access to do that data evaluation. Is that something that we have to actually have native capabilities within the platform? Is that something you actually do directly back into your back end? -- wrestled with forever, right? How do we actually do that data cleaning part... 
 

Sheryl Kingstone: Emily, do you have an idea? I mean, because most of my vendors that I work with are all building in these capabilities. But again, it's how you're going to use it. 
 

Emily Jasper: I was going to say, so I sit on the data customer side, and they are some squeaky wheels. They will tell you if something is not right. And so this is maybe where you have that trifecta of customer platform and data provider working together on not only quality feedback, right, I would be thrilled to have feedback from a customer that's implementing one of these solutions to tell me what ended up being the most valuable metrics in X, Y or Z that move the needle because -- not to say that you wouldn't still provide them the full data set, it will tell you something about their business, about their sales team, what is reliable indicators for prospecting or deal closure or reducing risk. We spend all of our days in trying to understand a banker workflow, a compliance officer's workflows, the salesperson workflows and what is the data most valuable there, where the agent is going to help pick things up is for this individual sales team, this indicator was more valuable than this indicator. But this sales team over here found these other things. That's, I think, going to be that language of feedback that becomes helpful. And I don't think you're going to be able to remove the 3 legs of that stool, I guess, you could call it, even if the agent is able to provide feedback because then you're still, again, having an action on the feedback. 
 

Eric Hanselman: So this is something where you still need to be building environments in which you've got process pieces that help you to understand what's valuable in your data and how you're putting it to work, as you're pointing out, find out what are those -- which aspects of the data, which particular elements are going to help you move the needle to achieve better results. And that gets right back to how you're using it, how you're measuring it and that full circle piece. That needs a level of maturity to make that happen. But is this something where agents can start to at least reduce the heavy lift that's necessary to do that integration? 
 

Emily Jasper: I think -- because I've seen this on the cloud provider side as well, which is those hygiene dashboards, really like it's this was the data that was hit the most. This is the data that was used the most. This is what may have been uploaded or downloaded depending on whatever those feedback metrics are and have started to see it in the CRM, whether you call them agent studios or again, it might be a dashboard metric, and it has that qualitative look at the data consumption. And I think you can't dismiss that then, right? Because it's just as valuable as the data itself, it's the how. And ideally, what that does is it shifts maybe the data scientists or data intelligence teams that you've hired to be looking at the how and how just as much as maybe the analytics that they used to spend all their time in. 
 

Eric Hanselman: So steps that actually can start to really make significant impacts. But taking again, it's so much -- so often, we wind up missing some of the second order pieces around this and really thinking about what are the -- it's not just the raw, it's, to your point, the house. It's what is it that's actually starting to make that difference and how did you actually get there. What's an organization really have to do? This is -- we've talked about -- one of the things that we have a lot of conversations with clients around is as they start moving towards better use of AI, the glaring problem has been how they handle data has been a challenge. And Emily, you pointed this out before. We went through these periods of data lakes and take all the data, all the data gets dumped in the data lake, and we've got all of who know heck knows what we've got out there. As long as we have it, we'll be good. to now maybe starting to think about how do we build an architecture that starts to put this together with an expectation that we we're agent ready, I guess, when you start thinking about this. Is that what we've got to get to, to tackle this? 
 

Emily Jasper: So Sheryl has this amazing model that I've been obsessed with since I first met her, which is the customer intelligence platform. And I think that is like the culmination of all of this. 
 

Sheryl Kingstone: Thank you... 
 

Eric Hanselman: But that's more because you're talking about the control plan. That is more than the control plan. That is a much more holistic approach to this. So for listeners who have not seen it, give us a little background. 
 

Sheryl Kingstone: It's a way to -- and you can't just go buy this today. You need to understand all the pieces of the puzzle to bring it together. I mean, yes, there are vendors that are offering soup to nuts type of approaches. But my customer intelligence platform strategy has been around for over a decade, right? So it's matured and finally come about. But it's about bringing first party, second party, third party together, all the data together, understanding what the value of unstructured data is versus structured data, make sure that you're bringing an AI and ML layer in there so that you can bring it all together and provide more of a structure and an identity and a graph to it. So we've really changed use cases to be more about the whole knowledge across the enterprise. So the use cases are not just for sales or not just for service or not just for marketing, it's across the enterprise and how we're leveraging this data. So it's a lot of technology behind it because you also have to have governance and trust and AI layer that brings it all together. But when I first came out of it, it was more of a vision -- and now with the advancements in just the pure data platforms out there that are able to bring together the structured and unstructured data, are able to even apply what is modernized a vector approach with semantic layer so that we can have the ability to have a RAG on top of it, which enables that conversational approach, which has really brought forward the CIP platform from a decade ago to a CIP platform today. Because if we really look to the future, it isn't about just bringing all the data together, it's having a conversational experience over it that provides the right insight, right? And we tried this like 2018, I wrote a report on Rise of visible interfaces because you could have a conversational overlay with some of your data on it. It never happened. It never happened because the conversational overlays didn't have an LLM driving and a reasoning engine behind it. Now that vision can come to reality, but it's only going to come to a reality if you've dealt with your data platform, right? You still have to make sure that, yes, you have your structured data, but you're leveraging all your unstructured data and that takes a new type of data architecture. 
 

Eric Hanselman: It's visibility across that entire data estate is critical. And the thing that we got with Gen AI was now the ability to actually have visibility and an understanding about that, the ability to now query in natural language kinds of ways. 
 

Sheryl Kingstone: And the data has changed. We've moved -- we've finally adopted towards streaming. We're using some of the streaming architectures. We have adopted more of the vector, the flu. We've gotten to more than just this omnibus data repository. 
 

Eric Hanselman: So what is it that enterprises need to be thinking about in terms of these next steps to get ready to really put agents to work? Is this -- we talked a lot about data and data architectures. Is that the focus? Is this making better sense of what you're... 
 

Sheryl Kingstone: You start there. You have to start there. 
 

Eric Hanselman: Data is job 1. 
 

Sheryl Kingstone: Data is job 1. Job 2 is understanding your processes. Job 3 is understanding what the value use cases on top of this. Or you start with the value use cases and go from there. Either way, you have to do all 3. It's a stool. 
 

Emily Jasper: And I would say don't be afraid to take a look at the backlog that might be 5 years old and see if any of those requests could be solved by an agent. I think that has been maybe the most painful part of working with technology teams that have these very deep backlogs and yet they're being asked to innovate at the exact same time, you almost need to separate perhaps out your forward-looking groups from your, let's call it, maybe a BAU team and take those opportunities to see if there's an agentic workflow that can solve for that so that you can start moving things along and you can test in small ways. Again, you're not trying to boil the ocean when it comes to the AI delivery of things. There's a lot of native integration that can solve some of these so that then you can focus on maybe the larger strategic initiatives that these tools are able to allow you to achieve. 
 

Sheryl Kingstone: Was operational efficiencies. 
 

Emily Jasper: Yes, start there... 
 

Sheryl Kingstone: Before it moved to revenue. Yes. 
 

Eric Hanselman: Yes. Before you transform, simplify... 
 

Emily Jasper: There you go. 
 

Eric Hanselman: Yes, we've been beating the drum on transformation for so long. But yes, just doing -- it's one of those basic parts of so much of this. Yes, a lot to do, but Emily, I think you raised what is a very valuable point, which is a lot of the concerns that organizations have had historically that they might have shell because there was that level of, oh, we could never do that are actually probably still real and serious business needs and are worth consideration in terms of what you're looking at for use cases because... 
 

Emily Jasper: Yes. 
 

Eric Hanselman: The mechanical... 
 

Emily Jasper: Yes, the configuration hydra that was CRM for years that makes you not want to touch it anymore, you can like -- I don't want to say you dismiss it, but you don't have to worry about cutting off one head and buy more pop up anymore. You can actually like maybe wheel this thing down a little bit. And I think that has been the biggest fear when it comes to CRM, especially if you have had one for a long time and it's configured out the Waoo, you don't want to touch it anymore because it might break something. That's where an agent can probably come in without a massive disruption and again, share or elevate some data that if you had to write a flow for... 
 

Eric Hanselman: That it can actually start to move beyond some of the brittleness and the risk that brittleness created to actually start to tackle some of these things. Well, this has been great. I think these are some really useful catalysts. Hopefully, our listeners will take these to heart and start to move this forward, but this has been great. Thank you both for being back on. 
 

Sheryl Kingstone: Thank you, Eric. Really appreciate the time. 
 

Emily Jasper: I always love seeing you, and I'm so excited to talk about this. I can't wait. Maybe have us back for a Part 2 someday. 
 

Eric Hanselman: Absolutely. And as I don't know, my listeners have heard me complain so often, so many questions that we don't get in the limited time we have, definitely a lot of stuff to follow up on. I will point also our listeners towards a research put together and our previous podcast, we made a whole bunch of these important points about what are the fundamental challenges that we got to overcome. But we are at time for this one. We'll have to get you back for next one, but thank you both. I also want to thank our production team, including Sophie Carr, Feranmi Adeoshun, and Kyra Smith on the marketing and events teams. I hope you'll join us for our next episode because there is always something, Next in Tech!