Next in Tech | Ep. 290: Headless Application Architectures

View Full Transcript

[00:00:00] Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry Research at S&P Global. And as we head into the Halloween season, I want to look at something that's, I guess a bit of a scary topic for some, headless application architectures.

[00:00:21] Eric Hanselman: And with me to delve into, this scary subject, is research leader Sheryl Kingstone, who focuses on application architectures. Sheryl, welcome back to the podcast.

[00:00:32] Sheryl Kingstone: Thank you very much. Looking forward to discussing this topic around the Halloween candy season.

[00:00:38] Eric Hanselman: It's certainly Halloween in, in our chunk of the world, but this is something that I think we look across application landscapes in their various forms, and certainly around SaaS and stuff. When people start talking about headless, I guess maybe for a little bit of background, can you give us really what is headless?

[00:00:59] Eric Hanselman: What does it mean? and is it, a- as maybe some have called for, the, the death knell, I guess I'm starting to sound more and more maybe like the application architecture crypt keeper here, but we'll see. Give us a little background on, on, on what headless happens to be.

[00:01:15] Sheryl Kingstone: Well, let's look into hallows of yester-year.

[00:01:19] Eric Hanselman: Ooh

[00:01:20] Sheryl Kingstone: Debunking this historical arc and hype. But the reality is headless has been around for a really long time. And if you look at what it was trying to do, it was trying to open up these monolithic systems of record, and we were trying to have these interfaces that were much more easier to use. They pulled in what I used to call a composite application. And the old problem really was the exact battle that we're fighting today: application scrawl, monolithic silos, CRM, ERP, billing screens. Seventy percent of the knowledge worker's day is lost in this administrative friction or not being able to find the data that they need, or they're tab hopping, or they're copying and pasting.

[00:02:12] Sheryl Kingstone: And so, we've been literally writing. So, one of the interesting things I did was I actually prompted one of my LLMs that I use, one of many, and I said, "How long ago did I write this?" And it went into its little archives of our database and it came back that we've been writing about this problem for about eighteen to twenty years, right? I know. And so, it started in the foundational days around mobility, and then it was all about this anywhere application. And then, it really did... This architectural vision was right, the infrastructure was premature. So, we started talking about web services and SOAP and XML and this middleware, and it was just too heavy and too expensive to maintain. And so, we kind of lost faith in this view. Then it popped up again with these task-based interfaces that were in the mid-2000s, and that didn't really work. And if you remember, there was also, Eric, the rise of this invisible interfaces with voice overlays, if you remember that, right? Didn't work. Didn't work. Wish you guys could see my hands right now. I'm talking with my hands, but this is a podcast. So headless, the bottom line is it's not this modern invention. It was a maturation, and maybe, just maybe, we're finally pulling the plumbing off and the intelligence layer and everything to go along with it to have this whole vision that we've been talking about even for the last couple of years around AI being the new UI.

[00:03:55] Eric Hanselman: So, with all of these regenerations, this is sounding a little bit zombie technology,

[00:04:00] Sheryl Kingstone: Zombie. The resurrection of the zombies.

[00:04:05] Eric Hanselman: But realistically, this is, I think the key point that you're making there is that we finally have gotten to an environment where maybe we have all the piece parts and the infrastructure readiness to really make this happen.

[00:04:20] Eric Hanselman: Is this something that we're on the edge of maybe we can go to a Body Snatchers comparison of headless actually starting to replace all of the SaaS apps that are there? We were trying to figure out, it was maybe two-ish years ago where, Satya Nadella at Microsoft had said, "Ah, agentic architectures were going to be the end of SaaS applications as we know them."

[00:04:46] Eric Hanselman: Strangely enough, hasn't happened yet. But are we getting towards some brave new future here?

[00:04:52] Sheryl Kingstone: Yes and no. Let me answer that question. Drives me nuts when something new comes out, and everyone says, "It's the death of XYZ." The reality is that it's not, right? It's the death of an interface that you hate, right? But what is underneath that interface? And everyone's like, "Oh, that Sheryl, she just covers the pretty UI." I wish I just covered pretty UIs. The reality is underneath and what these applications are... and someone else had said system of records are going away. I'm like, "So the UI's going away, and the system of record's going away. So, what do we have to use," right? So, it's not going away. What is changing is the underlying architecture that we can correlate across, right?

[00:05:44] Sheryl Kingstone: So, we don't have to bring together all the data, right? We don't have to have rigid application overlays, like objects and everything like that. Containers, right? We don't have to hard code a lot of this. We don't have these mashups and widgets and mobile portals and unified web pages on that presentation layer. Now we still have the underlying data infrastructure that's even more important today than ever. The SaaS applications have governance and trust, and they hold the orchestration engines. They hold the governance of our data. They hold access management capabilities. None of this is going to go away just because we have all our data, right? So really what we have to look at is the data integration layer is changing more towards MCP and vectors. The orchestration is moving away from hard-coded business process management rules to LLM reasoning engines and much more context engines. Presentation layer is much more finally dynamic UI, voice-enabled if we want it, natural language if we want it, headless if we want it, and I can go more into that once we discuss it. The role of the manager is more about understanding intent and delegating to some of these agents potentially. And the end goal is to break the silos, which always was to streamline worker tasks, right? But now it's to look more around intelligence, orchestration, automation end to end across the systems, whether that is the human or the agent.

[00:07:27] Eric Hanselman: And reducing, doing that thing that so much of the good optimization value that agentic approaches can deliver, which is reducing the toil. And because you were talking about it. We've had that sort of multi-screen whiplash kind of problem of going between the various systems and whether or not we're copy-pasting, ETL-ing, whatever the process was. Enterprise data busing. We’ve been through so many of these ways to integrate the information flow. This is something that now gives us the capability to be able to do that integration with intelligence, with an interface that allows us to interact in whichever form we want, you're identifying voice, traditional screens, what have you, and to do it with a level of understanding and context around what's actually getting done and what the outcome actually needs to be. Yeah, that, that seems like that's the big differentiating piece.

[00:08:27] Sheryl Kingstone: That and the cost, right? So, what people don't understand is what drove headless down in the past was because we were looking at the underlying complexity that we were trying to fix. So, developers had to write this massive amount of custom middleware code. They called it the glue. It was either point-to-point orchestration, whatever it was, it was expensive because it broke, right? Now, jump to where we are today around agentic headless, and that's really what's most important is the metadata around what we have in our systems. Catalogs, so declarative catalogs, so that platforms now can understand and read the metadata layers, and then they're automatically describing that back end, making them instantly usable. So, they're not hard-coded. That's changing. But what changed that was moving away from these integration standards, like what you're familiar with is these REST APIs, to that more what we've been talking about for the last, what, two years, is those MCP protocols. That changed almost everything because it allows that front end becoming that AI engine to query the data without custom code.

[00:09:42] Eric Hanselman: So, uh, okay. I want to dig into the MCP piece. And I think my concern for, in a lot of conversations we wind up having, is that MCP gets seen as this panacea, that it gets identified as all you have to do is put an MCP server on whatever you have, and then just go.

[00:10:02] Eric Hanselman: Suddenly you've got the ability to connect to everything. It seems like that, because you, you were calling out what was the real challenge, is that you need to ensure to get beyond what had been the challenge with a lot of the glue logic approaches that were out there, is that they were very brittle.

[00:10:18] Eric Hanselman: They were hard-coded in terms of what their expectations were or what they were talking to. MCP, it seems like risks going the other way, in which it is all of the everythings to everyone. All you have to do is figure out what the heck it is and what the capabilities are that it has. And if that MCP interface isn't well put together, that becomes just as much a complication as anything else.

[00:10:42] Eric Hanselman: Because you do have to understand what it is that's actually being presented. What are the practical realities of MCP, and is it as easy as that?

[00:10:51] Sheryl Kingstone: Well, it's a lot easier than where we were, put it that way. It’s not quite as easy as this, what I would call, this USB-C type thing. Just plug it in and go, right? So the way you would look at it is you used to have all these different power adapters, and you have to travel to 20 different countries, and now you just have this, one USB-C, and it's all standard.

[00:11:17] Sheryl Kingstone: It's not quite that easy, right? It’s a lot better than where we have because the industries and the enterprise applications are not arguing over it. Everyone's bought into it, right? So instead of hard coding these API routes into that front end, an MCP server is dynamically declaring, "Here are the tools I have, here are the permissions required, and here is how you can call them." Right? So, it's less brittle, a bit more plug and play. Is it solving everything? No. But it does help establish this vendor agnostic way of specifying an interface. It is why all of a sudden the executives are loving this. They go into Claude, they have their MCP connectors, they're able to bring in information from Smartsheets and from Salesforce and from Asana, whatever it is, right? And then they create their dashboards. And then all of a sudden they realize, "This is really great. This is pretty. This is beautiful. Look what I did." But then they realize what after that, Eric? What do you think these executives realize even though they've made this wonderful, pretty dashboard and this MCP connector was easy for them to use? What was the underlying problem?

[00:12:48] Eric Hanselman: The quality of the data behind it?

[00:12:54] Sheryl Kingstone: The data, right? Again, I can go back two decades, two, three decades. There's not one executive that wouldn't say the data's not important, but we never did anything about it because they got what they needed because they would go to their IT department, and they would, all of a sudden they would have their dashboards.

[00:13:12] Sheryl Kingstone: If they go into Salesforce and they would have sticks over to the salespeople to enter what they needed, and marketing would come up with theirs and by next quarter, they would have some numbers, but now they're realizing this fast pace, and they're developing their self, and they're realizing, "Oh my God, the data really does need to be much more real time. Needs to be much more connected. I have to have all this metadata language. I have to have it grounded in the truth. IT, what are you doing? Get my data fixed. Line of business, enter your information." So now it's coming top-down, not middle-down and middle-up, right? So before we always have middle managers struggling with the quality of their data, and the employees would go do it, but it would suck because it was too hard, and the executives would eventually get what they needed two months too late. So now we do have a real reason to finally understand what is the single source of the truth and what are we working from?

[00:14:13] Eric Hanselman: So, what's required to really understand all that data? And I want to invoke what has been one of the buzzier and I think starting to be overused terms, go data ontologies. And yes, for our listeners, I won't go into describing the face you just made, but, there is this idea that you have to have a broad ontological description of the world in which you're working in order to effectively ensure that context is communicated.

[00:14:45] Eric Hanselman: Because, model context protocol, context is its middle name.

[00:14:49] Sheryl Kingstone: Yeah.

[00:14:49] Eric Hanselman: If you don't have agreement in terms of how you're describing context, you run into the challenge of not really understanding what the nature of that description is. Is this the color white?

[00:15:03] Eric Hanselman: Is this the coloration? All sorts of different things can be identified in terms of context, but that gets into this whole big epistemological kind of thing that has to happen. Do we have to get to those levels? and are teams capable of that?

[00:15:21] Sheryl Kingstone: It's a good point. We have to get to fix the greatest failure of our legacy headless, which was its semantic blindness, right? So, what I mean by that is the metadata the system of record is what's giving the headless systems this self-awareness, and that's absolutely critical. If you call it an ontology, that's great. Let's just talk about the data that describes the data, the catalog, the semantic layers, the enterprise registries. Self-describing that is very important, because that's what the humans have always needed, but the agents need even more because they don't have that context at all, right? So, they need to declare that context. So what people are talking about is, some people are saying, "Well, we don't need a system of record anymore, and we just have this data storage, and all we need is-- or we don't need applications anymore. Maybe all we need is a data catalog and data storage with some orchestration layer and intelligence layer on top of it."

[00:16:28] Sheryl Kingstone: I mean, I literally heard a vendor say that. That's so naive. It's unbelievable naive because we're not going to get rid of our applications. So now what do we do, right? So, we still need to understand and make sure what I would say is the metadata system of record, such as your data catalog and your enterprise registry, is solid. That is what is IT's responsibility now. They have to make this enterprise directory and rule book so that everyone is working from the same thing. Meaning here is the data, here are the compliance rules, here are the actions that you're permitted, that's going to be the hardest part. So, the role of moves from these plumbing fabricators creating all these APIs, to curating this metadata catalog and governing that control plane. Absolutely critically, essential.

[00:17:32] Eric Hanselman: So, this winds up being that really big job that, that historically we haven't really tasked IT teams to do, but it's happening at a much higher level. This means that IT teams actually have to have a better understanding of the nature of the data as opposed to just managing the containers that are housing it, and doing things like access and all the bits and pieces.

[00:17:55] Eric Hanselman: They actually have to think about what that data looks like and have an understanding about how it's going to get used.

[00:18:02] Sheryl Kingstone: Yeah. So instead of building and maintaining code, which is what they've done in the past, they're like, "Well, you don't need me anymore, right? You can do it in Claude." That's not true, right?

[00:18:12] Eric Hanselman: But my little Claude does so much!

[00:18:14] Sheryl Kingstone: I know. But, now you have to understand and stop being reactive versus proactive, right?

[00:18:21] Sheryl Kingstone: Instead of patching security holes, you have to be proactive. You have to set global policy. You have to understand identity. You have to set the guardrails at the gateway, right? Absolutely critical part of an IT shift that must happen. Again, instead of maintaining that code of the front end, you're exposing headless endpoints and letting lines of business deploy their own user interface.

[00:18:46] Sheryl Kingstone: So no longer are you trying to say, "Okay, give me your specs. I'll come back to you in eight months, and I'll build it." And by then the executive comes back, and they say, “This kind of wasn't what I wanted."

[00:18:57] Eric Hanselman: And that shifts that responsibility from the gates for all this, the controls were always the application UIs and all we wound up doing from the IT side was having to say, "Okay, here's who has access to this app," and, maybe some bits and pieces of role-based capabilities we had to configure underneath that.

[00:19:14] Eric Hanselman: Now we've actually got to understand not the person accessing the data, but we have to translate that into the data itself

[00:19:24] Sheryl Kingstone: Yes. That's why I talk about the control plane is so important.

[00:19:28] Eric Hanselman: Yeah. But that's such a fundamental shift and I guess it gets to that question, so how do organizations start to think about how they make this transformation?

[00:19:41] Eric Hanselman: What do they do? And I guess maybe what's that first step? What are the things that they should be seeing as the most urgent next step in terms of this transformation?

[00:19:51] Sheryl Kingstone: I still go back to making sure you understand the data catalog and the metadata around it. Look at your systems of record today. What is able to do this and what is not? Because not all of your underlying applications and data stores that you have today can do this. You have to shift your attitude from being these IT as plumbers towards IT as curators of information and intelligence, and putting controls in place, and setting global governance and identity parameters and security parameters. And then let line of business move fast. Empower the line of business. We've been talking about this IT and line of business disconnect for three decades. I'm so sick of it. Let them go. Put the parameters and say, "You build it," because now they can. So, if we have shifted to AI as the new UI, why do you want to build it?

[00:20:51] Sheryl Kingstone: The most important job is assuring what they're using as the interface is the right underlying data. The bottom line is stop paying attention to these pretty front-end dashboards and invest in things like semantic catalogs, data layers, and the visual layer can be highly disposable. It's going to change based on what that executive needs at that point in time, and we all know how difficult they are to please.

[00:21:18] Eric Hanselman: So, I guess what this translates into is that, the lines of business are those trick-or-treaters out there looking to get data in their baskets as they go door to door from application to application.

[00:21:31] Sheryl Kingstone: Exactly, and they want the little candy that they can just go eat.

[00:21:36] Eric Hanselman: And they get to consume it themselves.

[00:21:38] Sheryl Kingstone: Hopefully they don't consume too much all at once, but, I'm not going to stop giggling.

[00:21:44] Eric Hanselman: Well, I, I don't know. May- maybe we've pushed this metaphor as far as it can possibly go.

[00:21:49] Sheryl Kingstone: I want some candy now.

[00:21:50] Eric Hanselman: We'll see if we've made our listeners hungry too. But this has been great, Sheryl. Thank you very much. I will point our listeners towards some of your recent research on this. It is something that is, I think, hopefully a welcome transformation for IT teams. And a lot of potential when we move forward, but, a very different way to be headless. Thank you.

[00:22:12] Sheryl Kingstone: I will be writing this all up, so we will get this out the door.

[00:22:17] Eric Hanselman: Ah, great. Well, and we'll have links to get people to there.

[00:22:21] Eric Hanselman: But that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Dylan Schieble, Sophie Carr, and Feranmi Adeoshun on the marketing and events teams. If you liked this episode, please subscribe or like us. I hope that you join us for our next episode, because there is always something next in tech.