Next in Tech | Ep. 278: DevOps and Observability

View Full Transcript

[00:00:00] Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, chief analyst for industry research at S&P Global. And today, we're exploring some recent research on DevOps and observability. And with me to discuss it is Mike Fratto. Mike, welcome back to the podcast.

[00:00:23] Mike Fratto: Thanks, Eric. Thanks for having me on

[00:00:25] Eric Hanselman: Well, it's great to have you back. And you've put some research out recently driven off of our automation and orchestration study, and it's one of those things that, I don't know, it just seems to be that, that constant struggle of getting organizations to higher levels of automation. And of course, now that we’re wading into all the implications of AI and agents and ideas around autonomy, it's right back up into the forefront again.Can you give us a little background about what you're seeing and what's your take on it?

[00:01:01] Mike Fratto: Yeah, so, the study that we, that we just, finished up, it fielded over the winter. It was seven hundred IT professionals, from IT managers up to senior level managers. So, it was a pretty good, sort of overview, both from a technical standpoint and sort of a technical management standpoint.

And, overall, what's interesting was, the respondents understood that they had to improve operations, in order to improve, what's going on within the business, right? Be more responsive to business needs, reduce that downtime when it occurs, maintain that uptime and application, reliability and access and all of those things.

Basically, get out of the way of the business and support the business, and they were looking at automation as a way to do that. And so, there's a lot of positivity about using automation, and automation is a way to really sort of help IT and help the business, as a, as its primary focus.

However, there's still a lot of hesitation about whether or not they actually get there, get to that sort of, that automation and orchestration nirvana, that's being continually pitched to them. And so that was really the top-line view of this particular survey.

[00:02:17] Eric Hanselman: Well, but this is something that, that enterprises have wrestled with, I don't know, it seems like forever. Because this is, you know, the, the, all of the AI-ness and autonomous operation-y kinds of stuff, I think is just the latest iteration. But we've had the tools to do various levels of automation for years and years, and it seems like each generation of automation improvements move us forward.

And yet there still seems to be this fundamental concern from the enterprise side, a- about the trustworthiness of those automation capabilities. Are we moving the needle or are we getting people further down the road? I'm curious, what has to happen to, to go to this next level?

[00:03:07] Mike Fratto: So, I think we're continually moving the needle a little further, right? Or moving that rock down the road a little further. We haven't gotten there yet. And I think fundamentally the challenge comes down to education and experience. If you're an IT person and you're thinking about IT systems and how to operate IT systems, you tend to think atomically.

It's like, I've got my network, and I need to operate my network. I have my servers, I need to operate my servers, and that kind of thing. Automation, relying on automation and building automation systems and understanding how they work is really looking at a systemic view of whatever systems you're looking at.

So, it's almost like you don't care if it's a switch or a router. It's a networking device which you can now program and you can do things with. So, it’s that kind of viewpoint that I think needs to shift and that's a non-technological answer.

But I think technologically, what's one of the interesting changes that we're seeing, and I'm going to bring up MCP because we can't talk about automation and AI without talking about MCP.

[00:04:22] Eric Hanselman: Yeah.

[00:04:22] Mike Fratto: But there's a very fundamental reason that I'm bringing up MCP in particular when we start talking about AI sort of managed systems that we're seeing more and more, is if MCP is, is deployed well, then that removes a lot of the integration problems that we end up having, which can stymie, which does stymie, I know it stymies, automation.

 So, all of a sudden if your networking company has an MCP server that, that you can talk to and the other networking company that you're using also has one, you can now talk to these two things in the same language, and they'll figure out the details. And that's just one example of some of the things that are going on.

[00:05:08] Eric Hanselman: So, it seems like we're getting to a point at which maybe that trust factor gets enhanced by visibility because when you talk about, MCP, I mean, we've had in some form or fashion, there are always APIs in, in a lot of services. The problem was the availability of that information wasn't great.

Everybody had their own specific API. Actually, getting it out of something was kind of a pain. But it sounds like what you're heading towards is that getting visibility into that automation flow starts to get you a greater level of trust in that you, you can see what's going on. One of my favorite topics to harp on is, the power of technology to raise the level of abstraction and the thing that you're identifying of, if it's like if we're doing network automation, the, do not really care whether or not it's a switch or a router, but you now have some abstracted controls that let do very high level things of, connect these two things or, block this traffic or allow this traffic or, whatever.

It seems like the missing piece has been the visibility and the observability piece to that. What's the extent to which you see that as actually starting to help maybe overcome some of this resistance or is that one more thing that teams have got to understand and trust in order to get there?

[00:06:30] Mike Fratto: They're going to have to understand and trust their observability platforms, more importantly, deploy them. But what's interesting about observability is there's sort of the big picture application performance aspects which are useful and beneficial. But when we're talking about IT automation, there's a lot of, I guess I would call it grunt work, foundational work that an observability platform that, that itself is automated, does for you.

So, for example, one of the hindrances of automation is your environment changes. And it may change all the time. And keeping up with those changes is extremely difficult. I mean, you have to be very detailed. Like, you change an operating system on a switch somewhere, and you've got to log it, document it, and make those changes and see if it impacts any of your automation. A lot of the observability platforms can do a lot of that work for you. There's, oh, I see the change, I can integrate it in. You now have this current network model, and there are startups that are focusing on exactly this problem. But so now we have this network model or this IT model, which is consistent and is always current.

So now you don't have to worry about that as much, as long as that model is correct. Once that model is correct, now the automation platform, the observability platform, automation platform, whatever it happens to be, can start coming up with playbooks and scripts and rules and other changes that can be affected because it knows the current state of the system.

And that starts to unlock a whole lot of things. And you don't even have to go to the point of, say, writing a playbook. If you've got playbooks for certain actions, then the system can just pick those out and go, "Oh, you want to add an entire new office? Okay, here's the playbook for that. I'm going to pull this over here. I'm going to populate it with the information that I need, and I'm going to push it out. I'm going to execute that playbook." You know, that simple. So, it can both use work that you've already done, or it can potentially create new playbooks and new scripts that it can then execute.

So, it takes care a lot of the basic work.

[00:08:33] Eric Hanselman: Well and it seems like that really addresses what has been one of the big challenges, which is the brittleness of a lot of automation tools. Because, you think back to, systems administration stuff, Puppet, Chef. These were things that back in the day, you had specific sets of scripts for specific devices.

This server configured this way, you could run these scripts on it, and it would do the Windows update on it or add whatever capability was or whatever was there. But you bought a whole bunch of new systems or you upgraded them, you added more capacity, you put new stuff in them, suddenly you got to change the scripts because now there's more stuff that's in there.

And we saw some of this change as we headed to tools like Ansible, when you're talking about building playbooks for some of this. Those are things that, that started to get better. The cloud automation stuff, was always much more expansive, but the nice thing was with the cloud, you had a really uniform surface that you were working across because you had a cloud provider.

Running like things like Terraform tools and things like that, hey, they're great because you always have the same means of doing the thing, and the services are all reasonably stable, and you've got this sort of uniform control surface across which to work. But, most enterprise environments, you don't have that same kind of capability and it's more of a challenge.

But I thought it was interesting, the point that you were making about using that observability data to do the thing that's really hard. Which is one, identify all of the stuff that you've got, and two, identify changes that are happening. Because, the environment is not static. Things are going to change, and the ability to be able to actually integrate that into the automation.

That's... I guess we're getting closer to closing the loop in both the visibility side and the automation side, but it seems like that, that's a big part of starting to automate away some of those challenges.

[00:10:36] Mike Fratto: Yeah, it is. And once organizations start to go down that path of merging observability with their automation platform, with or without AI, and that, that's already occurring, it starts to unlock a lot of, potential, a lot of time for IT, for example. Some very simple things that can save hours and hours of time is, for example, what happens when an event occurs?

An application goes down. There’s some kind of disruption. Well, the first thing IT has to go do is spend hours gathering up all that data, formatting it, processing it in a way that they can now start to understand it. And an observability platform already has that information because it's monitoring the systems.

So, it can create and/or collect, automatically, and then format that data much more quickly. So, it might take five minutes to collect all that data, where it might take a person an hour or two or longer to collect all of it, and it can do it continuously. So, you've just saved an hour and fifty-five minutes in your troubleshooting process.

On the back end, if your observability platform is plugged into, whatever you're using for workflows, Slack, Teams, whatever it happens to be, ServiceNow, looking at the tickets, looking at the systems, it can do things like postmortems. So, what happened? And it can write that report. And now all of a sudden, not only does it write the report, it can ingest it or send it off to an AI management platform, now it becomes part of local learning. And that becomes really useful. And so there's a bunch of sort of rather, seemingly uninteresting things that observability platforms can do today, but actually save IT staff time, allowing them to work on more interesting issues and solving the problem.

[00:12:27] Eric Hanselman: It's being able to take that stuff, which is the unsexy part of a lot of those tasks, and being able to tackle them in ways that improve reliability because guess what? Humans are singularly error-prone. One of the reasons why we should be automating, is because, repetitive tasks that you know how to define and constrain well, automation can do that a whole heck of a lot better than, than humans can.

And that next stage of being able to do all of the, the tidying, the documenting, all those things that, you know, who wants to go write down all of the various, address changes or, the configuration changes and what have you. I think, also the postmortem piece is another interesting part of that because, how many times do teams just get really busy after an event and nobody bothers to write it up because holy cow, we've got, one, we spent so much time recovering from the event, and then two, we had to catch up with all the other stuff that we weren't able to do because whatever the event was going on.

And so whatever potential for learning from that event happened to be, just simply drifts out into the ether because we didn't have time to document it. And that's, that has a lot of value in tune of itself.

[00:13:47] Mike Fratto: Yeah. Oftentimes people, or oftentimes teams don't go back and look at those postmortems. So, taking that postmortem and then feeding it back into whatever machine learning system, if you have one, all of a sudden that becomes part of the knowledge base automatically, and it becomes, immediately useful.

From the survey that we did, the benefit, some of the benefits are things that we're talking about right now, the benefits that the, the respondents found. It's things like less time spent on data collection and formatting and that kind of stuff. lowering time to resolution.

And oftentimes better time to have a higher likelihood of figuring out the remediation on the first try, right? And that's largely because the observability platform has the knowledge and the continual learning where it's able to provide better and better, say, root cause analysis and recommendations, the longer you run it and the more that you feed it.

[00:14:47] Eric Hanselman: That which brings us to the idea of those learning platforms, AI assists. What kind of role can AI play? And it seems like we're also in that, that early stage of getting to a point of trusting AI capabilities as well, until they get better proven out. Even though we seem to have a lot of really great examples, I mean, you know, with my security hat on, the ability to ingest vast amounts of data and make sense of what are potentially, very rare anomalous events.

AI is doing phenomenal things on that front, but on operations, we seem to be right back into that, that, that level of trust concern.

[00:15:30] Mike Fratto: Right. There’s a level of trust. It’s due largely to, to lack of experience and to some of the times where AI can be not just wrong, but hilariously wrong. We tend to remember those quite a bit. organizations are seeing benefits across the board.

There’s the unsexy stuff like feature discovery. So, you can go into the GenAI portion and say, "How do I do X? How do I make a report? How do I do this? How do I do that?" And it’ll surface up that information to you and offer other kinds of features. So, there's this, just that basic stuff, which is just, it saves you reading the manual, right?

Because it can take you right to that portion of the manual. That’s helpful and that, that's really useful. But more importantly, pulling off your example, in security, by looking at that vast amount of data, there's the potential for an AI platform to, see early warning signs that might get missed, in that sort of raft of data.

So if some temperature probe in a rack somewhere starts to see an increase in temperature, and then, shortly thereafter, you start running into other sort of hardware counter errors, and then shortly thereafter that you start seeing other sorts of issues, these are things that may be early warnings of a potential problem.

And if you see it once, twice, three times, it starts to form this pattern. And there's the potential of, it can start to alert earlier in the process and say, "Hey, I've seen these three things. It's led to an outage before. You might want to check and do something about it." There’s that kind of early warning. But then it's just making recommendations, helping, figuring out, is this an application problem? Is this some kind of code problem?

Is this some sort of system problem? And at least cutting off the paths that you might think that are not relevant. For example, it may not be the network nor DNS Right?

[00:17:34] Eric Hanselman: Wait, it's always the network, right?

[00:17:37] Mike Fratto: Everybody hates networking. But think of a, think of an AI platform, and we're talking, in reference, specifically to DevOps, but Ops in general, which can go in and look at the application data and say, "I see this problem, and here's the line of code. Here's the database query. Here's the whatever which seems to be causing that issue." And then you can now take that and give it to a human, and they can go figure it out. Or potentially the platform can say, "This is the line of code that keeps breaking," or whatever the problem happens to be, "Here's how to fix it." And then it can actually maybe write the new code block, which then gets reviewed, and then you can push it live. So, there's a lot more potential that we're starting to see, and organizations are starting to use those more advanced features as they gain experience with AI platforms.

[00:18:32] Eric Hanselman: It's a lot of, again, those big-picture correlation problems of, okay, we updated this library and rolled out new code. What was the problem? And so, we pushed to production, and suddenly there was an issue, and this one couldn't talk to that one. Well, it must be a network problem.

Well, you know, if in fact this one had a library update, new code got pushed out to this other system, to be able to then actually start to triangulate in ways that might be harder to correlate in other ways if you didn't have that information, if you didn't have the telemetry that was, letting you know that where changes were being made, where events were taking place, what challenges you actually saw when these things were happening.

[00:19:16] Mike Fratto: Yeah, exactly. And it's understanding and monitoring these dynamic systems and being able to pull that information together, analyze it, and then point IT in the right direction.

[00:19:29] Eric Hanselman: Well, yeah, I was working on a project many eons ago, and seemed like about every other week or so there'd be this one major performance problem. And they were trying to figure out what's going on, and all the systems seemed to be up, and things were kind of running. And they didn't have a lot of telemetry that, to actually tell them what was going on, but they knew that maybe there was a network performance problem.

And what was happening was, in fact, the team that was running a warehouse system would update the warehouse system, whenever they had new builds. And there was this huge push that impacted both the storage environment and the network environment. and because there were so many disconnected indicators, they couldn't pull them all together.

And it wasn't until they got all the teams together to be able to go think about this. These are the kinds of things that with greater observability, now you've got a place to be able to go pull all that information together, and finally some tools that can ingest and correlate that at scale, which, that seems to be the big piece to my mind.

So, from an advisory perspective, that seems reasonably straightforward. But you were making the comment that, that granted, everybody's got an AI fail story somewhere. How long does it take for people to get over those to get to a point at which they trust the AI in ways, whatever their deployment happens to be, in ways that they start to act on this? Or is this a different way of thinking that's needed around, AI augmentation?

[00:21:02] Mike Fratto: I think the trust will build quickly. I think that the idea of having a human in the loop is... I think it's going to be with us for a long time, especially for new processes, new issues that are not well understood. IT has, you know, they run into issues, it's well understood, they're- it's well documented, so much so that they could give a written document to a junior engineer and trust it that when they see these symptoms, they go do this thing, and you don't need to have a senior engineer signing off all the time.

It’s those kinds of processes that are well understood. You know what it looks like. You know how to fix it. Those are the kinds of things that, that, IT builds trust in their own workflows. And so, once they give them to an automation platform, they'll build trust that they can just go and automate those things.

And then for new events that are occurring or maybe are not as well understood, having that human in the loop so the AI platform can go out, it can use the observability platform, whatever data sources it happens to have, specialized, models that it's using or general purpose model, it doesn't really matter.

Go get all the information from all the IT systems that's current right now, format it, analyze it, present a report, say, "These are the symptoms that we're seeing. This is where I think the problem happens to be, and if there's more than one or two potential root causes, I, can list those out."

And then a human can go "Well, okay, this makes sense. That makes sense." Can do their own investigation. You now saved all that time that an IT person is spending doing that sort of basic work. And then once it goes and says, "Yeah, okay, I think it's this issue, what's the fix?"

And this is the great part because, back when I cared about, actually touched IT systems and actually did things, I was a horrible scripter. I was terrible at it. I was good, but I was bad, I was bad. And so having an AI platform which can write a script, suggest fixes, can do a lot of that work.

It doesn't have to be efficient. It doesn't have to be elegant. It just has to run and run well. That's it. That's all I care about, right? This is not going to be a core loop in my customer-facing application. I'm going to run it. And so, all of a sudden, that starts to open up a whole lot of possibilities for how to deploy remediations and fix problems more quickly.

Take the script that we've been using for the last five years, suggest some changes. Somebody can review the changes and then approve them, and then push it, execute it and make whatever changes and fix the problem.

So, it starts to unlock a whole lot of really interesting potential. And then we get started getting into the possibility of using digital twins to first test fixes and test changes on a digital twin to see what breaks or what the impacts are. That provides another sort of safety net in that whole workflow process.

[00:24:06] Eric Hanselman: One more level at which to take it to be able to ensure that you've got sufficient trust. And interesting your point about the digital twin piece because that, again, you build the digital twins from all of the telemetry data so that you can identify how it operates. One more reason why you have to have sufficient levels of observability built into that environment to help feed this process to build trust and to make that whole thing work right.

It’s hopefully something that comes full circle

[00:24:39] Mike Fratto: Exactly.

[00:24:41] Eric Hanselman: Well, this has been great, Mike. I appreciate all the perspectives, and hopefully this will help an enterpriser to move a little further up that ladder. But, clearly a, a ways to go, but a lot of benefit if you can get there.

[00:24:55] Mike Fratto: Yeah, there's a lot of benefit. There’s a lot of incremental benefit that organizations are seeing. We see it in our survey results. And we've continued to see it for the last three or four years that we've been, that we've been running various observability surveys.

It’s not like you're going to go from zero and have a fully automated system. There's all those benefits that you pick up along the way. It's a journey with a village.

[00:25:21] Eric Hanselman: Hopefully builds trust along the way. I will point our listeners to the research. It'll be in the show notes. But, appreciate all the insights. Thanks for being back.

[00:25:30] Mike Fratto: Great. Thank you for having me.

[00:25:33] Eric Hanselman: That is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun and Dylan Schieble on the marketing events teams. And if you liked this episode, please subscribe or like us. I hope you'll join us for our next episode, because there is always something next in tech.