Next in Tech | Ep. 247: Security and Observability

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, an S&P Global Market Intelligence Podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry and Company Data at S&P Global Market Intelligence. And today, we're going to be talking about the intersection of 2 key parts of the IT stack, security, and observability. And there are 2 areas that have often existed independently and yet there's so much in terms of the way in which they're entangled. And we've also seen some recent M&A activity that starts to be confirming the fact that these are really intertwined in ways that maybe we're starting to finally leverage some of the benefits of bringing together the kinds of things we've been doing on the operational side and the security side from the security capabilities. And with me to discuss it are an August panel of folks from all the various sides of that equation, Scott Crawford, Mike Fratto and Mark Ehr. Welcome back to the podcast to you all. 
 

Mike Fratto: Thank you. 
 

Mark Ehr: Thank you. 
 

Scott Crawford: Thank you. 
 

Eric Hanselman: It is great to have you back on. And when we think about what's been going on, well, maybe we'll start with just some of the obvious big picture things of the M&A activity. So, the Palo Alto Networks and the Cronosphere deal, which maybe on the surface of it, maybe had some sort of scratching their heads a little bit. What are your takes? I guess, maybe, Scott, from the security side, this is from a Palo Alto end of things, this on one side could be seen as a great integration from security. What's your take on how that bolts up? And what maybe was driving some of that? 
 

Scott Crawford: Well, it certainly is or could be a great integration from the security side, but Cronosphere is known -- arguably known primarily as an observability provider. And of course, they had a great deal to do with the M3 architecture at Uber, which is where they gave a lot of credibility for what they did for observability enablement. This isn't too unlike Palo Alto's approach to prior acquisitions, though. When they picked up CyberArk that wasn't too directly integrated a technology buy, and there will be a way to go to getting that fully integrated. In fact, CyberArk was really kind of an earlier generation of privilege management in that case. But a huge chunk of the market, huge presence of footprint in the market, very important to a company that has ambitions to achieve big numbers in share and overall revenue as a percentage of that market. So, it's not unusual to see Palo Alto buying big chunks of existing traction in a market where they don't have existing or current penetration. Technological integration may follow, but Palo Alto's approach to a platform is to bring those segments together first and integrate as they are able to do over time. Observability is a big chunk because it's -- in security, you typically have to go for a consensus sale in the first place. That means you've got to build confidence among all the buying centers that will have a say in where you're going to take some of their budget for security investment. And there's areas of observability, infrastructure monitoring, application performance in addition to log management that they just didn't have access to that same budget. So that's a good motivator right there for the acquisition. But technologically, yes, Security platforms do need to pull data from a lot of sources. They need to transform it to make it usable to security analytics and operational tools. And nearly everyone with platform aspirations, I shouldn't say nearly everyone, but quite a few of the larger names in security platforms have been acquiring in this data pipeline space over this past year. And in fact, Cronosphere was directly involved with CrowdStrike, with Humio and log management, log aggregation. So that's a motivation. A couple of motivations here for that. But yes, definitely a very strong footprint in observability, which is not typical for security vendors. 
 

Eric Hanselman: So that says observability is driving the bus. Mike, maybe I should have started with you on this. I mean observability is in the front seat of this. What's your take? Is this -- again, is this a final realization that observability is finally getting its place at the table? Where does it fit? 
 

Mike Fratto: Well, so I think it's not necessarily the final realization. There's been a lot of benefits of combining IT operations monitoring over the years with security, tooling, and security analytics. Those integrations have worked more or less fairly well with the advent of AI and the development of observability as a discipline, the -- what is being done with data, everything from the collection and the storage of these huge data lakes and the -- all the data processing that goes behind the scenes, which then makes the observability platform very responsive when you ask complex questions, right? You're not necessarily writing SQL joins or anything quite like that. They're much faster than they used to be, which is important when you're working with data. But the thing is you're taking all these different kinds of data, right, metrics, events, logs, traces basically from full stack and presenting it up to an operator in a way that they can easily consume it and they can drive up and down the stack and they can look at events and alerts and do root cause analysis and a lot of the things that have to occur when the security event is going on. And so that's what observability is driving and is sort of making it available not only to our friends in security, but across the IT landscape. Among the vendors, the observability vendors, there are some very large ones in the market, but there's a lot of really nice sort of start-ups like Cronosphere that were just ripe for acquisition. They've got all the right elements in place. They're getting good traction with customers. So, they've got revenue coming in. They know what they're building. Cronosphere, in particular, was very deliberate in their product development and their feature development, focusing very much on their end customers. They did an acquisition of their own of Calypia earlier in the year, which is an observability pipeline, which is really helpful, especially when you have -- you're collecting data, but you need to send it to multiple sources or multiple destinations. So, it's -- as an acquisition, it just -- it made a lot of sense for Palo Alto to pick up. 
 

Eric Hanselman: So, in terms of some of that evolution, are we finally starting to get some of that melding of all of the important operational side information and understanding what the security value is to that because that seems like that's been the challenge we've had. We did -- our mean time to AI mention on this episode was about 5 minutes. So we held off a little bit is -- are we getting to a point at which because AI piece of this is that we finally have a tool that can help us really do that translation of domains or the importance of that data because again, we've had operational data forever, never. And yet, we've always done the thing on the security side, which is we've tried to intuit a lot of that context and put it to work because the volumes of data were hard, the kinds of integration pieces that we needed to have maybe were complicated. And these are things that from a security operations perspective, it seems like we never really quite got there. I mean, Mark, in terms of sort of what that that integration effort that's always seemed to be a real challenge. 
 

Mark Ehr: Yes. When I think about it, I start back -- if you look at the kind of the evolution of SIEM, which was purely security signals at the beginning, right? And starting probably 2012-ish or so, they started to think about observability. But at that point, it was really around trying to get data into it from like a behavioral perspective, that kind of stuff, right? And later, forensics kind of came in. But it was always kind of difficult, right, because it was expensive, lots of data, that kind of thing. 
 

Eric Hanselman: And for listeners that aren't familiar with it, the SIEM SIM security information event management, this was going -- and it's a concept that's been around for decades, plural maybe now. 
 

Mark Ehr: 2003... 
 

Eric Hanselman: There you go, a couple of decades. And the idea was we had all the security data. And we didn't have a way to correlate it and bring it all together and leverage it. And what tended to happen is that we get a system, we have to bring in all this disparate and online data and then have a translation layer that sort of said, "Oh, these kinds of things get matched up with these kinds of things. And we've got knobs to turn for how severe is this kind of thing. And we get this kind of information from over here, we need to tweak it with this sort of -- with this particular bias. This is stuff we like. This is stuff that's important. But the correlation and integration were kind of difficult. And generally speaking, it took a lot of care and feeding to constantly tune and manage that. But now... 
 

Mark Ehr: Right, to kind of finish that timeline, right? So Splunk comes along, right? And Splunk was really the first major vendor to be able to -- and they kind of started on the observability side and later picked up security. But they were a big driver in that market and continue to be so. But what's happened, I think, is part of it is cloud native, right? So -- and storage has gotten a lot cheaper, right? So, it's easier to get that data into one place, coupled with standards like the Open cybersecurity framework, our schema framework and open telemetry, right? And so now we can get this data into one place conceivably, that's the vision anyway to a data lake that's already normalized in a usable format. Now you combine that with AI, which is a natural cloud workload anyway, right? And now we've kind of got this magic combination, I think, being able to get all this data in one place and then use advanced AI to actually process it and make sense of it. 
 

Scott Crawford: Yes, you'd have to build all that by hand, though. You'd have to build the rules and the correlations by hand and to have the data ready to do that back in the early days going back as far as you were going, Bart. So, you had to back the problem up further and further to -- from a DevOps perspective to the left. Beginning with the data, you actually had to structure the data and condition it to be directly consumable by SIEM and that meant you had to adopt specifications like there were some open specs at the time, but the most common was the ArcSight common event format. And everybody had to put their data in that kind of shape to make it usable. And that's not really tenable when you're trying to get input from all different kinds of data sources to normalize in that same kind of format. One of the things that a data pipeline approach brought is that as we started expanding so many different types of telemetry, we have this concept of extended detection and response. So, we really have no idea what the formats are going to come from. Splunk's advantage is that you could leverage search to do that and find look for what you knew was in the data formats and so on. That was really great. And then we have others like, well, who else is good at search? Let me think, oh, yes, Google is not bad it, and then they start investing in this same sort of space. But still, the problem was people had to do that. Now the ambition is that generative AI can identify patterns and structure and begin to service a lot of that for organizations, but they have to have access to the data. That's still the thing that really drives the bus. So without having a data architecture and a data platform from which AI can draw from the sources as it recognizes patterns, as it recognizes, I hate to say, the signatures of threats, but the characteristic of threats that would lead AI to look in different places for data sources and telemetry, it has to have those at its fingertips, if you will, to be able to find those data sources that are relevant according to the data that it's been trained on so it can recognize those threat patterns. Without having a data platform to do that, really, really difficult. And that's really what's been motivating in security for the last year plus. 
 

Eric Hanselman: Well, but we've had some of these stories that at least they were trying to fire on. You've talked about Google and the search-based focus. Chronicle, Google's Chronicle, which was -- the whole idea is, hey, just suck in all the data you've got, and we've got the ability to give you search capabilities, query that data easily. Chronicle came out, what, 5 years ago, something like that. 
 

Scott Crawford: Oh, longer than that, it was birthed out of Google's moonshot, their X initiative, albeit they're not using X quite as avidly as they used to, but that's where it started. And 3 things that they recognize that they had. For security analytics to work, you had to, first of all, take in as much data about everything as you could. So, storage at scale, which was always a big problem for competitors who had to not only store it but put it in shape that it was indexed or indexable and ready to be searched. So that gets to be expensive when you're doubling down on storage like that. They can provide storage at scale because they're a cloud provider. They have search already, obviously. And they also had acquired Virus Total. So, they have indicators of what could look suspicious in that data. put those 3 together and you have a peanut butter cup that should be very attractive. The problem with initial traction, at least anyway, was that there was such an investment that organizations had like for Splunk users, the Splunk processing language, which is an art of its own to know how to best condition your -- it's like creative SQL queries, but in a different domain, domain of search. And people develop a lot of expertise, and that winds up being one of the very sticky things about Splunk. So, moving towards a new direction that would make it amenable to what is now Google Security Analytics, they've since rebranded the offering as they've added more functionality to it. But it's seeing traction. So, this is starting to come around. But again, it's the data, access to the data and how to make best use of it. 
 

Eric Hanselman: And it seems like this is the fundamental part of all of these AI application questions, which is it's all about the data. The data is the raw material from which value is built. And we get right back to that question of you got to get access to the data. Now are we getting to a point at which you have to have all the data in one place because that -- the idea -- because there's so many things, both on the observability and security side, although I think maybe a little more on the security side. We talk about platforms. We talk about having -- you have to have one big thing into which all of this goes so that you can see all of the data. It seems like that's -- we're hopefully starting to maybe move away from the data lake, the data lake house, the one repository to rule them all kind of thing. 
 

Mark Ehr: Yes. I think that's where the data fabric concept and federation are starting to play because people realize that you can't -- it's too expensive and too time-consuming to move all this data in one place. unless you really need it in one place, there's to be a good business justification for it. But if you can federate it and then provide access to it that way through a common point, then you can democratize that data without having to move it around all over the place. 
 

Mike Fratto: It depends on the data we're talking about. Pretty much all the observability vendors want to pull all of the data into a single data lake so that they can process it, normalize it, and then present it outward. With the advent of Agentic, I don't know if you've got a time or Eric on time to Agentic mention. But with the advent of Agentic, and I guess we got to move on from just the mean time to AI to mean time to Agentic. But this was -- I appreciate you hold that off until later on. So that's a good thing. -- but the whole thing with the Gent is it does kind of what Mark was talking about. It really does assist with that federation capability of being able to pull data in from different locations, send rather than being attached to an LLM, can go out and query an LLM when it needs to or multiple LLMs as we're actually starting to see. And so there's a lot of stuff -- there's a lot going on in the infrastructure side, which is enabling not only the use of observability data in a security context and in other context, either in whole or in part to do troubleshooting and to do cause analysis and to figure out what's going on. The other side of this is when there's a security event, a lot of times the question is, okay, what's the impact? What's the blast radius? And if your system of record is going out and doing scans on a daily basis to figure out what's in my IT infrastructure, that's always going to be behind. observability platforms in theory are going to be current because you're collecting all of the data. They know what the moves ads changes are in the infrastructure itself. So, when you want to find out what's the blast radius of a security event, all of a sudden, you have that. And what's interesting is we were talking about like having to learn SPL or Prometheus query language and the observability side of things, which are all very specialized languages. The real -- one of the huge benefits of AI is the ability for anybody to go in and rather than typing a query, just saying, we had a virus, what has it touched? And boom, you can get an answer. You don't have to create a query; you can ask that question. And so, there's just -- there's a lot going on in AI that's not necessarily on the analytics side, just on that UI, that interactive side. 
 

Eric Hanselman: And so much of that is how you actually consume it. But yes, no, go ahead, Scott. 
 

Scott Crawford: I got a question for Mike, though, about -- we talked a little bit earlier about the buyers. And one of the things we hear a lot is that observability and security buyers are 2 very different animals, very different buying motions, very different interests. Historically, you just made a really great explanation of why security people should care about things like traces and metrics because it does surface that real-time operational visibility. We've always been saying, well, security people aren't that interested in traces and metrics. They're more interested in logs and event data. And we've used that to say that that's one of the things that distinguishes the buyers from the security and the observability side. That's going to be a problem trying to sell into both camps with one product is that they're very different buying camps, very different buying motions. But you're just making the case just now for why security people should be aware of how observability is different. Are you seeing that catching on? Or cross-pollinating from observability to security teams and going forward? 
 

Mike Fratto: Yes. I think there's growing awareness among security teams to have a deeper and a richer understanding of what's going on sort of operationally. But I think what's more important is you've got this data set, right, which spans multiple domains. And really, what the potential that, say, Palo Alto has, and other security event management and information management vendors have is to be able to take that information and present it to different roles in ways that are meaningful to them. So, like security people, they may never need to look at a trace because they're not going to get that far down the stack. They just need to know this is what happened. This is where it happened. These are the areas we need to address. After the event is over or even during you got your developers going out there and going, okay, I need to go patch this code, right? So, I'm going to go patch my code. And we're looking at the same data, but we're looking at it through our own lenses. And that's where operationally, you're starting to unlock benefit that right now has been sort of locked up. 
 

Scott Crawford: So, it's not just Splunk and Datadog making incursions into the security market that they have to worry about. There's some real value underneath there that I think security teams are just starting to become aware of or some at least anyway, some are more advanced than others. 
 

Eric Hanselman: Is this something where we've really -- we're finally starting to get into a single source of truth? Yes. 
 

Scott Crawford: The Oracle speaks. 
 

Eric Hanselman: Well, so this is an idea that has sort of kicked around and has been very difficult to achieve. How do you know enough? But Mike, you're identifying what's the big challenge that security teams always face, which is what are the assets that are out there? Well, guess what? The observability side of things knows where all it is because they're all plugged in, and we're getting data from them. So, we know what's alive. We know what's out there. We know what they're what they're actually doing. We can listen to those heartbeats and know that, yes, there's some stuff doing some things. And yet, it's always been that challenge of crossing between those silos between the security and the ops teams. And we've talked about asset databases, config databases and all these other things to try and manage it. And so, are we heading towards a practical implementation of single source of truth? 
 

Mike Fratto: Yes, I think we are. It's sort of the nature of observability platforms is pretty much all of them to build an accurate real-time view of all of the software, hardware infrastructure and crossing that chasm between software and hardware, different products do a better or a worse job. But that's really kind of the idea. And what's being offered is this to enterprises is this idea that you can go from an application and then you can drill all the way down to the hardware and back up and sort of have this expanding contextual view. And wouldn't that be really handy sort of in a security context to know, okay, there's a security event. What are all the systems that are dependent upon this thing and what things depend on this to start to look at what's that blast radius and what's that attack radius. I'm not current on security management systems, but I'm not aware of any -- and Mark and Scott might know better. I'm not aware of any that have that kind of view on application infrastructure, whether it's on-premises, in the cloud or split across that can even begin to deliver that kind of visibility. And that's just one example. I mean -- and there's a bunch of other examples. So, operation, observability platforms can serve multiple roles, and they should serve multiple roles. 
 

Scott Crawford: I think you tipped Palo Alto's hand there a little bit on saying that, Mike, and I think that's very much in view of one of the strategic objectives they have in mind. You also have to consider that one of their primary competitors now going forward is assuming the Wiz acquisition by Google actually closes, most of us believe it will, but it is definitely a long-drawn-out process given all the dynamics involved. But there's going to be some direct visibility into cloud stacks that's meaningful to operational as well as security teams, and they know that. And also, the point that we were making before as far as how much AI depends on a reliable source of data. Single pane of glass, yes, but multiple sources of data from multiple -- we mentioned federated repositories. You don't want to pay to move that data. egress is hell expensive. So, you want to be able to access it where it lives to the extent that you can. It's always a bit of eye-opening to potential customers of a data warehouse that's not a giant bit bucket. It's actually a user experience with that an awful lot of programming going on at the front end of that to make that actually usable. So having that sort of foundational fabric to make that possible is very much in view. And one of the things that Palo Alto also gets with Chronosphere since she mentioned that aspect of operational visibility is remediation, visibility into how to remediate, not just the blast radius, but where and how remediation needs to be applied. Palo Alto has already made a fair amount of noise about having acquired Xpanse, an acquisition that you know well, Eric. for visibility into the attack surface. And this gets into threat and exposure management, not a domain in which Palo Alto had historically been recognized, but they've been making noises about their visibility in that domain and the relationship between threat and exposure management and remediation. So, taking some pretty giant, although somewhat subtle steps towards being a more complete security platform that deals with the proactive as well as the reactive side of the market. And that's a trend I think we'll see more of going forward. 
 

Mark Ehr: Yes. And if I can jump in and build on that. I think one of the things that's happened is if you look back maybe not even 5 years ago, probably less than that, the average time to detect an issue was in months, 270 days at one point was the number that I remember, right? And now we're getting to the point where, okay, let's get to the point where we can actually know it's happening when it's happening in real time, which is great. But now what, right? You're going to need -- especially if you've got AI orchestrating all of this, you need real-time access to that data to be able to say, where are we? What's happening? How do we shut it down now, right, as opposed to, oh, what just happened or what happened 6 months ago, that's not providing value to anyone. But it's going to be a requirement for the modern AI-driven SOC to be able to get access to that. 
 

Eric Hanselman: No, it goes back to Mike's point that you got to be able to actually get into the application stack, understand what the problem is, understand what the various pieces are that it touches in order to be able to fix it effectively. 
 

Scott Crawford: We've just had an incident were remediation. It's like falling down a flight of stairs. You deploy the remediation. So Cloudflare and response to react to Shell exploit. You put the remediation in place -- and oh, the remediation had more of an impact than you anticipated. So yes, a more complete view is really necessary. 
 

Eric Hanselman: Well, it sounds like what organizations really need to start doing as they look forward, start thinking about the fact that this integration from an operational standpoint is going to have to be part of not only the security strategy, but also the operational strategy. 
 

Scott Crawford: Yes, very much so. They have to be joined up better together. And we've known this for a long time. It's just the domains are disjointed; the tools are fragmented. We've had problem enough trying to get the tools just strictly within the security domain to be less fragmented than they are, and we're making progress, but it's uneven. And now trying to rope in the operational aspect of that becomes probably the next most ambitious frontier for that. So yes, it's something we did call out in our 2026 trends in information security. We expect to see more alignment, not just in the technology, but in the practices, bring those more closely together. 
 

Mike Fratto: Yes. And we're not -- I don't think we're talking about a future where we're bridging teams and these specialized roles sort of go away or melded into one. We still need to have these operational teams that they're still effective. But we're just -- we're really talking about ways to remove hurdles and speed bumps in being able to collaborate and work as a team across multiple domains. And that's operationally, I think, the goal. 
 

Eric Hanselman: Just got to get all together, get out the guitar, start singing Kumba, and we're good to go. 
 

Mark Ehr: Just one quick comment. One of the things that I'm seeing a lot in messaging, especially around security for AI is the autonomous SOC. -- right? And I just want to put in a plug for humans, right? Because, Scott, to your point, like the remediation, the cure might be worse than the problem. And the thing is with -- if you have AI, they're probably -- an AI will probably look for the most expedient solution to a problem, which the blast radius of the resolution may not be so great, right? And so that's where a human needs to be able to look and say, maybe I've got 3 different ways that I can solve this, what is the best of these 3 and pick that one, right? And so that's why I think humans in loop are going to be really important. 
 

Scott Crawford: We've always been a little reluctant to enable the mother of all denial-of-service vehicles in response. Every time that's come up, it's always been there, and you could hit the big red button if you really wanted to. It's there if you want it. No one wants to hit it in real-time operations. And now we call that human in the loop for some indefinite period of time going forward until you get enough confidence in those automated processes to say, yes, let them run. That's going to be incremental and it's going to be slow, but certain parts of it will come over time. But you got to have that foundation first without building on a solid foundation of data that's accessible in a reasonable time of response. And that's really job #1 for these data fabrics regardless of who it houses a door, either security, observability, or someone else. 
 

Eric Hanselman: While we respect our robot overloads, they may still need a little bit of time to fully comprehend the full expense of enterprise infrastructure. 
 

Mike Fratto: What may be good enough is just getting to that time to resolution, that time to fix, shrinking that a significant amount may just -- it's probably going to be good enough and development just sort of stops there. We don't need to go to the AI overloads. We just have AI assistant. 
 

Eric Hanselman: Some of my best friends are agents. Well, we'll have to see how this plays out. So much more that we could potentially be discussing on this, but we are at time for this episode. Thank you all for all of these insights, and we'll have to see how this evolves and how we hone this and certainly bringing teams together as we go forward. 
 

Scott Crawford: Always a pleasure, Eric. Thank you so much, and thanks, guys. We got to talk more often. 
 

Mark Ehr: Yes, it's been fun. 
 

Eric Hanselman: And that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun, and Kyra Smith on the marketing and events teams. I hope you'll join us for our next episode where we're going to get the gang together to talk about everything that happened in Las Vegas at the AWS reinvent. What stays in Vegas, or what happened in Vegas is not going to stay in Vegas, it is actually going to turn up on the podcast turn and hope you'll listen then because there is always something Next in Tech.