Next in Tech | Ep. 282: Black Hat and DEF CON Security Conferences

View Full Transcript

[00:00:01] Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for industry research at S&P Global. And today we're going to be talking about the Black Hat and DEF CON conferences that just took place.

Welcome17] Eric Hanselman: And joining me to talk about them are Dan Kennedy and Justin Lam. Welcome back to the podcast you both.

[00:00:24] Dan Kennedy: Thanks, Eric. It's nice to be back

[00:00:25] Justin Lam: Great to be back. Great to be in the air conditioning and back. Yes.

[00:00:29] Eric Hanselman: I was going to say, you survived the heat of Las Vegas in a roasting in August. I saw some of the weather reports. It sounded like it was pretty gosh darn warm out.

[00:00:39] Justin Lam: It was.

[00:00:40] Dan Kennedy: It was a dry heat, 115.

[00:00:43] Eric Hanselman: I'm thinking even the dry heat, 115 is getting a little toasty out there. But the conference of course was predominantly spent indoors. It would be great to get your takes on what did you see and what were general themes? I'm guessing this little AI thing probably focused strongly in some of the discussions that were taking place, but, what’s your take broadly?

[00:01:06] Dan Kennedy: It was interesting. You know, I managed to, for analysts and you know this, we get off the plane, and our first vendor meeting happens like within an hour. And then we have our last meeting and we just get on another plane.

[00:01:18] Eric Hanselman: Ah, so you skipped the vendor meeting in the taxi, yeah?

[00:01:21] Dan Kennedy: I did skip that.

[00:01:22] Dan Kennedy: Yeah, I had a nice taxi drive to gather my thoughts. I did manage to get to the keynote. It was interesting. Jeff Moss, in prior years he's had the super enabled human theme, and last year it was AI's prediction machine. This year I didn't really get the sense of that unifying theme.

[00:01:36] Dan Kennedy: It was more just like AI, whoa. Which is a little bit where we're at. Announced it was the 29th Black Hat, and I thought, "Wow, I've been to a lot of these things." And I guess there were 23,000 people that showed up, which is significant growth over last year.

[00:01:52] Eric Hanselman: Wow. Yeah

[00:01:53] Dan Kennedy: I felt like I was in line for a long time getting my badge.

[00:01:56] Dan Kennedy: So, it's gotten even bigger than it used to be. But yeah, you said it, AI, right? OptNova was the sort of startup competition winner. Autonomous identity governance enabled by what AI is able to learn about, what access rights you need, right along. And it's not just identity, it's AppSec, it’s security operations, it's data security.

[00:02:17] Dan Kennedy: So, across the board I mentioned the keynotes. They were interesting. They were sort of initially presented as like a spy versus spy thing on basically how AI is enabling vulnerability detection. But it ended up being both speakers agreeing with each other with some nuanced differences.

[00:02:33] Dan Kennedy: So, one was David Weston, who's agentic research at Microsoft. He noted that they're processing nine times their vulnerability volume they saw since March, so it doubles every six weeks. And so, it's like an unsustainable pace. And overall theme, if I had to summarize it, is the idea that we're going to move to machine speed response is unrealistic.

[00:02:56] Dan Kennedy: We're not going to be able to patch faster, detect faster, and so we need to move towards more of a prevention approach enabled by AI. Moving towards application hardening, secure by default languages, identifying formal verification methods, but fixing the apps themselves. The second presentation, Jan, was from the perspective of a professor, an outside researcher. Same idea that they're finding, you're using agentic techniques to find vulnerabilities at a rate that they were not able to do before. He sort of went after the just rewrite it in Rust, memory safe language thing.

[00:03:36] Dan Kennedy: He noted you could correct a lot of the memory safe bugs, but that just exposed a lot of logic level flaws. Race conditions, cryptographic issues, things like that. The language level rewrite removed one bug class, which then revealed all the design level stuff below that. So, it was an interesting discussion.

[00:03:54] Eric Hanselman: And it's an interesting point, which is I think so much of the focus has been around, oh, vulnerabilities are coming at us fast and furious that are fundamental coding flaw kinds of problems but if you've got bad app design as you're pointing out, bad logic errors, race conditions, whatever else happens to be in there those are just as big an issue and a little more complicated to fix.

[00:04:22] Dan Kennedy: I would agree with that, and I would agree that we're moving from this. Like, the big question for me was, yeah, AI is the central theme of the conference, but what makes 2026 different than 2025? And it's just the level at which we're discussing it. AI hasn't made attacks faster. It has, but that's not the end of the discussion.

[00:04:41] Dan Kennedy: It's affecting the entire pipeline really, from discovery to remediation to exploitability. That was a big point. It can find the vulnerability, but then it can suddenly give you the context and environment around that vulnerability to give you an exploit. And so like, as an old AppSec person, always the question from developers was, "Yeah, you found this vulnerability. Is it exploitable? Do I really have to fix it?" And some of those decision-making criteria, AI is enabling it so fast that it, it's changing the nature of the discussion.

[00:05:12] Eric Hanselman: Those things that, that maybe might have been a challenge back in the day of, you have to pat your head, rub your stomach, let the dog out the back door rather than the front door. It now, of course, to your point, AI being able to create those conditions, more easily, starts to become a bit more of a challenge.

[00:05:30] Dan Kennedy: It's interesting. It's almost identifying where we're still talking about fundamentals, but like AI can expose those weaknesses much, much faster and much more comprehensively. And so, we're still having the same discussion about these hygiene controls, but it's a different discussion when AI can exploit the machine speed.

[00:05:48] Justin Lam: But I think also there's big opportunities for anyone who's been in the hygiene business to actually be more proactive about finding what those deficiencies are. I think that, you hear about vulnerability management and being able to exploit, or find new exploits and what have you, but how about actually just being able to enumerate what actually should be a least privileged model for all the dozens of folders that I have in, a document sharing repository like, a SharePoint or a OneDrive? These are, I think, things that are really interesting that AI could bring to the forefront to help teams out. I think being able to simulate even, use AI to actually simulate, if I make a policy change in hygiene, will it be overly restrictive or not restrictive enough? I think these are some very interesting things that, you know, that certainly I heard about at Black Hat that might point to that direction.

[00:06:48] Justin Lam: And ultimately being able to do deeper analysis as to not only what have identities, endpoints, network points, servers, workloads, how have they been provisioned, what's their providence, but actually what's their activity? What's their trustability over time? What’s their intent and their purpose? I think that AI could really help unlock what these things are, building off of the hygiene controls that are actually in place.

[00:07:16] Eric Hanselman: It's the scope and scale story on one side, right? In that it, it's got this ... it can digest this much wider lens. But I think the other piece of that simulation, of okay I'm going to simulate what this policy is, what's it going to break? It's that reduction of risk for a lot of the things.

[00:07:34] Eric Hanselman: Because if you look at what's always the challenge for zero trust, right? It's the, I don't want to lock this down because it might break this and it's going to break this build, and maybe, this other team won't be able to get access to that. And to reduce some of that, the fear that comes into this in terms of some of the way this gets presented.

[00:07:51] Eric Hanselman: So yeah.

[00:07:52] Dan Kennedy: Well, and we're adding this, this autonomous software entity that gets to access, gets access like a high-privileged human into the equation. So, the question is, how do you operate zero trust at machine speed? The agent can acquire identities, credentials, secrets, API access, SaaS access.

[00:08:12] Dan Kennedy: So how do we sort of see what it's doing, introduce guardrails to let it know what it can and can't do? I mean, jump into the test environment escapes of the last month. We were working with X point Jim and I guess it's both OpenAI and Anthropic at this point, both had incidents. So, the question becomes for an enterprise, we're sort of forced into these positions.

[00:08:33] Dan Kennedy: We can't stay out of agentic, so how do we actually do this? And I see that across the board. It's there, it's security operations where it's, okay, attackers are operating somewhat autonomously. We have to too. Is that human in the loop? Is that human on the loop? Where's the human involved?

[00:08:51] Dan Kennedy: And we've moved so quickly from AI can help me triage an alert to, okay, how much autonomy am I going to hand SOC operations? How close to source can it start to take remediation actions and which actions do I trust? And the products are in all different places. Enterprises implementing the products are in all different places, yet the external factors forcing this situation are consistent and becoming more sophisticated.

[00:09:18] Eric Hanselman: Yeah, we've got the forces in play are pretty consistent, and yet now we've got a set of potential controls and capabilities that may be able to allow us to wrap this in more detail or at least to be able to put controls in place that maybe go a bit beyond what we've historically been able to assemble.

[00:09:36] Eric Hanselman: So, what's your take about what this trade-off happens to be? In the keynotes, you've got this sort of back and forth of, are we coming at this with greater levels of attacker tools? Are we actually going to be able to bring to bear sufficient levels of remediation?

[00:09:51] Eric Hanselman: Sounds like the reality is that the remediation part is, it still winds up hitting the same kind of stumbling blocks we've always had. I don't know. Is that something where we start to be able to overcome some of that? a lot of that get, gets to a point at which we, we've got to trust some of those actions, and it gets right back to the whole automation problem we've always had, right?

[00:10:11] Eric Hanselman: Which is how do we get to levels of trust in which we trust the automated activity? And that, that seems like that, that gets a little more complex in AI. a little more of an issue as we start dabbling into some of the agentic remediation pieces that we're working with.

[00:10:29] Eric Hanselman: It gets back to, Dan, the point that you were making. How do we bound what the whole set of actions are that an agent can take into an area that, that we know, that we can sufficiently trust, in a point at which we actually are finding that we're actually getting pretty bad at bounding agent activity.

[00:10:46] Eric Hanselman: All we got to do is talk to the people at Hugging Face, right?

[00:10:48] Eric Hanselman: So, I guess it gets back to that question of are we in a situation in which we can let the automation loose and be able to really trust it sufficiently to be able to get this to really, to be able to get implementation levels that make sense?

[00:11:03] Dan Kennedy: I just wonder if we're just, we're in the early stages of, entire paradigm changes based on this stuff. We had a great discussion, Justin and I, with one vendor about managed services, managed security services. I think that area is going to change tremendously over the next two or three years.

[00:11:15] Dan Kennedy: Beause we, we have this SecOp model. You collect telemetry, you send it to some sort of processing place. I'm going to call it a SIEM, but, call it whatever you want, a data lake, whatever, some central place. You correlate it, you generate an alert, an analyst looks at it, analyst does something, analyst responds.

[00:11:32] Dan Kennedy: We're, like, layering AI into that process. Enrich the alert, correlate other content, put it in front of that human, they decide. And I have to question whether attackers are just doing whatever. They're always asymmetric. They're always minimum viable product, so they're going to just throw everything at the wall and let it rip.

[00:11:51] Dan Kennedy: And that was the point the keynote was making, that your response measures are going to fail in response to that. And so, okay, what can we automate? As you said, the classic problem of automation, what do we trust? And our automatic answer is, human in the loop. Is it? Is it human on the loop?

[00:12:07] Dan Kennedy: Just guard, see the overall activity and determine whether you think it's okay or not, do exception management. And then further questions: Do we have to get a lot closer to where the incident is occurring? Does the centralized SOC model even work anymore, or do we have to take some percentage of what's coming in and deal with it, at a point of presence, right where it's happening, and just immediately isolate the attacker in an automated fashion close to where it's happening from a physics standpoint?

[00:12:34] Dan Kennedy: So, these are esoteric ideas, but I think the difference between '25 and '26 is this realization that the AI is not just going to be an assist. It’s somewhat fundamentally changing the way we're going to behave, and what further needs to change to accommodate that. And so, Justin brought up the identity piece.

[00:12:52] Dan Kennedy: I'm talking about the SecOps piece. A lot of the AppSec piece, you mentioned it, there's success stories. There is, "Hey, listen, I rewrote this entire application in a new language in a fraction of the time that would historically take because I can generate code much faster, and because it had a model to work off of."

[00:13:10] Dan Kennedy: But there's a ton of caveats to that. What context did I give the AI in prompting refactoring of code to get deterministic results out of a non-deterministic process? And how do I not create more technical debt or problems or things I can't maintain? So every area of security, no, like I said, it was a packed floor, is affected by these changes and people are reconciling with them right now, and it's interesting to watch.

[00:13:35] Justin Lam: Yeah, just to build off of what Dan is saying, the underlying economics absolutely fascinate me. When you are thinking about things being more agentic, either in detection, remediation, hygiene, what have you, there gets to be a fundamental change in the value proposition, especially among vendors, where I have typically licensed based on a per-user basis or per some other form of capacity.

[00:14:03] Justin Lam: And now you are seeing in other sort of AI-native startups this idea of service as software. And so rather than licensing based on even necessarily a piece of compute, capacity, you're licensing based on what was the outcome. Was there any satisfaction in the result of that outcome? And I think that brings a lot of different economics to it.

[00:14:27] Justin Lam: And then in the same vein, you have so many of the vendors who are AI-driven or AI-powered, tokenomics, cogs, all these kinds of things will be a very different set of parameters for company growth, for how these solutions will scale, and whether or not security layers will be able to pass on those token costs to their downstream customers.

[00:14:52] Justin Lam: I think that just remains to be seen.

[00:14:54] Eric Hanselman: Yeah, we haven't even begun to talk about the cost of this. It does seem like in a number of quarters, the cost of AI is finally coming home to roost. There was that initial, "Hey, let's throw models at it, and it doesn't matter if it's a, billion parameter model or whatever."

[00:15:09] Eric Hanselman: And now suddenly moment of, "Oh yeah, this actually, this costs something."

[00:15:16] Dan Kennedy: Well, and that's opening these crazy economic discussions, which are, which model gives me the right ROI for this activity? And these conversations are just, they're nascent and they're complicated, and people have a sense they should work. Like, well, wait a second, if I can do effective vulnerability discovery at this model that's cheaper, that's this much per request, why am I using this one?

[00:15:38] Eric Hanselman: And it just gets into all these parameters. Never mind the, hey, wait a second, the locally installed model, yeah, it's not as good, but I don't necessarily need to use the frontier model. It's giving me enough ROI for this activity, and I can run it locally and absorb that cost my own hardware, yeah.

[00:15:55] Dan Kennedy: And the example from Hugging Face, it doesn't include restrictions that the people attacking me don't have in forensically recreating what's happening.

[00:16:05] Dan Kennedy: So, there's so many interesting effects here, and God, it just reminds me of the cloud pricing services discussion, but so much more complicated. And the token economics piece, it came up, like it did for Justin, multiple times. And I feel bad because as an analyst, I'm just sitting there describing the water.

[00:16:24] Dan Kennedy: Yeah, no, you think it's complicated. Let me tell you how complicated it is.

[00:16:29] Eric Hanselman: If you didn't think it was complicated before, boy, can we make it more complicated for you now. Hey, that's our value as analysts, right?

[00:16:39] Eric Hanselman: Justin, I, I was going to say, before we get too far down the road, you were also at DEF CON. For those that aren't familiar, people sort of think about Black Hat as the, the one big thing. Those 23,000 people who are turning up for Black Hat, a small fraction of them wind up, heading to DEF CON afterwards. Actually, that's my presumption. How many people turned up at DEF CON? And what'd you see?

[00:17:01] Justin Lam: No, I think somewhere around, I want to say somewhere around 27-28,000 people showed up at DEF CON. I think that one of the things that we had been watching a little bit earlier was, from last year was, obviously some big changes. Not least of which, federal involvement was a little bit more demure this year than last year.

[00:17:23] Justin Lam: Paul Nakasone, the former director of the NSA, yeah, they did make an appearance, but you have other things like the AI Cyber Challenge brought about by CISA that didn't exist this year. And so, you had a lot of other things fall into place, in its place that were more organic, keeping with the spirit of DEF CON. I was encouraged by the theme of DEF CON was this idea of agency, was that one of the things that's just changed over the course of years with DEF CON is that it's actually much more of an inclusive community today than I would say a few years ago. I would just say that the idea of, you know, the thought of something like a noob village, first time visitors to DEF CON, would you ever welcome them, create a village for the first time DEF CON experience? That was antithesis to say, "Welcome to the cool kids," right? That was the last thing you would want to do.

[00:18:18] Justin Lam: But I think that between this, between DEF CON Next Gen being able to make it more family friendly even, some villages even having challenges for kids, it just opens up the community. And to the point about the agency, I think everyone realizes that AI is out there. I don't necessarily think that in the short term or even the intermediate term it replaces personnel, security practitioners, hackers or whatever.

[00:18:44] Justin Lam: It just becomes another tool to help them achieve what they need to achieve. I think that was one of the key themes. Even with all the CTFs, all the challenges being modified to accommodate AI, you still see this individual, and you still see people trying to learn as much as they can, communicate and connect as much as they can so that they can go leverage whatever's changing in the technology and let them, let them succeed in whatever they're trying to do

[00:19:13] Eric Hanselman: I'm interested, you mentioned changes to the capture the flags. They were adapting the capture of the flag exercises to account for AI?

[00:19:23] Justin Lam: Yeah. I mean, there's some categories that say, "Hey, you can use AI." Others would design a challenge where you can't use AI, or there's certain rule sets where I can and can't. There are some CTF designers who actually purposely made their challenges so that you couldn't use AI, you were at a disadvantage from doing so.

[00:19:45] Justin Lam: So, there's different things, and there's all kinds of different ways to be able to make that still attractive for people to participate in.

[00:19:50] Eric Hanselman: Well, uh, great to see that it's expanding the community. I honestly, I'm surprised at the numbers for DEF CON. Yeah, because it's always been the Black Hat's younger sibling, smaller event, little lower key.

[00:20:04] Justin Lam: What I would also say is, and I don't know if there's any self-selecting bias here, but there is believe it or not, there's actually more sponsorship going on for each of the individual villages. So, DEF CON is organized in about 40 different villages, various topics around the security thereof.

[00:20:20] Justin Lam: So, aviation security, IoT security, control system security, payments, you name it, cloud, what have you. And you just saw more and more, because these things cost and money, individual passes for DEF CON, people pay out of their pockets. It's not subsidized like it is in other trade shows.

[00:20:41] Justin Lam: But still, to put on villages, to put on this within the convention center, it still costs money. And so, there's still, more and more of the vendors underwriting some of the talks, some of the villages in there. And I think it's a healthy balance right now, for community and industry coming together.

[00:20:57] Eric Hanselman: Wow. Oh, that's good to hear. And again, great progress to be able to get it to that point as well.

[00:21:03] Dan Kennedy: The are humans are huddling together in the face of AI.

[00:21:08] Eric Hanselman: Maybe we get to a point where we're forced to actually gather around the fires and figure out how to hold off the coming AI demons.

[00:21:15] Justin Lam: For sure. For sure

[00:21:17] Eric Hanselman: We'll see. Any final thoughts just to wrap up, what your takeaways are from, for both of these events?

[00:21:27] Dan Kennedy: For me, it's just this underlying tension that we talked about. The industry needs to automate security using AI, because humans can't operate at machine speed. The threats are coming or vulnerabilities are coming at machine speed, and yet the amount of discomfort that raises is incredible.

[00:21:46] Dan Kennedy: And so, the entire discussion and floor is about all the identity, governance, observability, control issues, focus on resiliency. Can we limit blast radius? How fast can we recover? And so, it's interesting. There's an acknowledgement that the human-centric workflows are becoming bottlenecks, but we're nowhere near prepared for what that means.

[00:22:11] Dan Kennedy: And I think the security industry has challenges, but as Justin correctly framed it, opportunities to sort of address these deficiencies. And then there's a great deal of opportunity around some of these problems.

[00:22:26] Justin Lam: I think that Black Hat and Def Con and just all the other cons, the Holly con, and frankly all the subsequent follow-on conversation are good things, for the way people are learning about technology, the way that they are convening. Yes, we hope that they learn things on this podcast as well, but also the amount of informal that goes on after the show on a more repeated basis.

[00:22:54] Justin Lam: I think this continuous communication, continuous education needs to happen given just how fast the ecosystem is changing. And also frankly, to allow ourself room to breathe so that the next time something hits the headlines tomorrow, we can take it with a better context than just simply reacting to it, as if the sky is falling.

[00:23:15] Justin Lam: I think there's some balance in there and community and that constant communication, they enable those kinds of things.

[00:23:22] Eric Hanselman: I think that's a great point to close on because I think the security community has always had, a, a pretty strong focus on community and the strength that offers. But especially in light of all of the chaos that's being generated by AI and agentic, this is a time to refocus and to look at that, that, the core of that community and what value it provides and to, to reinforce that centering power that is that and something to build on.

[00:23:55] Eric Hanselman: So, a good point to close us out on, thank you both. This has been great. There's so many more things to talk about, but as usual, we are at time. But thank you both for being back on.

[00:24:05] Justin Lam: Thank you, Eric

[00:24:07] Justin Lam: Thank you, Eric

[00:24:08] Eric Hanselman: And that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun, and Dylan Scheible on the marketing and events teams. If you like this program, please favorite, or subscribe to us. It keeps us informed about what you like and where you want us to head.

[00:24:27] Eric Hanselman: I hope you'll join us for our next episode, because there is always something next in tech.