Next in Tech | Ep. 280: Claw and Agentic Optimization

View Full Transcript

[00:00:02] Eric Hanselman: Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, chief analyst for industry research at S&P Global. And today we're going to be looking at the world of claws, OpenClaw, claw skills, and all of the things that are happening with this interesting end of the agentic realm.

[00:00:25] Eric Hanselman: And with me to discuss it are analysts Jean Atelsek and Melissa Incera. Welcome back to the podcast, you both.

[00:00:33] Jean Atelsek: Thank you

[00:00:35] Eric Hanselman: The Claw world has, I don't know, scuttled into the world by storm. We can use, I don't know, whatever analogies work around this, but this has been something that has been, I think you can identify as pretty impressive impact when you have one particular technology that is causing all of the IMAX on the planet to be scooped up as, vehicles for powering agents.

[00:01:02] Eric Hanselman: This is a different kind of piece of AI and agentic interaction, transition, transformation than other things we've seen. But I guess before we get too far down the road, maybe we should dig into a little bit about what it is, get our audience a little more up to speed in terms of what we're actually dealing with when we start talking about all the various Claw iterations that are out there.

[00:01:27] Jean Atelsek: Yeah. So, um, I took a good look at claws because they were such a phenomenon, especially at the beginning of the year. It was well-documented that it was the fastest adopted, or GitHub starred, project by far in GitHub history. And basically, claws are local first, so they're on your personal machine agents. You know the whole thing about agents is you have to give them agency, right? So, you are giving them leave to do things on your behalf. And so, these agents, by default, or OpenClaw in particular by default, had access to everything on your computer, including your bank account, your file system, your documents and so on.

[00:02:17] Jean Atelsek: That made them actually very powerful. But obviously from a security point of view, there were lots of vulnerabilities. And sometimes agents... agents often don't have good judgment. One of the funnier things that I heard happen, during an experiment with OpenClaw, was that somebody spent two hundred dollars in tokens buying a box of paperclips. So that, that's the sort of thing that was happening.

[00:02:48] Eric Hanselman: Well, the agent's going to agent, right?

[00:02:51] Jean Atelsek: Exactly. It’s unintended consequences. But the whole concept of claws and having that access, it really pushed the limit of what people understood about what agents could do and how powerful they could be.

[00:03:09] Jean Atelsek: Now at this point, most companies have banned OpenClaw in particular just because of its design. But, the whole popularity and the phenomenon, spawned a bunch of alternatives, some of them focusing on being very lightweight. NVIDIA launched its own version that's very integrated into its stack.

[00:03:29] Jean Atelsek: So, there are advantages to it that are undeniable. Now, another thing they're LLM agnostic, it gives you access to the whole AI ecosystem, in terms of reasoning, without having to change how your claw is set up. A consequence of a Claws was that skills, which was a standard developed by Anthropic, actually, I believe at the end of last year as a replacement for its agents MD file, are granular capabilities for agents to have that are loaded progressively into the context window of the LLM in a way that doesn't cause bloat in the context window, doesn't consume a lot of tokens.

[00:04:19] Jean Atelsek: And skills began to take off quite a bit. And now that is one of the chief ways that people are interacting with LLMs or agents are interacting with LLMs rather.

[00:04:31] Eric Hanselman: So, you've now got the ability to expand the fundamental capabilities with markdown language-based extensions through skills and the ability to be able to now do more. A great pathway, but I guess the, the question is where do we go from an implementation standpoint, and what kind of things are people tackling?

[00:04:51] Melissa Incera: Jean outlined how architecturally OpenClaw helped to pioneer a bit of a different take on the agent paradigm. So, she mentioned it was local first architecture. We also saw the separation of the interface and the runtime environment, so that rather than navigating to an OpenClaw app, users were actually interfacing with various apps they already use, like WhatsApp or Slack, while the runtime was really just sitting quietly in the background.

[00:05:16] Melissa Incera: And we also really started to see the concept of computer use, whereby these agents are actually navigating around a visual environment like you or I would to get to different apps and to even handle a browser, which is really cool technology. But one thing I'll harp on, as Jean mentioned, the OpenClaw framework really decoupled the concept of an agent's capability from, A, the interface, but also importantly from the model itself.

[00:05:42] Melissa Incera: So, we are seeing the ecosystem shift in focus even further away from the model, instead to this paradigm called the agent harness, which is actually the software infrastructure that surrounds the LLM, and in essence, it enables all of the fiddly little logistical bits that allow the agent to do something or really exist as an entity as we would perceive it.

[00:06:04] Melissa Incera: So, for instance, context and memory. We know memory is the number one capability users expect an agent to have. The agent harness is what would define how the conversation history is managed or how state is maintained, or even in an enterprise context, really important, the different guardrails and constraints.

[00:06:26] Melissa Incera: So, we're seeing a lot of vendors shifting to this paradigm where it's essentially almost like a prepackaged agent entity, and you can plug in the models of your choice, which is incredibly important. You want that flexibility and dynamicism in the models you're interacting with. And OpenClaw and Claw Code are two of the most prominent examples of agent harnesses right now.

[00:06:47] Melissa Incera: So architecturally, it really helped drive a shift in how the industry thinks about the agent from a packaged perspective

[00:06:56] Eric Hanselman: I think that's one of the challenges, and I guess also one of the opportunities. If you think about that integrated capability that we've seen with OpenClaw and Claude and a lot of the agent packages, especially the ability now that we've seen, to be able to run that locally so that users can manage some of that cost, it gets us to an interesting place where the agent functionality starts to become a much bigger part of some of what really is being undertaken here.

[00:07:28] Eric Hanselman: But it does, I think to your point, require that level of integration, and we still have to deal with concerns about what the cost is to be able to manage any of these.

[00:07:41] Jean Atelsek: Yeah, I think that's the key, like a background for all this is the cost and the consumption of tokens. Now, agents at the beginning of 2026, all of the frontier model vendors had to change their pricing models because these are always on, components and they would, as some developers would, be consuming tokens and so the all you can eat pricing plans had to go away because the model vendors were subsidizing usage by some of these very token hungry applications.

[00:08:17] Eric Hanselman: Oh, we-we've seen that before with the shrimp buffet, right? We know how that story ends. Although I guess AI providers are getting ahead of that.

[00:08:26] Jean Atelsek: Yeah, and so it just goes to show, what Melissa's referring to, shows the reaction to having that constraint now in terms of quotas put on engineers so that they have to think of and devise and normalize and come up with ways to use fewer tokens to get the job done.

[00:08:47] Jean Atelsek: And this is something that we're just starting to see, the whole concept of tokenomics and token usage really being highlighted, especially because if you're not working for NVIDIA or Google or, OpenAI,I mean, you have to pay retail to use their models.

[00:09:07] Jean Atelsek: On the other hand, a lot of the models, especially coming out of China, are showing themselves to be very capable and costing much less. And anecdotally, I've heard of people building applications on the commercial models, and then deploying them on open-source models. So open weight models, yeah.

[00:09:30] Eric Hanselman: Well, I think that's, again, we're looking at cost optimization. So, the big models are expensive to run, but yet are producing quality results. Now it's something where I think we've got people looking to be able to go balance some of that with some of the cost allocations.

[00:09:47] Melissa Incera: Yeah, and I'll just add on what you were saying, Jean. We talk to companies all the time where their budget, their AI, used to be they were talking like tens, hundreds of dollars per month, and it was a pretty negligible line item. And now they're talking about thousands of dollars per month per software developer.

[00:10:04] Melissa Incera: So, this is like orders of magnitude more, this cost piece of the equation. And I would also add, so you're talking about tokenomics, of course, tokens becoming probably the core primitive of AI economics in the AI world. But I would also say it's not just tokens, because now that we're talking about agents, these are incredibly complex systems.

[00:10:26] Melissa Incera: So alongside the raw intelligence, we're also seeing costs grow, beyond the models themselves. So, for instance, I was just talking about memory, long-term memory. In order to maintain that, you have to have indexing capabilities. You have to have retrieval infrastructure for an agent to actually take actions, you have to call APIs, you have to manage cross-aging workflows.

[00:10:48] Melissa Incera: I would expect that the gap between just raw token consumption and actual computational demand is going to widen as these systems grow more sophisticated and autonomous and they're doing long horizon tasks, and we have multi-agent architectures. It’s a really important problem to solve alongside, I mean you already mentioned the security thing, but security and cost are the two dominant drivers of decision-making right now we see across our data as it relates to AI and agents.

[00:11:15] Eric Hanselman: Well, the security piece, of course, is that aspect that's near and dear to my heart. But it comes into that category of not only the agent security, the action security. Of course, we've seen what happens when there aren't good bounds or where security capabilities may not be withholding some of those activities. But I'm curious about your takes on, what as the primary challenges that enterprises are looking to go address.

[00:11:41] Jean Atelsek: I'd say one of the biggest challenges is understanding the use cases, for a lot of enterprises anyway. Understanding the use cases for agents and where you're going to have sustainable return on the investment that you're making in developing and running them. And the reason I say that is because, we know that AI coding agents are really the best sort of the poster child for AI efficiency and, you know, investment.

[00:12:14] Jean Atelsek: But what we're seeing is that some of those mechanisms are pushing problems a little bit right. Everyone talked about shifting left. Some of the automatic coding assistants are pushing problems to the right and creating difficulties that need to be handled by platform teams and AI engineers.

[00:12:35] Jean Atelsek: In other words, there's a failure mode that needs to be handled. So, I think one thing is, and this has to do with skills as well, the way that an agent decides if a skill should be invoked is based on a very limited amount of information that they have at the top of the file.

[00:12:53] Jean Atelsek: And so I think it's important to reach a level where, you know... I know the promise of having thousands of bots going around and doing stuff. The fact is that agents don't have good judgment unless you give it to them, and you have to decide the use cases where that is going to bring you value that's worth the cost and without compromising the systems that you've built up and have to maintain.

[00:13:23] Eric Hanselman: And we can talk about good judgment with the people at Hugging Face. Yeah, they might have a thought there. We’ll see.

[00:13:29] Melissa Incera: This exact conversation is actually pushing my research in a super, super interesting direction all around this topic of optimizing agents. And it's optimizing agents along several different dimensions because these systems are so complex, right? So, you could be optimizing for performance, you could be optimizing for cost, you could be optimizing for speed. But essentially, it's such an interesting space because we're so early stage. These systems are so rudimentary compared to what they will be.

[00:13:59] Melissa Incera: If we're well-behaved, maybe Eric will invite us back in a year.

[00:14:06] Eric Hanselman: I'll invite your agents back. That's what I'll wind up doing.

[00:14:09] Melissa Incera: It's true! So, I think if we were going to look back a year from now, we'll see that agents today are producing significant levels of what you might term as inferencing waste, right? So, they're going through repeated planning cycles, or they have way too many verification processes and it's slowing things down and it's making things expensive.

[00:14:28] Melissa Incera: And this is driving some really interesting research and mechanisms to govern how resources are allocated within these systems, these agentic systems. And the four main cost centers within these agentic systems are the raw intelligence, the context layer, the coordination, so like how are the agents interacting with one another, speaking with one another, and then the execution.

[00:14:53] Melissa Incera: So, is the agent reaching into a database? Is it reading to it, writing to it, etc. And all of this is to make these agents operating at scale economically feasible. I believe you and Melanie talked about model routing, Jean, is that accurate? In the FinOps episode, which was fantastic, and everybody should go listen to it. But, intelligence allocation, super interesting area of research because it's becoming one of the most powerful levers for controlling for computation. So, this should enable, in theory, enterprises to use smaller, more configured models for simple tasks like summarization and larger more performant models for running a multi-day process.

[00:15:32] Melissa Incera: But also, intelligence allocation can be much more complicated than that. You can play with how deep a model is allowed to think or how does it go about solving a problem? Does it scale its thinking vertically, meaning you give more tokens to one agent, or you allow it to spawn three or four agents with smaller token budgets but try to optimize through each of them taking a different approach.

[00:15:55] Melissa Incera: And then there's a lot of interest in looking at memory systems and compressing content, text, or even manipulating when in the thinking process an agent might look to retrieve context. These systems are so complex and there's so many things to play with to make them more efficient, to make them faster. And it's so interesting because we are so early days in how these systems will actually be running in the long run.

[00:16:25] Jean Atelsek: Yeah. One thing I'd like to add is that I find it interesting that, it seems like the bloom is off the rose for probabilistic models, not for the models themselves, but reinforcement learning, like good old reinforcement learning that's been around forever is becoming so important for doing the evaluations and the experimentation and also for putting those guardrails, around the agents, because you don't want to let them run wild.

[00:16:52] Jean Atelsek: And yes, they can have context, but you need to give them the right context.

[00:16:57] Eric Hanselman: Context management. Yeah, there's so much that has to be built.

[00:17:01] Jean Atelsek: Yeah, I just find it interesting that reinforcement learning is resurgent as an AI technology, as a way of taming large language models and gen AI.

[00:17:13] Melissa Incera: Our data tracks this really closely, generative AI has been really leading adoption of AI broadly, but it's pulling along more classical forms of AI because they're cheaper and often, they're better suited for some tasks. Yeah, I totally agree with that, Jean.

[00:17:29] Eric Hanselman: And billion parameter models are not needed for every single task in the enterprise. There is a lot that can be tackled, and a lot of it has to be right-sized to work well.

[00:17:39] Eric Hanselman: What, where should we step to next? Because the thing that's hovering out there is maybe talking about a control layer. So, I was thinking that maybe the control layer piece, so maybe I'll just ask the question.

[00:17:51] Eric Hanselman: So how do we actually manage this, and what do we have to do? You know, it's great that everything old is now new again. We've got generative approaches that are back in place, but we're getting into areas in which we've now got to start thinking differently. So, it's interesting that, I guess basically everything old is now new again. We've got so much of the old paradigms that we've done historically. We've got GenAI capabilities coming back into use because of its efficiency. We need to start thinking about architectural efficiency because think about back in the database days, right?

[00:18:28] Eric Hanselman: We had to figure out exactly how many database queries and who was querying what. Seems like we're right back into that. How do we start to really manage this? And do we have to make architectural shifts? Or are there other pieces that have to be in place to help us manage this and really get our arms around the control side of this?

[00:18:49] Melissa Incera: Yeah, there are a few things to discuss here. Big question, Eric.

[00:18:54] Eric Hanselman: I don't ask small questions. I leave small questions to my bot, you know? The big questions go to the analysts!

[00:19:00] Melissa Incera: I actually want to take us back to what Jean was talking about as it related to some of the adaptations of OpenClaw that spun up for the enterprise. So, I think you mentioned NemoClaw.

[00:19:12] Melissa Incera: So just to refresh, NVIDIA, a few months ago, realizing there was real appetite for this, literally took the OpenClaw concept and added control mechanisms both at the harness level and at the infrastructure level and one of the big focuses of NemoClaw was containment.

[00:19:27] Melissa Incera: So, this is a big push I'm seeing as I talk to vendors at the orchestration and runtime layer. Instead of running directly on a user's machine with full access, these agents, NemoClaw specifically, run inside secure sandboxed environments with serious restrictions on what they can access and what tools they can call. So, they really can't break anything substantial is the idea.

[00:19:51] Eric Hanselman: Kind of one of those things that we went through with the shift to containerization. Initially, lots of containers ran with root privileges and wow, you could do whatever you wanted, and everybody realized, oh, wow, letting things do whatever they want isn't all that great. And so it seems like that, that's where we're getting to with the agentic world as well

[00:20:11] Melissa Incera: Definitely. But I think there's a broader push across the entire industry, and there's truly not a single technology vendor that's pointing at the agent ecosystem not talking about some variant of agent control, agent safety, agent governance or agent observability, because it's a real hurdle that has to be solved.

[00:20:29] Melissa Incera: And, often they're taking slightly different angles, right? Some of them are solving for controlling access and identity, some solving for output safety, PII leakage, some of them looking specifically at the cost, control piece. But you have this powerful, very unpredictable resource. How do you make it, how do you make it controllable, right?

[00:20:50] Melissa Incera: So, we're seeing this concept of the AI control plane emerge and it has a very heavy lift. It really must serve the triple role of security, interoperability, and economic control. Most of that is based right now in using telemetry data to get into the head of the agent, understand the steps it took and why and then from there, moving into governance, so actually being able to control and optimize the behavior in real time, and ultimately, improve these agents in the long run, right?

[00:21:24] Melissa Incera: That serves as a reinforcing self-learning cycle whereby these systems are actually self-optimizing.

[00:21:31] Eric Hanselman: You basically get the telemetry for how well the agent is performing. The control plane is the piece in which you actually start to generate the feedback, because presumably you can also look for things like drift and other kind of challenges in terms of the operation of the overall system.

[00:21:45] Melissa Incera: Exactly

[00:21:47] Eric Hanselman: All you have to do is put it in place.

[00:21:50] Jean Atelsek: Yeah. But all of this is, the actions and the telemetry, it's all happening super fast, right? It's happening at machine speed. And you're going to need automation to build in that control. And yet, as we've seen, you can't just let agents run wild. You need to have humans reviewing what they're doing and constantly doing testing and evals and so on to steer them in a desirable direction.

[00:22:23] Eric Hanselman: Well, and like with most of the general operational environments that we've typically had in technology, this is something where getting to that level of autonomous action, we in this case, we got to autonomy a little bit ahead of our ability to fully control it, and are now working to really retrofit that to an environment to make it as well as useful, also practical and a little safer in terms of where that all goes.

[00:22:52] Eric Hanselman: But yeah, I guess we're doing the typical technology thing, taking a big step forward, maybe a little further ahead of our ability to understand the implications, and then taking a few steps back to put it back in place.

[00:23:05] Melissa Incera: Just circling back to the concept, why did Open Claw kind of seize the moment the way it did is because it showed the art of the possible in a way that we hadn't seen before, right? So yeah, to your point, we're now, we're retroactively building in the mechanisms to make it safe now that we know what can be done.

[00:23:24] Eric Hanselman: Well, and nothing like a little overeager action to get people actually a little more aware and a little more visible in terms of where this goes. Well, it is certainly an area that is changing so rapidly. I appreciate both of your insights in terms of being able to really get a feel for where we are on this.

[00:23:42] Eric Hanselman: But I'm sure we'll have to have the bots update this episode relatively soon with the latest. But thank you both. This has been great.

[00:23:51] Jean Atelsek: Thank you.

[00:23:52] Melissa Incera: Thanks guys.

[00:23:53] Eric Hanselman: There is so much more that we could potentially talk about, but we are at time. And that is it for this episode of Next in Tech. Thanks to our audience for staying with us, and thanks to our production team, including Sophie Carr, Feranmi Adeoshun, and Dylan Scheible on the marketing and events teams. If you liked this episode, please subscribe or like us. I hope you'll join us for our next episode, because there is always something next in tech!