This website uses cookies

Read our Privacy policy and Terms of use for more information.

What you'll learn

  • Why markdown files can't govern a coding agent — instructions compete for the LLM's attention, and any user can just have Claude open the file in Visual Studio and delete it.

  • What "pre-execution interception" actually requires: accumulating session context, computing agent intent, and choosing between five authorization decisions — allow, block, modify, defer, and human-in-the-loop — instead of the old binary of allow/deny.

  • How a research paper written inside Vanta became a Cloud Security Alliance specification with 93 companies behind it, and why Herman kept it out of any single vendor's hands.

Description

Herman Errico is a product manager at Vanta with a decade in security operations and incident response, and a past life at the British Standards Institution. He went looking for what secures an AI agent at the moment it stops generating text and starts taking action, found that nobody had named it, and wrote the paper that became the Autonomous Action Runtime Management specification — now housed under the Cloud Security Alliance. This conversation is about that boundary: what happens at the point of action, why the tools already on your stack don't watch it, and how a community of builders converged on the same problem independently. 

The through-line is that agents have moved from generating text to orchestrating actions, and traditional systems — DLP, EDR, firewalls — were never built to compute agent intent. Herman calls the point of action "layer eight," sitting above the application layer, where no existing control does context accumulation or pre-execution interception. The spec has gone from one person's Notion notes to 93 companies in roughly a year, including Microsoft, Okta, Rubrik, and Noma, the first to run a conformance review. This one is for security leaders deciding how to evaluate agent-security vendors, and for anyone trying to figure out where governance for autonomous action actually belongs.

What we cover

  • "Now we have like layer eight, where the action of the agent happens." — why Herman frames the action boundary as a new layer above the application stack, and why no existing tool watches it.

  • "Were you reading my Notion side?" — how 93 companies turned out to be building the same primitives — pre-execution interception, semantic distance tracking — before the spec named them.

  • "It did change 241 files... it did change 10,000 lines of code." — the copy-change-gone-wrong story that grounds the whole argument for intercepting before execution.

  • "The ARM specification introduces five different authorization decisions." — moving past allow/block to modify, defer, and human-in-the-loop.

  • "Markdown files and skills do compete for attention." — why you can't govern a coding fleet with an MD file, and where the bypassable layers rank.

  • "Rather than making everyone pay for the car seat, they just gave it out for free." — the Volvo seatbelt analogy behind keeping ARM under CSA instead of Vanta.

  • "The name says autonomous action. It does not say agents, it does not say AI." — why the spec is built for robots and autonomous vehicles, not just today's agents.

 

Thank you to our Sponsors:

→ Hampton North is the premier US based cybersecurity search firm. Start building your security team with Hampton North.

→ Sysdig is the leader in AI-powered real-time cloud defense; stop watching and start defending.

The conversation

Layer eight is where the action happens — and nothing was built to watch it

Herman's starting move is to relocate the problem. Zero trust and the traditional stack top out at layer seven, the application layer. Agents introduced something above that — the point where reasoning turns into a tool call, a code push, a real-world change. He calls it layer eight, and the whole ARM argument follows from taking that layer seriously.

"You have like a layer seven, the application layer, and now we have like layer eight, where the action of the agent happens. And no system today was really built to do pre-execution interception, do context accumulation."

— Herman Errico

The reason existing tools miss it is that they don't compute intent. A firewall sees a connection, DLP sees data leaving, EDR sees a process — none of them know what the agent was trying to do. Herman's point is that securing an agent means accumulating the session context, capturing the plan the model produced as machine-readable data, and then reasoning about whether the action in front of you matches the scope of the work. That's the primitive nobody had, and it's the reason he thinks agent security is its own system category rather than a feature you bolt onto what you already own.

The system-category framing does real work here. Herman leans on the antivirus lineage — signature-based detection, then behavioral, then EDR — to argue that categories are how practitioners pair a problem to a solution and then raise the floor over time. Once everyone agrees what a category is supposed to do, you get more out of every vendor in it, instead of a scatter of point solutions each defined by whoever built them.

The copy change that touched 241 files

The most useful thing in the episode is a concrete failure. Herman told Claude to make a simple copy change on a landing page — swap the text to read "ARM 2.0." Claude got confused, started editing other files, and reconstructed changes it didn't have locally.

"Claude got confused, started making other copy changes to other files, and then it did change 241 files... it did change 10,000 lines of code."

— Herman Errico

The PR got blocked at merge, so no harm done — but that's the point. The only thing that caught it was a downstream control, and only because a human happened to be paying attention. With pre-execution interception, the gap between the stated plan (one copy change) and the actual action (250 files) is exactly the kind of misalignment you can catch before anything executes 

This is where the five authorization decisions matter. Security has always run on allow or block. Agents need more range: modify, defer, and escalate to a human in the loop. The system can hold the action, surface it — "are you happy that Claude is about to change 241 files?" — and let you correct course or kill it. Conor's framing lands the stakes: the danger isn't malice, it's confident wrong action. "I can confidently do the wrong thing. That is the world of AI right now."

Why markdown files won't govern your agents

Conor puts the popular myth on the table: if I write the right markdown file, I can govern Claude, Codex, Gemini. Herman's answer is a flat no, and the reasons are worth internalizing.

First, the file is trivially removable. Any user can ask Claude to find the global MD file, open it, delete it, and proceed. Second — and this is the deeper point — instructions in markdown and skills compete for the model's attention against everything else in the context window.

"Skills and markdown files do compete for attention... Once you start competing for attention, those skills might lose in terms of priority with some of the actions of the system prompt. Therefore, agents will hallucinate."

— Herman Errico

 He stacks numbers on top of it: hallucination rates he cites run 36 to 50 percent even on advanced models, higher on weaker ones and on complex tasks. The longer an agent runs, the more its context shrinks and recompacts, and the more your carefully written guardrails degrade. Instructions inside the same statistical machine you're trying to constrain are not a control — they're a suggestion.

 The takeaway for a security leader is that guardrails have to live outside the thinking machine. Herman ranks the layers by how bypassable they are: markdown at the top (most bypassable), then application-level interception (an agent can code an API call or use a CLI to route around it), and least bypassable at the network or eBPF layer, where the agent gets a 503 and doesn't even know it was stopped.

Keeping it neutral: the Volvo seatbelt move

ARM started as a research project inside Vanta. Herman is a product manager whose job includes deep technical research, and he could have kept a genuinely novel idea proprietary. He didn't, and his reasoning is the most instructive part of the governance story.

"The moment you put something that is proprietary to a single company, then you get a very small adoption and then it becomes pay for play and then there's no innovation."

— Herman Errico 

The analogy he reaches for is Volvo's three-point seatbelt — patented in 1959, then given away to every manufacturer because it saved lives. Herman's hope for ARM is more modest ("we're probably gonna save you some tokens or some bad things"), but the mechanism is the same: put it in the open, let anyone join the working group, and let the standard improve faster than any one vendor could push it.

That neutrality also changes the buyer's position. Because the spec is built in the open — Herman notes a working group that meets on a two-week cadence — a CISO can benchmark agent-security vendors against a shared reference instead of guessing. Conor's read is that it cuts both ways: on the buying side you can hold vendors to a defined bar; on the building side you have something to anchor to. Either way the effect is a consistent set of capabilities you know you're getting.

The competition is healthier than expected

Conor pokes at the obvious tension: you've got direct competitors — Noma, Okta, Sysdig, Runlayer, Mint, and others — in the same working group. Doesn't that turn into a fight? Herman expected exactly that and it didn't happen.

"I was really expecting some fights happening. But they never happen because... it's all very eager engineers that want to collaborate."

— Herman Errico 

The distinction he draws is between the sales-room and the engineering-room energy. CEOs aren't on the Friday calls; engineers, product managers, and enthusiasts are, and they're trading feedback and even joining forces to build together. Companies solving the same problem for the same customers are still competing — Herman is clear about that — but they're doing it on execution, dividing up features and priorities in the open rather than fighting over the definition of the category.

His mechanism for how a standard improves is worth holding onto: builders solve pre-execution interception in their own clever ways, feed the approach back to the working group, and the sharpest solution becomes the reference. He described one company (unnamed, for neutrality) that blocks the action, runs a separate LLM to predict the output of where the agent wants to go, and decides from there — a form of computed trust that sidesteps the false-positive mess of writing deterministic policies for every case. Each cycle raises the bar for whoever builds next.

Autonomous action, not agents — and eventually hardware

The name is deliberate. It says Autonomous Action Runtime Management, not AI, not agents. Six months ago these were copilots; now they're agents; next they're something else. Herman built the scope to survive that churn.

"What we want to govern is managing this autonomous action that happens without human intervention. That's the core component."

— Herman Errico 

His twelve-month wish is a piece of hardware that runs an ARM solution — because the same primitives apply to a robot deciding whether a basket is laundry or a child. That's not a throwaway. He points out that some companies have solved the robotic hand, the hardest part to build, which to him signals that robotics, AI, and LLMs converge sooner than people expect. The teaser for his Black Hat talk with Camille sits upstream of the spec itself: before you apply any runtime control, ask what you're delegating to an agent, who's responsible, and who's accountable. Layer eight is the headline; delegation and accountability are the prerequisites.

Show notes

  • Guests — Herman Errico, Product Manager at Vanta; ~10 years in security operations and incident response; former British Standards Institution; author of the research paper that became the ARM specification.

  • Books mentioned — None named in the conversation.

  • Frameworks / models / tools named — Autonomous Action Runtime Management (ARM) specification; Cloud Security Alliance; MCP (Model Context Protocol); MAESTRO; Agentic Trust Framework; antivirus / EDR / DLP / firewalls / gateways; eBPF; Claude / Claude Code; ChatGPT; Codex; Gemini; Opus 4.8; Visual Studio; Microsoft agentic governance toolkit.

  • Other people / shows / resources referenced — Camille (Black Hat co-presenter); Jim Reavis (President and co-founder, Cloud Security Alliance); Noma Security; Okta; Rubrik; Microsoft; Sysdig; Runlayer; Mint; Volvo (three-point seatbelt, 1959); "Attention Is All You Need"; Black Hat.

Hosted by Conor Sherman and Stuart Mitchell.

Keep Reading