Dogfooding the vulnerability apocalypse

We spent a month pointing AI agents at our own code, to see how the “vulnerability apocalypse” people keep talking about holds up against a codebase that ought to be a hard target: a security company, staffed by security people, with the usual scanners already running in CI. This is the field report: how we ran the audits, what our existing defenses missed, what the agents found, and what we’d tell anyone about to do the same.

TL;DR

• The agents found high-severity bugs in our own code. A binary-serializable API injection, a trivial multi-tenant denial-of-service, and a broken-access-control path in our authorization layer. All fixed.
• Nothing we had in place caught them. Not our tests, not the scanners in CI, not human code review. Nor the stack: Go rules out memory-safety bugs, so the agents went for what’s left, like authorization logic, resource exhaustion, and a tainted value crossing a trust boundary inside a dependency we don’t own.
• This isn’t a story about AI-generated code. Nearly all of these bugs were written and reviewed by humans, years before any agent touched the repo.
• We now run these audits continuously. If a team like ours was carrying these, most codebases are carrying more. Point the agents at your own code before someone else does.

We’d still push back on the word “apocalypse”. But the claim underneath it held up: these models find real bugs in code a security-conscious team believed was clean.

Why we did this

“Vulnerability apocalypse” is a phrase we keep hearing, from customers and at conferences: models have gotten good enough at offensive reasoning to make finding real bugs cheap and repeatable, and attackers will make use of that before defenders do. We wanted to know what that means for our code specifically, not for a hypothetical legacy codebase that was never going to hold up anyway.

On paper, we should be a hard case. Chainloop is at its core a security company, staffed by people with security backgrounds alongside senior software and platform engineers who have built and operated zero-trust environments. The codebase is Go behind protobuf-defined APIs, and we already run the tooling you’d expect us to run: software composition analysis, static security analysis, third-party dependency and vulnerability scanning, secret detection, all of it running in CI.

That is roughly the recommended-practice checklist, which makes the experiment a clean one. If a codebase like that holds up under a month of agentic auditing, the apocalypse talk is overblown. If it doesn’t, then most codebases are carrying the same gaps or worse. It didn’t hold up.

So over the last month we pointed a rotating set of agentic security assessments at our own repositories. There is no single “scan the repo” button here; each run varied the model, the orchestration, and the security tooling wired into the loop. We keep the write-ups short below, because the value is in how the agents found them.

How we ran the audits

We treated this as an experiment with a lot of knobs, and turned them independently.

Signal tools

The usual suspects, run to generate leads and feed context to the agents: Semgrep, gosec, govulncheck, Grype and Syft, and gitleaks. Two less familiar ones earned their place: zizmor, which analyzes GitHub Actions, Dependabot and pre-commit configs, and NVIDIA’s SkillSpector, a pre-install scanner for prompt injection, exfiltration and supply-chain risk inside agent skills themselves.

Skills and prompts

The tools produce findings; what an agent does with them is where the runs diverged. Everything in this group rides on top of an agent we already run, Claude Code, and changes only what it is told to do: the built-in /security-review command and the GitHub Action that runs the same analysis on pull requests; Cloudflare’s open-source security-audit skill; SnailSploit’s Claude-Red offensive-security skills; our own multi-phase audit skill (recon → parallel hunting agents → adversarial validation → independent verification); and hand-written prompts when we wanted to aim the model at one subsystem.

Standalone runtimes

The opposite approach: instead of layering instructions onto an agent we already run, these bring the whole loop themselves (model, tools, orchestration and isolation) as one thing you point at a target. We ran two open-source ones, and they disagree about containment in interesting ways, which is why we ran them side by side.

Strix is a fleet of autonomous pentesting agents: a root orchestrator delegates to per-attack-class agents that can spawn their own children, and every finding has to survive a validation step that builds and runs an actual proof-of-concept. It’s Python on OpenAI’s Agents SDK, and runs one Docker sandbox per scan, shared by every agent in that run.

Strix fans the work out to per-attack-class agents that can recurse, then fans the results back in. Every finding has to survive a validation step that builds and runs a proof-of-concept.

IronCurtain comes at it from the containment side: the security policy is compiled from a plain-English “constitution”, then enforced deterministically on every tool call, with no model in the decision path. Vulnerability discovery is one of the workflows it ships, a state machine that drives a containerized coding agent (Claude Code, Codex or Goose) through analyze, harness building, discovery and triage, with human gates on escalation and on the final report. It self-describes as an early-stage research prototype and behaves like one; the containment model is why we ran it.

IronCurtain routes every stage back through a single orchestrator and keeps humans on the escalation and final-report gates.

Models and orchestration

Mostly Claude Opus, with second-opinion runs on Sakana AI’s Fugu Ultra and on Kimi and GLM variants.

The recipe that consistently worked best was fan-out, then adversarial fan-in: one recon pass to map the architecture and trust boundaries, many parallel “hunter” agents each owning one attack class or subsystem, then a validation phase whose entire job is to disprove each finding. A finding only survived if an independent agent, often a different model, could reproduce it or trace the tainted path end to end. That last step is what kept the false-positive rate low enough to be worth a human’s time.

Which model ran which stage mattered. Claude was strongest at the analytical half: reasoning about a bug, tracing the flow, and writing it up so a human could act. For the mechanical step of turning an already-confirmed finding into a runnable proof-of-concept against our own code, less-restricted open-weight models were the better tool, because one that declines a legitimate security task on its own codebase is just friction there.

That split is really a spectrum, trading capability against compliance:

• Frontier, safety-aligned models did the analytical work, and their alignment showed up two ways: intermittent refusals or hedging on the offensive steps (Claude Opus did this to us until we moved to an access path intended for cyber work), and, further up the safety-hardened tier, routing the request away entirely. Claude Fable 5 doesn’t so much refuse cyber work as hand it off: when its classifiers flag a request as cybersecurity, the response is handled by Claude Opus 4.8 instead, and the user is told. None of this is a defect; it is deliberately trained behavior.
• Popular open-weight models like Moonshot AI’s Kimi and Z.ai’s GLM sat in the middle: close to frontier capability, more permissive by default, but still safety-tuned.
• Abliterated models are the far end: open-weight models whose refusal direction has been suppressed in the weights. Two caveats before reaching for one. Refusal is inhibited rather than removed, so they still moralize. And the procedure costs capability, unevenly and sometimes badly.

One warning on that word “independent”, because it is easy to lose. Fugu Ultra is an orchestrator rather than a single model, and it routes work across a pool of frontier models Sakana doesn’t disclose. A run on it isn’t guaranteed to be independent of your Claude run, which is most of the reason you’d want a second opinion in the first place.

Prompting

Prompting mattered more than we expected, and both examples we borrowed from teach the same thing: be concrete. Cloudflare’s skill opens with “You are a security auditor. Your job is to find exploitable vulnerabilities with real impact,” and rules out the alternative by name: “If you need the word ‘potentially’ or ‘theoretically’, you haven’t done enough research.” Baked into a system prompt, that keeps the output concrete instead of a checklist dump.

Searchlight Cyber’s Adam Kues found the recent WordPress pre-authentication RCE by adapting a prompt OpenAI had published for a hard math problem. What he rewrote was the opening task statement: rather than hinting a bug might exist, it asserts one does, fixes its class and reachability, and defines success concretely as a bug that reads /flag from the root of the filesystem. Being that specific is deliberate, to stop the model inventing unrealistic preconditions.

What the agents caught

We won’t catalog the individual findings; the ones in our open-source core are fixed in public PRs. What generalizes is their shape.

The highest-impact finding of the month had gone through every defense we have and come out clean: it was written by hand, reviewed by a human, covered by our test suite, and scanned in CI on every build since. No SAST rule, secret scanner, or vulnerability database flagged it, because it doesn’t look dangerous on any single line: the unsafe operation happens deep inside a third-party dependency, several hops from user input, and the application code that introduces the taint looks benign. Seeing it required following one value across functions and packages, into a library we don’t own, and then reasoning about who could reach the sink. That’s what an LLM is good at, and what a pattern-matcher can’t reach.

The signal tools weren’t useless, though. In one case, a cheap, noisy linter finding that most teams would wave off turned out to be the loose thread on a live production exposure, once an agent asked why the flagged endpoint was reachable at all instead of accepting that it “shouldn’t be.”

What worked, and what we’d tell you

A few things generalized well beyond these bugs:

• Scanners are for lead generation. They point agents at interesting code and occasionally hand you a lead outright, but the finding that mattered most was several hops downstream of anything they flagged.
• Adversarial self-verification is not optional. Agentic audits will happily generate plausible, well-written, wrong findings. The single most valuable phase in our harness was the one whose only job was to disprove each finding, ideally with a different model. If a finding survived a skeptic and a reproduction, it earned a closer look; if it didn’t, we dropped it.
• Reproduce, then quantify. “This could be a DoS” is a suggestion; a runnable proof that a few tiny requests leak unreclaimable work until a shared container dies is a decision-forcing artifact. Proofs-of-concept are what moved findings from interesting to urgent.
• Humans still own severity, remediation and threat modeling. The agents were excellent at mechanism and reachability. Grading each finding against our real deployment model, and choosing whether to bound a risky capability or remove it, were judgment calls: informed by the analysis, made by people. The severities in this post are ours, not the agents’.
• No single run is enough. Each audit explores different paths and finds different things; the best single run found maybe half of what several runs found together. So we run them repeatedly, on a rotation.

Is the “vulnerability apocalypse” real?

After a month of doing this to ourselves: yes. Every advantage we named at the top of this post is real, and none of it was enough to stop a binary-serializable API injection, a trivial multi-tenant denial-of-service, or a broken-access-control path in our own authorization layer. These are not toy bugs.

None of them is a memory-safety bug. Go took that category away, and the agents went for what it doesn’t cover. A memory-safe language and a typed API boundary protect against one class of mistake and do nothing about an authorization rule that reads one way to a reviewer and matches far more broadly at runtime.

Given the age of our codebase, nearly all of these issues were introduced by us, humans, long before any agent touched the repo; only a minority date to the period since we adopted agentic coding. They were written the traditional way and reviewed the traditional way. The process that produced them is the one most teams still treat as the safe default. That flips the usual worry on its head, and raises a sharper question than “is AI making our code worse?”: when an agent writes code, does it account for the security posture of the project at all?

The same tools that make this cheap for an attacker make it cheap for a defender, and right now defenders can still run them first. No team comes out of this with zero vulnerabilities; the ones that come out ahead will be the ones who pointed these agents at their own code before someone else did. So: get started.

Where Chainloop fits

Pointing these techniques at ourselves is the same discipline the product is built on: treat all third-party input as untrusted, and contain what you can’t verify. It’s what shapes how we sandbox our own AI agent features: auto-assessment, auto-remediation and AI scoring all run untrusted code inside locked-down sandboxes.

Chainloop didn’t find these bugs, an agent did. Where Chainloop helps is turning an audit like this into something that keeps working after it ends. Several of the signals we leaned on (SBOMs, vulnerability scans, CI-workflow audits) are exactly the evidence Chainloop collects, attests and gates on. A one-off finding becomes a policy-as-code rule, so a known-vulnerable dependency or an unsigned artifact can’t reach production on any build, not just the day you ran the scan.

Your own source code is only half the attack surface. The other half is the code you didn’t write, arriving through your build: the Trivy and LiteLLM compromises are what that looks like when it goes wrong. Same discipline, same answer — know what went into the artifact, attest it, and gate on it every build.

The scanners themselves are already part of the product. Several of the signal tools we leaned on for this audit ship as built-in scanners, so you run them from Chainloop instead of wiring them up yourself, with their output collected and attested like any other evidence and fed straight into the policies you already gate on. We’re now extending that set with a new scanner, in preview, for AI source code analysis: the kind of agentic auditing described above, pointed at your own repositories. The point is that an audit like this stops being a project somebody has to own for a month and becomes something your pipeline just does. Stay tuned, and let us know if you’d like access.

We’re going to keep running these audits and folding what we learn back into the product. Chainloop itself is open source, and the fixes for the issues we found in our core landed there in public: github.com/chainloop-dev/chainloop. We’d rather you learned from our bugs than repeated them.