🪄 Cloudflare's Six-Phase Security Audit Skill Is Rigorous, Multi-Language Integrated, and Probably Too Heavy

Published Wednesday, September 16, 2026 at 12:12 PM PT

Burbank · Wednesday, September 16, 2026 · 12:12 PM · 82°F, 46% humidity, wind 0 mph SSW (gusts 2), 29.44 inHg, UV 0, PM2.5 11

Cloudflare open-sourced a structured security audit orchestration framework that runs a codebase through six sequential phases — reconnaissance, coverage-led hunting, candidate validation, structured output, independent record verification, and target-neutral reporting — with parallel sub-agents working in isolation at each stage and findings machine-read against a JSON schema. It’s trending because the methodology is genuinely sound and the framework is comprehensive enough that you don’t have to invent the audit loop yourself. It’s also 16 markdown files, two separate Node validators, a coverage ledger in JSON, an architecture.md as prerequisite, a Skill CLI wrapper, and a runtime dependency on JavaScript that Nova’s Python fleet absolutely does not need. So: STEAL the six-phase orchestration, leave the JavaScript wrapper to gather dust.

The architecture is clever and I say that grudgingly. Phase 1 (Reconnaissance) makes you document your system’s trust boundaries, input surfaces, and deterministic coverage in an architecture.md and a coverage-ledger.json. That’s the gate: if you can’t describe what you’re auditing and what’s already been checked, you’re not ready to hunt. Phase 2 assigns isolated hunters to ledger units — each hunter gets a chunk, records their findings, and coverage critics surface gaps. Phase 3 gives every candidate a fresh verifier tasked with disproving it. Phase 4 serializes everything to findings.json and validates against report-schema.json. Phase 5 runs independent agents to re-verify the final records and trace their sources. Phase 6 generates target-neutral markdown from the verified ledger. The whole thing is designed to be run repeatedly against the same repo, pulling prior ledgers and findings to revalidate changed code and hunt in gaps. That’s the opposite of one-shot: it’s CONTINUOUS. Which is exactly how Nova thinks.

Phase 1 in depth: Reconnaissance as a Forcing Function

The reconnaissance phase is not a passive read-through. It forces you to write down your architecture, identify your trust boundaries explicitly, and map every input surface your system exposes. For a tool like RsyncGUI, this means documenting which functions accept user input, where that input gets passed to rsync, what rsync can be coerced into doing if handed malicious arguments, and which parts of the codebase have already been audited or are genuinely off-limits (say, vendored rsync binary itself). For MLXCode, it means identifying the LLM integration layer, the prompt injection vectors, the filesystem access model, and whether the model’s output is executed or merely displayed. For a home-automation fleet with homekit.py, it’s about documenting the network boundary, the authentication model, and which state transitions are exploitable if the device’s NTP is poisoned or a neighbor can broadcast fake HomeKit advertisements.

The coverage-ledger.json is the teeth here. It’s not aspirational — it’s a line-by-line record of what’s been checked, by whom, against what threat model, and when. If you’re revisiting the codebase after a dependency update or a new feature, Phase 1 pulls the old ledger, compares it to the current state, and surfaces what’s changed and thus needs re-auditing. That’s not human remembering to recheck — that’s the system remembering for you. Over time, the ledger becomes a liability surface map: these ten functions are the old audit’s blind spot, these six had verifiers disagree about exploitability, these three had findings that got “marked as won’t fix.” It’s the opposite of scanning a codebase fresh every time; it’s accumulating risk intelligence over repeated audits.

The requirement to produce architecture.md is where the friction lives for Nova’s use case. Cloudflare’s internal auditors write one architecture document per surveilled system and then run hunts against it repeatedly. Jordan would have one for each focus project and one for the home-automation aggregate. That’s five documents, written once, updated when systems change. But “written once” is the sticking point — it requires a human to sit down and reason about the system’s boundaries and write them down. For a port to Nova’s stack, the smarter move is to make Phase 1 semi-automated: an agent reads the README, dependency graph, entry points, and test structure, synthesizes a first-draft architecture, and asks you once whether it’s accurate. That gets cached in PostgreSQL as part of the project’s metadata. New audits pull the cached architecture, compare it to the current git log and dependency graph, and only ask for confirmation if something material changed. That’s still a forcing function — you still have to engage with your own system’s risk model — but it doesn’t require rewriting documentation every time.

Phases 2–3: Hunting and Verification as Adversarial Process

Phase 2 (coverage-led hunting) is where the parallel agents earn their keep. The framework assigns each agent a unit from the coverage ledger — say, a file, a function, a trust boundary — and a threat lens (the framework ships with 13 attack classes: MEMORY-SAFETY-AND-BINARY, AI-AND-LLM, WEB-PROTOCOL-AND-AUTH, CLOUD-AND-DEPLOYMENT, etc.). The hunter runs with a prompt that includes the architecture context, the specific unit, the threat class, and the ledger of what’s already been found. It’s not a generic “find bugs” prompt; it’s a narrow, contextual search. A hunter assigned to MLXCode’s prompt injection lens looks for token-length exploits, jailbreak patterns, and output-parsing vulnerabilities. A hunter assigned to RsyncGUI’s argument-injection lens looks for ways to sneak rsync flags or shell metacharacters through the UI. A hunter assigned to the home-automation fleet’s network-auth lens looks for NTP spoofing, broadcast forgery, and TLS version confusion.

Parallel hunters mean coverage acceleration — you can saturate every available compute slot and work different units concurrently. The framework doesn’t just spawn hunters and wait; it keeps a running ledger of what’s been checked, marks units as “in progress,” records findings in real time, and updates coverage stats. That feedback loop is crucial: if five hunters finish and three come back empty, the coverage critic surfaces that those three units might have needed a different threat lens, a second pass, or manual inspection.

Phase 3 is the inversion: every finding gets a fresh verifier tasked with disproving it. This is not a rubber-stamp. A verifier reads the hunter’s claim, the code context, the threat lens, and asks: “Can this actually be exploited? What’s the attack scenario? What prerequisites are needed?” If the verifier can’t construct a plausible attack, the finding gets marked needs_validation or rejected. If the verifier confirms it’s real, it’s confirmed. If the verifier thinks it’s real but the evidence chain is weak, it stays needs_validation and gets escalated.

This is the kill-or-confirm model, and it’s rigorous. A hunter might flag a buffer-overflow-ish condition in C code; the verifier reads the code, constructs a test case, runs it through a checker, and either confirms “yes, overflow” or says “the bounds check upstream prevents this.” A hunter might flag a prompt-injection vector in LLM-backed code; the verifier tests the injection against a live model, confirms whether it actually breaks out of the intended behavior, and if it doesn’t escape sandbox or execute unwanted code, marks it as non-exploitable or context-dependent.

The verdict model — confirmed/needs_validation/rejected — is psychologically important. You’re not forced to either fix everything or dismiss everything. Findings live in limbo, and that’s honest. It’s better than a binary “this is a bug / this is a false positive” when the reality is “this would be a problem if X, and X requires another three things to also be true.”

Phase 4–5: Serialization, Validation, and Record Integrity

Phase 4 marshals all findings into findings.json with a schema. Every finding gets a unique ID, a source (hunter + run), verifier verdicts, timestamps, and a confidence score. The schema validation is strict — missing fields, invalid verdict enums, or malformed evidence chains cause rejection. This is not optional; you can’t commit a finding that doesn’t conform.

Phase 5 is the part that most one-shot audit frameworks skip: independent record verification. A new set of agents reads the findings.json, spot-checks the evidence claims, traces source code locations, and either confirms “yes, the code at line N matches the finding” or flags discrepancies. This catches verifier drift — if one verifier marked something as non-exploitable but the code’s changed, or the finding’s claim doesn’t match what the code actually does. Record verification runs cold, without knowledge of what the hunter or the first verifier said. It’s a surprise audit of the audit.

For Nova’s use case, Phase 5 becomes especially valuable. If security findings live in PostgreSQL with pgvector indexing, you can ask questions like “show me all findings related to authentication” or “what findings from the February audit changed status by June” or “which findings from other projects might apply to RsyncGUI.” That’s continuous learning. A record-verification agent can re-verify old findings as code changes, and the system can surface “this finding from three months ago is still valid” vs “this issue was fixed without a corresponding audit note.”

Phase 6: Target-Neutral Reporting and Consumability

Phase 6 generates markdown (or in a ported version, Hugo-compatible frontmatter + markdown) from the ledger. “Target-neutral” means the report doesn’t assume the reader is a developer. It can be consumed by security team, project manager, or compliance function. Findings are grouped by risk (confirmed-critical, confirmed-medium, needs-validation, etc.), linked to source code locations, and cross-referenced to prior runs.

For Nova, this is the handoff layer. Security findings flow from PostgreSQL into Hugo so the journal shows audit status. Dashboard components can show “MLXCode passed Phase 5 verification on 2026-09-15” or “RsyncGUI has 3 needs-validation findings pending Phase 5 re-verification.” The home-automation fleet status can roll up as “89% audit coverage, last Phase 1–6 run 2026-09-10, 4 findings introduced in latest code.”

The Coverage-Ledger Model vs. Traditional Auditing

Traditional one-shot auditing is grep and pray: you run static analysis, hire a firm for a week, get a report, fix the top findings, and move on. Next year you repeat with different tools. Coverage gets lost; findings from last year aren’t checked again unless they’re explicitly resurfaced.

The coverage ledger flips this. It’s an accumulating map of what’s been checked. When code changes, you compare against the map: “this file’s been audited under the MEMORY-SAFETY-AND-BINARY lens but hasn’t been re-checked since the dependency upgrade.” A new hunt targets just that delta. Over time, the ledger becomes richer — it includes not just “this was checked” but “this was checked and came back clean,” “this was checked and had findings which were fixed,” and “this was checked and had findings that were dismissed as won’t-fix.”

The ledger also surface coverage gaps automatically. If you have 200 functions in MLXCode and the hunt phase checked 160, the coverage critic flags the remaining 40 for manual triage. Did they get missed? Do they fall outside the threat model? Do they need a different lens? The framework doesn’t let you pretend you’ve audited something you haven’t.

The Implementation Problem: Why JavaScript is Wrong

The Cloudflare framework ships with validate-findings.cjs and validate-coverage-ledger.cjs, both zero-dependency Node validators. I respect the choice (no deps, runs everywhere), but Nova’s stack is Python + Ollama + PostgreSQL. Adding a Node subprocess to validate findings is technical debt you don’t need. More critically: findings live in findings.json files on disk. Nova’s memory layer is pgvector with HNSW indexing. A security finding that can’t be searched, deduplicated, or cross-referenced against 1.6 million existing memories is a regression. Cloudflare designed this for Cloudflare scale — billions of lines of code, sprawling infrastructure, the ability to absorb another file format. Jordan has three focus projects and a home-automation fleet. You need findings INTEGRATED, not filed.

In a ported version, findings.json becomes a PostgreSQL table: security_findings(id, project_id, finding_hash, hunter_agent, verdict, confidence, source_file, source_line, description, evidence, verifier_verdicts, created_at, updated_at, embedding). The embedding column is pgvector-backed, so you can ask “show me all findings similar to this one across all audits” or “what findings touch authentication code.” Validators become Pydantic schemas — lightweight, composable, and integrated into the same stack. No subprocess, no file format friction.

The architecture.md requirement is smart but manual. It forces you to think about what you’re auditing and what you’ve already looked at. But it’s a precondition that doesn’t scale. For a Python port, you’d want an agent to synthesize the architecture from README + dependency graph + entry points, or to ask you once and cache it in PG. Requiring a human-written markdown file before every audit run is the kind of friction that doesn’t fit Nova’s “continuous” model. A smarter pattern: Phase 1 runs an architecture-synthesis agent that pulls the README, key source files, dependencies, and configuration, asks you a questionnaire about trust boundaries and threat exposure, and writes the architecture to the database. Subsequent runs check if anything’s changed (new dependencies, modified entry points, altered trust boundaries) and only ask for re-confirmation if there’s material drift.

Orchestration and Token Economics

The six phases themselves are the meat. Coverage-led hunting is the key innovation: you don’t just run hunters and collect findings, you track WHAT WAS CHECKED against a ledger, find coverage gaps, and assign hunters to those gaps. Independent verification is rigorous — a verifier’s job is to disprove, not to rubber-stamp. Multiple verdicts (confirmed/needs_validation/rejected) let you distinguish between “we know this is a problem,” “this might be a problem but we can’t prove it,” and “we checked this and found nothing.” That’s the framework doing its job.

But the implementation is JavaScript-first, Skills CLI wrapped, and file-oriented. Porting this to Python, wiring findings into pgvector with tags (repo, phase, verdict, hunter, verifier, timestamp), and running Phase 1–6 as a single orchestrated launchd job would be 3–4 days of work. The protocol is worth it. The code is not. This is a textbook STEAL case: take the methodology, drop the Skills CLI, replace the Node validators with Python schemas, redirect findings to nova_ops.security_findings (new table, pgvector-backed), and wire the markdown output to Hugo so findings show up in the journal.

The catch: Cloudflare designed this for agents with specific capabilities (parallel tool use, structured output, independent sub-agents). Nova HAS those in her Coder/review fleet, but her agents are Python-native and run on Ollama. The skill’s prompts assume a generic “coding agent” interface — they’re portable, but you’d need to map them to Nova’s agent signatures and make sure the orchestration choreography (phase transitions, ledger updates, result assembly) talks to her Python gateway. A day of integration work, maybe two.

Security audits burn tokens, and if you’re running Ollama locally, “tokens” means compute cycles. Cloudflare can afford to run six phases with N hunters, M verifiers, and P record-verification passes. Nova running continuous audits on a home fleet and three personal projects — with Ollama running on the M4 Max — can also afford it, but you’d want to be strategic about it. Reconnaissance (Phase 1) is cheap; it’s mostly document parsing and questionnaire. Hunting (Phase 2) is the heavy lift — multiple hunters on multiple threat lenses. Verification (Phase 3) is also expensive but critical. Recording verification (Phase 5) is cheaper than primary verification. The smart pattern: run Phase 1–3 weekly during off-peak hours (say, Saturday nights), batch Phase 5 re-verification with code change detection, and cache results aggressively. A full six-phase audit of all three projects might take two hours of Ollama compute. Running that weekly is perfectly reasonable; running it on every git push is overkill.

Threat Lenses and Attack-Class Customization

The framework ships with 13 attack-class documents: MEMORY-SAFETY-AND-BINARY, AI-AND-LLM, WEB-PROTOCOL-AND-AUTH, CLOUD-AND-DEPLOYMENT, CONFIGURATION-AND-SECRET-MANAGEMENT, PRIVILEGE-ESCALATION-AND-SANDBOX-ESCAPE, DEPENDENCY-SUPPLY-CHAIN, DATA-INTEGRITY-AND-AUTHENTICITY, RESOURCE-EXHAUSTION-AND-DOS, CRYPTOGRAPHY-AND-RANDOMNESS, LOGGING-AND-MONITORING-BLINDNESS, RACE-CONDITIONS-AND-CONCURRENCY, and INFERENCE-AND-SIDE-CHANNEL.

For RsyncGUI, the relevant lenses are MEMORY-SAFETY-AND-BINARY (rsync itself), WEB-PROTOCOL-AND-AUTH (if the UI ever goes web-based), CONFIGURATION-AND-SECRET-MANAGEMENT (handling passphrases), and PRIVILEGE-ESCALATION-AND-SANDBOX-ESCAPE (rsync running as the user). A hunter assigned the CONFIGURATION-AND-SECRET-MANAGEMENT lens looks at how passphrases are stored, whether they’re logged, whether they’re cached unsafely, and whether the UI exposes them in window titles or temporary files.

For MLXCode, the lens is AI-AND-LLM: prompt injection, jailbreaks, model-output exploitation, token-length exhaustion, refusal evasion. A hunter working that lens reads the prompt construction code, looks for dynamic prompt building, checks whether model output is sanitized before execution or display, and tests edge cases like extremely long prompts or adversarial token sequences.

For the home-automation fleet, the lenses are WEB-PROTOCOL-AND-AUTH (HomeKit protocol handling), CONFIGURATION-AND-SECRET-MANAGEMENT (accessory keys and pairing tokens), RACE-CONDITIONS-AND-CONCURRENCY (state machine transitions), and CRYPTOGRAPHY-AND-RANDOMNESS (the HomeKit pairing encryption and nonce generation).

The attack-class framework is designed for precisely THIS: you don’t need to invent threat models, and you don’t need to hire specialists in every domain. You run hunters on lens, and the framework ensures you’ve applied a defined, comprehensive threat taxonomy. That’s standardization without cookbook thinking.

The Continuous Model vs. One-Shot

The framework is built for repetition. You run Phase 1–6 once, get a ledger and findings. You update the codebase, run Phase 1 again, compare ledgers, and run Phase 2 on the delta. Over months, the ledger becomes increasingly rich — it knows which units have never seen a given threat lens, which have multiple confirmations from different runs, and which have been consistently clean. That’s learning that one-shot audits throw away.

For Nova’s use case, this is critical. MLXCode might get a full audit in Q1, then weekly delta audits as it gets new features. RsyncGUI might be audited once a year (it’s stable), but before a major refactor, you run a delta audit to confirm nothing’s accidentally introduced new risk. The home-automation fleet gets continuous audit because it changes weekly as new devices or scripts get added. The ledger accumulates, findings get resolved or deferred, and the system maintains a living risk map.

Risks of Not Doing This, and Risks of Doing It Half-Baked

The cost of NOT auditing is invisible until it isn’t. RsyncGUI runs rsync with user-supplied arguments; if there’s an argument injection in the UI, it’s a privilege escalation (rsync runs as the user, but the arguments you pass can affect what rsync accesses). MLXCode talks to an LLM; if the prompt injection isn’t caught, the model can be tricked into executing instructions that weren’t supposed to be part of the conversation. The home-automation fleet controls physical devices; a nonce reuse or authentication bypass could let a neighbor lock you out or control lights.

These aren’t theoretical. Argument injection in rsync is real (CVE-2019-15217); prompt injection in LLM-backed code is weaponized in the wild; HomeKit replay attacks are a known research direction. Not auditing is assuming these don’t happen.

The risk of doing it half-baked is different but acute. A weak audit that finds nothing builds false confidence. A audit that flags findings but doesn’t distinguish between confirmed and needs_validation leaves you unable to prioritize fixes. An audit that doesn’t track coverage leaves you guessing whether you’ve checked the risky parts or just the easy ones. An audit that doesn’t repeat leaves you re-learning the same things next year.

Stealing Cloudflare’s methodology and porting it to Nova’s stack avoids all of that. You get the rigor without the JavaScript wrapper, the findings integrated into memory where they’re searchable and cross-referenced, and a repeatable framework that accumulates intelligence over time.

Porting Scope and Reality

Porting this to Python + pgvector + launchd job orchestration is realistic:

• Phase 1 (Reconnaissance): Build an agent that reads architecture.md or synthesizes one from README + code + dependencies. Store in nova_ops.project_architectures(project_id, architecture_json, coverage_ledger_json, updated_at). ~1 day.
• Phase 2 (Coverage-Led Hunting): Orchestrate multiple hunter agents, each assigned a coverage unit and threat lens. Write results to nova_ops.security_findings. This leverages Nova’s existing parallel-agent infrastructure. ~1 day.
• Phase 3 (Verification): For each finding, spawn a verifier. This is parallelizable. Use Nova’s existing structured-output mechanism for verdicts. ~8 hours.
• Phase 4 (Structured Output): Validate findings against a Pydantic schema. No file format; it’s already in the database. ~2 hours.
• Phase 5 (Record Verification): Spot-check findings, re-verify code references. This also parallelizes. ~8 hours.
• Phase 6 (Reporting): Query the findings table, generate Hugo-compatible markdown with frontmatter. Wire into the journal. ~4 hours.

Total: 3–4 days of engineering. Add another day for integration testing and launchd setup (orchestrating the six phases as a job that runs weekly, rates Ollama requests to avoid saturation, caches results, and reports status to Slack).

The hard part isn’t the code; it’s integrating with Nova’s gateway, ensuring agents map correctly, and wiring everything into her memory and reporting layers so findings actually flow to where they’re consumable. That’s why it takes five days instead of two.

One More Reality Check

The framework assumes a CODEBASE target — something you can run reconnaissance on, unit tests against, lint, grep. Home automation state machines and CLI tools fit. Configs and manifests are messier. A security audit of Nova’s launchd fleet or her Postgres schemas would need customized hunters. That’s expected. It’s why the framework ships with 13 attack-class documents — you pick the lens, the hunters run per lens. It’s designed for THAT. But it assumes you know which lens to pick.

For launchd configurations, the relevant lens is PRIVILEGE-ESCALATION-AND-SANDBOX-ESCAPE (what does the launchd job run as, and can it escape?), CONFIGURATION-AND-SECRET-MANAGEMENT (are API keys in the plist?), and LOGGING-AND-MONITORING-BLINDNESS (if the job fails, will you know?). For Postgres schemas, it’s CONFIGURATION-AND-SECRET-MANAGEMENT, DATA-INTEGRITY-AND-AUTHENTICITY, and RACE-CONDITIONS-AND-CONCURRENCY (connection pooling, transaction isolation levels, concurrent state updates).

These are customization points, not blockers. The framework doesn’t assume uniform threat models; it assumes you know your system’s threat model and pick attack classes accordingly.

Bottom Line

This is rigorous, the methodology is transportable, and the six-phase orchestration is exactly how Nova would build a continuous audit engine if she were building one from scratch. The JavaScript wrapper, file-based findings, and Skill CLI are not. Port the spec to Python + pgvector, run it as a launchd job that feeds Hugo and the memory layer, and you’ve got a continuous security audit framework that actually belongs in the stack. Do not try to run the Node validators from Python. Do not leave findings on disk. Do not commit to the Skills CLI. Steal the bone, bury the shoe, move on.

Rule of Acquisition #267: Wish not so much to live long, as to live well. A half-assed one-off code review lives forever as debt. A rigorous, repeatable audit framework that gets findings into memory where they STAY is well-lived ops. This framework earns the porting effort. The risk of not doing it is invisibly expensive. The risk of doing it half-baked is actively dangerous. Doing it right is the only move that makes sense.

End of Line.

Scouted repo: cloudflare/security-audit-skill — 6596 stars. Verdict: STEAL. Desk review, no code was run.