Project Argus.

01 · the problem

I found an XSS in a production AI chatbot by hand. That bug got fixed, but the class of bug didn't go anywhere: teams are shipping AI agents with tool access, file access, and network access, and almost nobody audits what those agents are actually allowed to do before they go live. The question Argus answers is the one I kept asking manually: would I let this agent near production?

02 · the constraints

• Local-first. A security audit tool that phones home is its own vulnerability. V1 runs entirely on your machine; live probes are opt-in.
• Deterministic where it matters. An LLM judge can nudge confidence, but it can never erase a canonical failure. Findings must be reproducible.
• Read-only recon. Probing a live MCP server must never call its tools. Enumerate, don't execute.
• CI-gateable output: structured JSON and Markdown reports a pipeline can block a release on.

03 · the architecture

  ┌─────────────────────────────────────────────┐
  │ ARGUS · two gates for AI agents             │
  ├─────────────────────────────────────────────┤
  │ V1 · release gate (local-first)             │
  │  repo/config/skills → 27 canonical rules    │
  │  → live probes (opt-in) → mcp-probe         │
  │  → R = (S_base × C_env) × P_conf            │
  │  → report.json / report.md → PASS | BLOCK   │
  ├─────────────────────────────────────────────┤
  │ V2 · runtime gateway                        │
  │  agent → [request policy → approval 428     │
  │  → upstream → redact/block → audit chain]   │
  │  → model · /healthz · /metrics              │
  └─────────────────────────────────────────────┘

Two gates. V1 is the release gate: it audits an agent's repo, tools, permissions, secrets, MCP servers, and network config against 27 canonical rules with explicit capability contracts. AST analysis for Python, structured parsing for JSON/YAML/TOML, text rules for the rest, and a structured rule is never silently run against the wrong file type. Risk comes out as R = (S_base × C_env) × P_conf, clamped to documented bounds, with deployment profiles making context explicit: a banking agent scores differently from a public FAQ bot, on purpose.

V2 is the runtime gateway: it sits between the agent and its model. Prompt injection gets a 403 before the request reaches the model. Destructive tool calls are held. Business-impacting ones force human approval with an HTTP 428. Responses are inspected too: credential-looking output is stopped with a 502, sensitive fields are redacted, and every decision lands in a hash-chained audit log that tampering would break.

04 · what I owned

All of it. The two-gate design, the rule taxonomy, the risk model and its clamping behaviour, the decision that mcp-probe would be read-only, the audit-chain format. This is the tool version of how I already work: break it by hand first, then encode what I learned so it catches the next one automatically.

05 · failures & fixes

• rules firing on the wrong file types

  Early versions ran text rules against structured files and produced garbage findings. The fix was capability contracts: every rule declares what it can parse, and the engine refuses mismatches instead of guessing.

• the LLM judge being too persuasive

  A semantic judge that can downgrade findings is a prompt-injection target. Now it can only nudge P_conf within bounds; canonical failures are immune to it by construction.

06 · results

• ▸ real-world run: official pinned MCP filesystem server, 14 tools enumerated, 0 tool calls made, 2 HIGH findings, verdict BLOCK
• ▸ has scanned the configs of Claude Code, Codex CLI, and Gemini CLI
• ▸ 53 tests + Black + Flake8 + mypy + schema checks, green on every push
• ▸ reproducible JSON/Markdown reports, documented end-to-end in WORKFLOW.md