🇫🇷 Version française 💻 Source code (AGPL): github.com/patlegu/cyber-agent-engine

A LLM that can block an IP

There is something vertiginous about giving a language model the ability to call block_ip or add_filter_rule on a production firewall. You feed it a sentence — “block this IP that’s scanning the LAN” — and a few hundred milliseconds later, a rule shows up in OPNsense. That is exactly the promise of AI-assisted DevSecOps: translate intent into action, with no console, no ticket, no CLI in between.

It is also exactly the kind of promise that should make any security engineer’s hair stand on end. An earlier article on this blog, opnsense-ai-firewall (fr), told the story of a PoC where a fine-tuned LLM (LoRA) ran embedded directly inside the OPNsense VM, driving the firewall’s REST API straight from a natural- language intent — 101 out of 102 functions validated, latency measured, everything worked. And the article closed with four concrete reasons why that in-box topology should never go to production: attack surface, CPU contention, a model lifecycle coupled to the appliance’s own lifecycle, and no exploitable audit trail.

cyber-agent-engine is what’s left once you take that lesson seriously. The LLM moves out of the box (that’s the subject of article 4). But pulling the model off the appliance only solves one of the problems. The underlying problem — giving a probabilistic, non-deterministic system, trained on petabytes of public text, the ability to execute destructive actions on network infrastructure — remains entirely open until it is structurally constrained. Not with a better prompt. With an architecture.

The four concrete dangers

These aren’t theoretical dangers waved around to justify an article. They are the four failure classes that materialize the moment you connect an LLM to a network control plane — each with an identifiable incident scenario.

(a) Prompt injection → destructive action. The content the LLM “reads” isn’t always what the operator typed. A malicious hostname in a log, a comment in a ticket, a scanned service banner — any text that passes through the model’s context is an injection surface. If that text can divert an intent from “list the rules” to “delete the default deny rule,” the attacker no longer needs to exploit the firewall: they exploit the model that administers it.

(b) Hallucinating a catastrophic rule. Even absent an active attacker, an LLM can propose a function call that makes no sense — an add_filter_rule with a reversed direction, a 0.0.0.0/0 CIDR where the operator wanted a specific IP, a mass deletion order. A language model optimizes for the plausibility of text, not for the safety of a network control plane.

(c) PII leaking to a third-party LLM. As soon as a coordinator is wired to an external LLM API (OpenRouter, a cloud provider), every prompt transits through that third party’s infrastructure. If the prompt contains real LAN IPs, internal hostnames, VPN credentials — that’s infrastructure data leaking to a service you don’t control, on every single request.

(d) No exploitable audit trail. Without an explicit boundary, tracing “who asked for what, who decided what, what actually got executed” comes down to parsing natural-language conversation logs — unverifiable, and generally stuffed with the very same sensitive data you’re trying to protect in the first place.

These four failure modes share one thing in common: none of them is fixed by polishing the system prompt. “Never do that” in a system instruction is a statistical preference, not a control. A model that follows its instructions 99.9% of the time will, at the scale of an infrastructure handling dozens of requests a day, produce an incident sooner or later.

The thesis: a trust architecture, not a better prompt

The LLM must never be in a position to authorize itself. That is the thesis structuring the whole of cyber-agent-engine, and it breaks down into five concrete principles, each materialized in code — not just in documentation.

1. Fail-closed. By default, everything is denied. A policy that matches no explicit rule denies the action — it does not allow it “out of caution.” It’s literally written in the docstring of the module that evaluates decisions (core/policy/engine.py) — quoted here verbatim, in the original French, since the repo’s docstrings stay in French per project convention; in short, it says “policy evaluator — pure, deterministic function, fail-closed by default […] first matching rule wins (order IS priority, like a firewall); no rule matches → deny”:

« Évaluateur de politique — fonction pure, déterministe, défaut fail-closed. […] Première règle qui matche gagne (l’ordre EST la priorité, comme un firewall) ; aucune règle ne matche → deny. »

2. Least privilege. The LLM can only propose capabilities that exist in an explicit catalog (core/policy/catalog.py), with arguments validated against that catalog before they’re even evaluated against the policy. There is no generic execution channel — no eval, no shell, no arbitrary HTTP call left to the model’s discretion.

3. Human in the loop. Irreversible actions do not execute automatically: they suspend the run and wait for an explicit human decision (resume or reject). That’s the full subject of article 3 in this series, but the principle is simple — a model never cuts off a production VPN access all by itself.

4. Tokens only. The LLM never sees a real IP, a real hostname, a real identifier. It manipulates tokens (IP_1, VPN_USER_2) generated by a server-side vault; the real value is only substituted at the very end, at the execution boundary. That’s the subject of article 2 — and it’s probably the most counter-intuitive principle in the series, because it fundamentally changes what “a leaked prompt” means.

5. Auditable. Every policy decision is written to an append-only log, before it’s even known whether the action will be executed. And, like the rest of the pipeline, that log contains only tokens — never real values.

These five principles aren’t marketing slogans bolted onto a product that already worked. They are the invariants the core/ code is built to enforce structurally — to the point that some of them are enforced by the type system itself, as we’ll see in article 3 with the Authorized class, which cannot be constructed outside the two sole factories that prove either an allow verdict or a resolved human approval.

Overview: the core/

Concretely, this trust architecture lives in one specific module of the repository, separate from the rest of the application. Here is the actual tree:

core/
├── decision.py         # decide(): validate → evaluate → audit → verdict
├── orchestrator.py     # orchestration mono-action, partage decide() avec la boucle
├── policy/             # Moteur de politique fail-closed + catalogue de capacités
│   ├── engine.py        #   evaluate() — fonction pure, déterministe
│   ├── catalog.py        #   CapabilityCatalog — validation des intentions
│   ├── models.py          #   Intention, Rule, Verdict, Match, ArgMatch
│   └── loading.py          #   Chargement de la politique (YAML)
├── approval/            # Approbation humaine — fail-closed par défaut
│   └── store.py          #   pending → approved/rejected, liaison par hash
├── audit/               # Journal d'audit — append-only, jetons uniquement
│   ├── sink.py            #   AuditSink (protocole) + puits mémoire
│   └── file_sink.py         #   Puits fichier, rotation bornée
├── auth/                # Authentification par clé API — fail-closed au démarrage
│   └── api_key.py
├── tokens/              # Vault de tokenisation PII — le LLM ne voit que des jetons
│   └── vault.py
└── execution/           # Frontière d'exécution — seul point où une valeur réelle réapparaît
    ├── authorization.py  #   Authorized — preuve infalsifiable (sentinelle privée)
    └── boundary.py          #   execute() — détokenise puis appelle l'agent-outil

(Tree kept as-is — comments are quoted verbatim from the repo, which is documented in French. In short: decision.py orchestrates validate→evaluate→audit→verdict; policy/ is the fail-closed policy engine and capability catalog; approval/ is the fail-closed-by-default human-approval store; audit/ is the append-only, tokens-only audit log; auth/ is fail-closed API-key authentication; tokens/ is the PII tokenization vault; execution/ is the execution boundary — the only place a real value reappears, gated by the unforgeable Authorized proof.)

This tree isn’t an arbitrary split into folders. It materializes the path an intent follows between the moment the LLM proposes it and the moment a network packet actually changes fate. The decision.py module orchestrates the common sequence — “validate, evaluate, audit, return a verdict” — and that same decide() function is shared by the single-action orchestrator (orchestrator.py, for an isolated call) and by the coordinator’s gated loop (coordinator/loop.py, for the full ReAct flow), so that neither path can drift from the other. That’s DRY applied to a security-critical path — the worst thing you could do is have two implementations of “is this action authorized” that eventually diverge.

The organizing principle is simple to state: an intent proposed by the LLM can only reach a real device by traversing, in order, tokenization, policy, optional approval, and the execution boundary. No shortcut exists in the code — no direct-call function to a tool agent that would bypass core.execution.

flowchart LR
  Op["Operator / UI"] -->|natural language| Coord
  subgraph Coord["Coordinator — trust core"]
    direction TB
    Tok["PII tokenization"] --> Pol["Fail-closed policy"] --> App["Human approval"] --> Bnd["Execution boundary"]
    Aud[("Bounded audit — tokens only")]
    Pol -.-> Aud
  end
  Coord -->|"function + args"| Agents["Tool agents<br/>opnsense · wireguard · crowdsec"]
  Agents -->|"REST API"| Dev["Network devices"]

The operator sends natural language; the LLM, inside the coordinator, never crosses the boundary on the right by itself. It produces an intent — a capability and arguments, expressed as tokens — that the core/ validates, decides on, and audits before a tool agent (agents/opnsense/, agents/wireguard_agent.py, agents/crowdsec_agent.py) actually executes it via the targeted device’s REST API.

What’s next

This article laid out the problem and the thesis: without a trust architecture, wiring an LLM to a network control plane is an unmanaged risk, no matter how good the prompt is. The next five articles in this series each drill into a concrete pillar of this core/.

The next one focuses on the most counter-intuitive principle: the LLM only ever sees tokens. No IP, no hostname, no real identifier — never, at any step of the reasoning. It will detail the vault in core/tokens/vault.py, why this token↔value bijection radically changes the blast radius of a prompt injection, and why it’s also, incidentally, a matter of data sovereignty against external LLM providers.

To go further:

• opnsense-ai-firewall (fr) — the in-box PoC whose limits motivated this external architecture.
• Agents en Production — migration Qwen2.5 (fr) — the lineage of LoRA/agent experiments that preceded cyber-agent-engine.