🇫🇷 Version française 💻 Source code (AGPL): github.com/patlegu/cyber-agent-engine

A question of topology, not just security

The first three articles in this series detailed how cyber-agent-engine constrains an LLM that drives a production firewall: tokens rather than real values (article 2), a fail-closed policy and human approval (article 3). This one tackles a prior question, almost architectural before it is a security matter: where must this LLM physically run? On the equipment it drives, or remotely, on the other side of a network call?

The answer isn’t arbitrary — it has already been measured, in the other direction, by an earlier project on this blog.

The contrast: what the in-box approach measured

opnsense-ai-firewall (OAF) is a PoC that pushes the reverse logic all the way through: the LLM runs embedded inside the OPNsense VM itself. No sidecar, no network call to an external service — a llama-server compiled natively for FreeBSD, with a fine-tuned LoRA adapter, running on the same kernel as the pf that filters traffic, and driving the firewall’s REST API directly from a natural-language intent. The PoC works — 101 out of 102 functions validated, latencies measured — and it’s precisely because it works that its own conclusions matter: its docs list, methodically, four reasons why this topology must never go to production.

Attack surface. OAF itself puts a number on the cost: the added llama.cpp/ggml/OpenBLAS stack weighs roughly 125 MB of extra code to audit, patch and monitor, on a minimal OPNsense base of about 600 MB — a roughly 20% increase in the code surface of a device whose doctrine, historically, is precisely to stay minimal. Every CVE in that stack becomes a CVE of the firewall itself.

CPU contention. Inference measured by OAF takes about 10 seconds on the VM’s cores — the same cores that run pf. During that time, filtering continues, but under contention, and the data plane’s network latency feels it directly. A firewall normally has no reason for its filtering capacity to vary depending on whether a language model happens to be “thinking.”

Coupled lifecycle. llama.cpp evolves on a weekly cadence; OPNsense ships stable releases at a much slower pace. Embedding the former inside the latter forces two incompatible update cadences to cohabit in the same deployment artifact — every update to one becomes a regression risk for the other.

Compromised audit. When an LLM modifies config.xml directly, it isn’t, in compliance terms, an identifiable actor in the same way a named administrator going through an audit console is. OAF itself notes that the existing guardrail at this level (--confirm on the agent side, local log) is an operator control, not a legal audit trail nor a rampart against malicious action.

These four points aren’t an external critique leveled at OAF: they are its own conclusions, obtained by pushing the PoC all the way through so they could be measured rather than assumed. It’s precisely that work that makes the opposite conclusion — the one cyber-agent-engine implements — actionable rather than dogmatic.

The external answer: nothing gets inside the box

cyber-agent-engine inverts the topology term for term. The component that “thinks” — the CoordinatorLLM in coordinator/llm/coordinator_llm.py, which proposes intents from tokenized context — never runs on the network equipment it drives. It runs on the coordinator’s host, whatever that host is, and only talks to the firewall through an HTTPS call to its REST API, over the LAN. Term for term, OAF’s four objections dissolve:

• Attack surface: the firewall gains no code, no dependency, no extra binary. It stays the OPNsense device it always was, queried by an external HTTP client — exactly the same kind of traffic as a monitoring dashboard or an ordinary automation script.
• CPU contention: inference consumes the CPU/GPU cycles of the coordinator’s host, never the ones that filter network traffic. pf shares nothing with the LLM.
• Lifecycle: the coordinator deploys, updates and restarts independently of the OPNsense firmware. Updating the reasoning backend implies no maintenance window on the firewall.
• Audit: every call to the OPNsense API goes through the boundary described in articles 1 and 3 — fail-closed policy, human approval on sensitive capabilities, bounded audit log, tokens rather than real values. The actor modifying the configuration is no longer an opaque process inside the firewall: it’s an authenticated HTTP call, traced end to end, where every decision left an audit entry before execution.

The firewall becomes again what it should always be: a network device that exposes an API, nothing more. All the intelligence — and all the risk it carries — moves to the other side of the wire.

Portability: a GPU-free core, interchangeable backends

Taking the LLM out of the box only has value if the component that hosts it isn’t itself pinned to a particular machine. That’s the second benefit of the external topology, and it reads directly in pyproject.toml: the package’s base dependencies —

dependencies = [
    "pydantic>=2.0.0",
    "httpx>=0.25.0",
    "pyyaml>=6.0",
    "jinja2>=3.0.0",
    "cryptography>=42.0.0",
    "fastapi>=0.110.0",
    "requests>=2.31.0",
    "anthropic>=0.40.0",
    "uvicorn>=0.30.0",
]

— contain neither torch, nor vllm, nor any model weights. torch, vllm and unsloth are isolated in an optional extra, [gpu]:

[project.optional-dependencies]
gpu = [
    "torch>=2.1.0",
    "vllm>=0.6.0",
    "unsloth",
]

clients/gpu.py makes this boundary physical in the code: the in-process vLLM loader is never imported at module load time, only on demand, and if the [gpu] extra isn’t installed, the failure is an explicit GpuExtraRequired rather than an opaque ImportError in the middle of a stack trace. The coordinator’s core — policy, tokenization, gated loop, audit — installs and runs without ever touching a graphics card.

The corollary of this lightweight core is that the reasoning backend becomes a parameter, not a structural dependency. coordinator/llm/coordinator_llm.py states it unambiguously:

Backends configurables via COORDINATOR_BACKEND :
  - "anthropic"      : Claude API (claude-sonnet-4-6) — recommandé, aucun service local requis
  - "openai"         : API OpenAI-compatible — fonctionne aussi avec vLLM HTTP (port 8000)
  - "vllm"           : NativeVLLMClient direct (instance séparée, charge Qwen2.5-7B)
  - "ollama"         : Ollama local (si disponible)

(Configurable backends via COORDINATOR_BACKEND: "anthropic" — Claude API, recommended, no local service required; "openai" — OpenAI-compatible API, also works with vLLM HTTP on port 8000; "vllm" — direct NativeVLLMClient, separate instance loading Qwen2.5-7B; "ollama" — local Ollama, if available.)

And in the code, the selection is a simple branch on an environment variable, defaulting to anthropic:

COORDINATOR_BACKEND = os.getenv("COORDINATOR_BACKEND", "anthropic")

async def init(self) -> None:
    """Initialise le backend LLM sélectionné."""
    if self._backend == "anthropic":
        self._init_anthropic()
    elif self._backend == "openai":
        self._init_openai_client()
    elif self._backend == "vllm":
        await self._init_vllm()
    else:  # ollama
        self._http = httpx.AsyncClient(
            base_url=OLLAMA_BASE_URL,
            timeout=httpx.Timeout(120.0),
        )

Four paths, a single public interface (chat(messages, max_tokens)), and none of the coordinator’s three other callers — proposer, loop, audit — needs to know which one is active. In practice, that means a deployment can start on the Anthropic API (zero local service, latency of a few seconds, as mentioned in an earlier article on the OpenRouter/LoRA trade-offs (fr) on this blog), then switch to a self-hosted vLLM or a local Ollama for sovereignty or cost reasons, without touching a single line of core/, the policy, or the gated loop. Reasoning is a replaceable module; structural trust is not.

This portability also rests on the fact that cyber-agent-engine doesn’t depend on a private training pipeline to function. The public repo this whole article is built on is self-sufficient: coordinator/, core/, clients/ and agents/ are the only modules the reasoning depends on, and none of them import any proprietary LoRA training package. Manufacturing the LoRA adapters — when the vllm backend is chosen with a fine-tuned model anyway — is the subject of a separate, decoupled repo: the coordinator knows nothing about how a weight was produced, it consumes a generic chat interface. It’s this decoupling, as much as the absence of a GPU in the core, that makes the backend choice reversible rather than fixed at design time.

Targeting a real firewall: a REST client like any other

Concretely, how does the coordinator reach the equipment? Not through a homegrown protocol, nor through any privileged access — through a standard HTTP client, OPNsenseAPIClient (clients/opnsense_api_client.py), whose constructor hides no magic:

def __init__(
    self,
    base_url: str,
    api_key: str,
    api_secret: str,
    verify_ssl: bool = True,
    timeout: int = 30
):
    self.base_url = base_url.rstrip('/')
    self.api_key = api_key
    self.api_secret = api_secret
    self.verify_ssl = verify_ssl
    self.timeout = timeout
    self.client = httpx.AsyncClient(
        base_url=self.base_url,
        auth=(api_key, api_secret),
        verify=verify_ssl,
        timeout=timeout
    )

auth=(api_key, api_secret): HTTP Basic authentication, the same the OPNsense REST API natively expects — nothing specific to this project. verify_ssl stays an explicit parameter, never implicitly disabled in the client’s own code.

On the agent-server side, server.py reads four environment variables to build this configuration:

OPNSENSE_URL    = os.getenv("OPNSENSE_URL", "https://192.168.1.1")
OPNSENSE_KEY    = os.getenv("OPNSENSE_API_KEY")
OPNSENSE_SECRET = os.getenv("OPNSENSE_API_SECRET")

"verify_ssl": os.getenv("OPNSENSE_VERIFY_SSL", "False").lower() == "true",

OPNSENSE_URL has a code default of https://192.168.1.1 — the most common LAN gateway of a freshly installed OPNsense, not a secret, just a reasonable default to override. OPNSENSE_API_KEY and OPNSENSE_API_SECRET have no default: without them, OPNSENSE_KEY stays None and the OPNsense agent switches to simulation mode rather than failing silently. OPNSENSE_VERIFY_SSL defaults to False — consistent with the self-signed certificate a freshly deployed OPNsense admin interface exposes on its LAN, to be overridden to true as soon as a trusted certificate is in place.

The project’s README documents this section under the heading “Targeting a real OPNsense (interop)”, with the same variable table and the same recommendation: reachability is verified by a GET /api/core/system/status, and it’s exactly this endpoint that server.py probes at startup to confirm the connection works before marking the OPNsense agent as operational. The README goes as far as explicitly naming OAF as a possible test target for anyone wanting to experiment with this external architecture without standing up an OPNsense VM from scratch — the loop closes: the project that measured why the LLM must not live inside the firewall also serves as the test bench for the topology that makes it live next to it.

The non-negotiable caveat

Pointing the coordinator at a real OPNsense widens the exposure surface of exactly one thing: the REST API itself. Hence a rule that isn’t a suggestion but a deployment condition: the OPNsense API must be exposed on the LAN interface only, never on the WAN, and a firewall rule must explicitly authorize the host running the coordinator to reach that port — not a generous any, a named rule. It’s the same least-privilege discipline the rest of the architecture applies to the LLM’s reasoning; it applies just as strictly to the network connectivity that links it to the equipment it drives. Taking the model out of the box waives nothing on the network side — it simply moves the perimeter to protect from an embedded process to a LAN HTTPS path, easier to isolate, log and cut off.

flowchart LR
  subgraph Box["OPNsense VM"]
    API["OPNsense REST API — LAN interface"]
    inbox["in-box approach (OAF)<br/>LLM inside the firewall"]
  end
  Coord["cyber-agent-engine<br/>off-box coordinator, on the LAN"] -->|"HTTPS LAN + key/secret"| API
  Coord -->|"LLM backend (off-box)"| LLM["Anthropic / OpenAI-compat / vLLM / Ollama"]
  inbox -.->|"CPU contention, attack surface"| API

What this article changes in how the series should be read

The previous articles showed a trust boundary internal to the coordinator — tokens, policy, approval. This one adds a topological boundary: the LLM itself sits on the other side of a network call, never colocated with the equipment it governs. The two boundaries complement rather than replace each other — taking the model out of the box waives none of the guarantees described in articles 2 and 3, and conversely, a perfectly designed fail-closed core/ doesn’t make an inference running inside the firewall it drives any less consequential. It’s the combination of both that makes the whole thing operable in production — the subject of the next article in this series.

This article is part of the cyber-agent-engine series. The first article lays out the five principles of the trust boundary; an upcoming article will show how these building blocks — policy, gated loop, approval, audit, and now the off-box topology — assemble into a system operable in production.

To go further:

• opnsense-ai-firewall (fr) — the in-box PoC whose four measured limits (attack surface, CPU contention, lifecycle, audit) directly motivate the external architecture described here.
• Article 1 — trust core — the series' general thesis and the role of the fail-closed core/.