🇫🇷 Version française 💻 Source code (AGPL): github.com/patlegu/cyber-agent-engine

The invariant

The first article in this series laid out the general thesis of cyber-agent-engine: an LLM that drives a production firewall must never cross, on its own, the boundary between intent and execution. This one goes deep into the most counter-intuitive pillar of that architecture, and probably the simplest to state: the reasoning LLM never sees a real IP, a real hostname, or a real secret. It only ever sees tokens — strings of the form IP_ADDRESS_1, HOSTNAME_2, CVE_3. The real values stay server-side, in a table that is never serialized out to the model, and only reappear at the very last moment: at the execution boundary, when an action is actually sent to the targeted device.

This isn’t an implementation detail. It’s an architectural decision that changes what the LLM can do — even if it is compromised, even if it hallucinates, even if it falls victim to a prompt injection.

How: tokenize, Vault, detokenize

The mechanism lives in core/tokens/vault.py. The file opens by explicitly stating its intent:

"""Tokenisation réversible des valeurs sensibles, liée à la session.

Le LLM et les logs ne voient QUE des jetons (``IP_1``, ``VPN_USER_2``). La table
jeton→valeur (le ``vault``) reste côté serveur et n'est jamais sérialisée hors de
celui-ci. La détokenisation n'a lieu qu'au tout dernier moment (cf. ``execution/``)
ou dans la vue d'approbation humaine.
"""

In short — the repo’s docstrings stay in French per project convention — this says: reversible tokenization of sensitive values, tied to the session; the LLM and the logs see ONLY tokens (IP_1, VPN_USER_2); the token→value table (the vault) stays server-side and is never serialized out of it; detokenization only happens at the very last moment (see execution/) or in the human-approval view.

The core of the mechanism is a Vault class, which embodies a single- session token↔value bijection — no state is shared between two sessions, so no cross-leak between two operators or two runs is possible by construction:

class Vault:
    """Bijection jeton↔valeur pour UNE session. Aucun état partagé entre sessions."""

    def __init__(self) -> None:
        self._to_real: dict[str, str] = {}
        self._to_token: dict[str, str] = {}
        self._counters: dict[str, int] = {}

    def token_for(self, label: str, value: str) -> str:
        existing = self._to_token.get(value)
        if existing is not None:
            return existing
        self._counters[label] = self._counters.get(label, 0) + 1
        token = f"{label}_{self._counters[label]}"
        self._to_real[token] = value
        self._to_token[value] = token
        return token

    def resolve(self, token: str) -> str | None:
        return self._to_real.get(token)

Two points in this code deserve emphasis, because they determine everything that follows.

First, the real token format: it’s f"{label}_{counter}", where label is the detected entity type (IP_ADDRESS, HOSTNAME, CVE…) and counter an integer that increments per label, scoped to the session. A given IP therefore becomes, say, IP_ADDRESS_1, the next one IP_ADDRESS_2, and a distinct hostname HOSTNAME_1 — counters never mix across labels. (The sequence diagram further down, taken from this series’ plan, simplifies the label to IP for readability; in the real code, it’s IP_ADDRESS that actually comes out of the extractor — more on that in the next section.)

Second, token_for is idempotent per value: existing = self._to_token.get(value) means that if the same IP reappears twice in the same session, it gets exactly the same token. That’s what lets the LLM correctly reason about “the same IP shows up three times in these logs” without ever knowing which one — referential coherence in the reasoning is preserved, only the value is masked.

Two free functions orchestrate the round trip around this vault:

def tokenize(text: str, vault: Vault, extract: ExtractFn) -> str:
    """Remplace chaque entité sensible détectée par son jeton stable de session."""
    entities = extract(text)
    pairs: list[tuple[str, str]] = []
    for label, values in entities.items():
        for value in values:
            if value:
                pairs.append((label, value))
    for label, value in sorted(pairs, key=lambda p: len(p[1]), reverse=True):
        text = text.replace(value, vault.token_for(label, value))
    return text

tokenize takes a raw text (typically the operator’s request or a log excerpt), an extract: ExtractFn — a text-to-dict-of-labels-to-value- lists function — and replaces each detected value with its token. The descending-length sort (key=lambda p: len(p[1]), reverse=True) isn’t cosmetic: it prevents a value that is a substring of another (for instance an IP contained in a longer CIDR) from being replaced first and corrupting the next replacement.

Symmetrically, detokenize walks back the other way, recursively over Python structures (str, dict, list):

def detokenize(obj: Any, vault: Vault) -> Any:
    """Remplace récursivement les jetons ÉMIS par le vault par leurs valeurs réelles.

    On ne remplace que les jetons que ce vault a effectivement produits (pas de
    reconnaissance par forme) : rien qui ressemble à un jeton mais n'a pas été émis
    n'est jamais touché.
    """

That detail — “no pattern-based recognition” — is essential from a security standpoint. detokenize doesn’t pattern-match on strings that merely look like LABEL_N: it only replaces tokens present in vault.items(), i.e. the ones this exact vault actually emitted during this session. A compromised or hallucinating LLM that invented a string IP_ADDRESS_99 never emitted by the vault therefore triggers no resolution — it passes through detokenize unchanged, like any other piece of text.

That leaves the question of who detects the entities to tokenize. That’s the role of ExtractFn, a str -> dict[str, list[str]] function implemented by default in coordinator/extractor.py.

The regex extractor

build_regex_extractor() returns a pure function, with no heavy dependency (no embedded NER model), calibrated for “precision over recall”:

def build_regex_extractor() -> ExtractFn:
    """Renvoie un extracteur pur : texte → {label: [valeurs uniques, ordre stable]}."""

    def _extract(text: str) -> dict[str, list[str]]:
        remaining = text
        result: dict[str, list[str]] = {}
        for label, pattern in _PATTERNS:
            found = pattern.findall(remaining)
            result[label] = _dedupe(found)
            if found:
                remaining = pattern.sub(" ", remaining)
        return result

    return _extract

The returned dict maps each label to the deduplicated (stable-order) list of values found for that label. The seven labels actually recognized, in matching order (most specific first), are defined by _PATTERNS:

• IP_SUBNET — CIDR (203.0.113.0/24), matched before IP_ADDRESS so a subnet doesn’t also re-emit its bare IP.
• MAC_ADDRESS — matched before IP_ADDRESS, because a MAC (6 hex groups separated by :) also matches the generic IPv6 pattern.
• IP_ADDRESS — IPv4 and IPv6 (full, mid-compressed, and leading- compressed forms), bounded by explicit lookarounds rather than \b (the code comment explains why: ::1 doesn’t present an exploitable \b boundary).
• CVE — the CVE-\d{4}-\d{4,7} pattern.
• HASH — case-insensitive hex, at MD5/SHA1/SHA256 lengths (32, 40, or 64 characters).
• HOSTNAME — generic FQDN.
• PORT_NUMBER — a 2-5 digit integer preceded by : (lookbehind).

Each pattern “consumes” the text as it goes (remaining = pattern.sub(" ", remaining)), which guarantees that a segment already captured by a more specific label isn’t re-detected under a more generic one — a CIDR never also shows up as IP_ADDRESS, an FQDN never shows up as anything else.

The file itself flags the existence of an alternative:

Une variante spaCy (``NERExtractor``) reste disponible via l'extra [ner] pour le
NL riche, mais n'est pas câblée par défaut.

In short: a spaCy variant (NERExtractor) remains available via the [ner] extra for richer natural language, but isn’t wired in by default. build_regex_extractor() is therefore the default ExtractFn, but the ExtractFn interface (a plain Callable[[str], dict[str, list[str]]]) is deliberately generic: you can plug in a richer extractor without touching Vault or tokenize/detokenize.

The flow, in sequence

The diagram below illustrates the full path of a request, from the operator down to actual execution on the device — keeping in mind that the label shown here (IP) is a pedagogical simplification of the real label the extractor produces (IP_ADDRESS):

At no point in its reasoning — not in the prompt, not in its chain of thought, not in the function it proposes to call — does the LLM ever handle the real value 203.0.113.7. It handles an opaque identifier, stable for the duration of the session, sufficient for it to reason (“block this IP”, “the same IP shows up in three different logs”) but giving it nothing to exfiltrate.

Why it matters

Reduced blast radius from a prompt injection

The dreaded scenario with an LLM agent wired to network tools is prompt injection: malicious content — slipped into a log, a User-Agent, a DNS comment — manipulates the model into executing an unwanted action, or into revealing sensitive data in its response. With systematic tokenization applied upstream, the second half of that threat loses much of its substance: the LLM doesn’t know the real value an attacker would try to make it exfiltrate. At best it can return a token — a string like IP_ADDRESS_7, worthless outside this session’s vault, useless outside the system. This isn’t a claim that tokenization neutralizes prompt injection as a vulnerability class (the first article in this series already laid out why core/ keeps a separate, downstream control over what the LLM is authorized to do) — but it mechanically reduces what a successful injection can leak.

Data sovereignty

cyber-agent-engine can route reasoning to external LLM providers — that’s a deliberate architectural choice made elsewhere in the project. That immediately raises a sovereignty question: which pieces of an operator’s infrastructure data transit, in plaintext, to a third party? With tokenization applied at the entry point, the answer is structurally tight: no real IP, no real hostname, no secret ever leaves the server perimeter toward a third-party-hosted model — whether that’s OpenRouter, a cloud provider, or any other exit point of the operator’s network. This isn’t a declarative policy one hopes a prompt instruction will enforce; it’s a property of the pipeline: the text that leaves the server process to reach the model’s API has already gone through tokenize().

Token-only audit

A direct consequence: the logs and reasoning traces produced by the LLM can be kept, replayed, shared with an external security team, or archived long-term without themselves constituting an infrastructure inventory. A log line reading block IP_ADDRESS_3 on HOSTNAME_1 tells nothing to anyone without access to that specific session’s vault — and the vault, as we’ve seen, only survives for the duration of the session, server-side. That opens the door to a subject this series will dig into later: an exploitable audit trail, built entirely out of tokens, still useful for forensics (“the same IP came back five times”) without ever being a data leak in itself.

Lineage: the cousin on the logs side, Victor/AnonyNER

This principle — never letting sensitive data reach an external process in its real form — isn’t unique to cyber-agent-engine. An earlier project on this blog, Victor, the security-log anonymizer (fr), poses a variant of the same problem on the logging side: how to anonymize security logs before sharing them (with a third party, an analysis tool, an external LLM) without depending on an external service to do it. Victor combines a NER engine (spaCy) and regex rules, with a self-learning loop and local validation by a small language model — a richer approach than cyber-agent-engine’s deterministic regex tokenization, designed for free-form text rather than structured action intents.

The two projects share the same underlying intuition: sensitive data should never be the price of admission for benefiting from AI-driven processing. coordinator/extractor.py explicitly acknowledges that kinship by keeping the door open to a spaCy variant (NERExtractor, available via the [ner] extra) — closer to Victor’s architecture — for the day the input text becomes too rich for deterministic regexes.

Honest limits

It would be dishonest to present this tokenization as an absolute guarantee. Three concrete limits, directly readable in the code:

• Coverage is closed and explicit. The default regex extractor only recognizes seven entity types: IP_SUBNET, MAC_ADDRESS, IP_ADDRESS, CVE, HASH, HOSTNAME, PORT_NUMBER. A device serial number, a user account name, an API key slipped into a free-text field, a sensitive file path: none of that is detected by these seven patterns. The file’s own comment says as much — “calibrated for precision over recall […] we’d rather not over-tokenize the noise” — that’s a deliberate design choice, not an oversight, but it means anything that doesn’t fall into these seven categories reaches the LLM in plaintext.
• A regex extractor has no semantic understanding. It detects shapes (a run of digits and dots, a hex pattern), not intent. An IP written with an unusual separator, a malformed hostname, or a sensitive value that doesn’t resemble any of these seven patterns slips through.
• detokenize guards against pattern-based recognition, not against missing upstream tokenization. detokenize’s guarantee — only replace tokens actually emitted — is solid, but it doesn’t compensate for an extractor that missed a value on the way in. The system is only as robust as its extractor, no more.

So this isn’t magic: it’s a deterministic filter, with a defined and documented scope, that covers the central use case of cyber-agent-engine well — the network entities that flow through operator intents and device logs — without claiming universal detection of every conceivable piece of sensitive data.

What’s next

This token only makes sense because, on the other side, there’s a core/ capable of deciding what an intent expressed in tokens is allowed to become as a real action. That’s the subject of the next articles in this series: how the trust core validates, logs, and executes — always without ever needing to see the real value behind the token.

To go further:

• An LLM with admin rights on your firewall: how not to get pwned — article 1 in this series, laying out the general thesis of the trusted core/.
• Victor, the security-log anonymizer (fr) — the cousin on the logs side: NER + regex, no external dependency.