🇫🇷 Version française 💻 Source code (AGPL): github.com/patlegu/cyber-agent-engine

Rigor as a deliverable

The five previous articles in this series described technical boundaries: tokens rather than real values, a fail-closed policy, an off-box topology, bounded audit, and reproducible packaging. This last article shifts angle. It is not about what cyber-agent-engine does, but about how it was built — and why that how is part of the deliverable just as much as the code.

The thesis is easy to state and harder to hold to: on a project where an LLM ends up with execution rights on a production firewall, quality cannot be a catch-up exercise. It gets proven at every step, or it does not get proven at all. That translated into a structuring choice made before the first line of core/ was written: break the product into independent sub-projects, each carried by its own spec → plan → execution chain, rather than advancing through opportunistic large commits on a single line of development.

Concretely, the repository contains a docs/superpowers/specs/ folder and its counterpart docs/superpowers/plans/, where each sub-project leaves a written trace before being coded. There you find, for example, 2026-07-22-coeur-confiance-surete-design.md (the foundation described in articles 1 and 2 of this series — policy, tokenization, audit), 2026-07-22-portabilite-modeles-runtime-design.md and 2026-07-22-assemblage-runtime-design.md (articles 4 and 5 — taking the LLM out of the box, assembling the multi-server runtime), or also 2026-07-23-durcissement-exploitation-design.md and 2026-07-23-ci-release-design.md, which cover operational hardening and the release chain. Each design has its corresponding execution plan in plans/ — a document broken into numbered tasks, with, for each one, the list of files touched, the expected interfaces, and the verification commands to run before committing. This very article series followed the same circuit: 2026-07-23-vitrine-blog-series-design.md then 2026-07-23-vitrine-blog-series-fr.md in plans/. The method does not apply only to the runtime; it also applies to its own showcase.

This split into sub-projects is not cosmetic. It sets a context boundary: a subagent implementing task 3 of the “trust core” plan does not need to know the details of the CI chain or Docker packaging to do its job well — it has the plan for its own sub-project, the current task, and the code already in place. That reduces drift: less context to carry, fewer shortcuts taken out of context-window fatigue.

The loop: implement, review, fix, review again

The core of the method is a loop repeated for every task in a plan. It reads as follows:

Each task in the plan is handed to an implementer subagent that starts on a fresh context — the plan, the task, the existing code, nothing more. That is deliberate: a context that accumulates over dozens of tasks ends up carrying more noise than signal, and a context-fatigued agent tends to cut corners a fresh agent does not.

Once a task is implemented, it goes through review — not the final review, a review per task, which checks two distinct things: conformance to spec (does what was coded match what the plan asked for, no more, no less) and intrinsic quality (naming, complexity, error handling, tests). When this review surfaces findings, they are not fixed by the agent that just wrote the code — a dedicated fix subagent picks up the findings one by one, applies the fixes, and the whole thing goes back through review. The loop closes once review has nothing left to flag, and the next task begins.

This implementer / reviewer / fixer separation borrows a classic code review principle — whoever writes the code is not best placed to judge whether it is good, because they have already rationalized their own choices while writing it. Applied to subagents, this principle becomes mechanical rather than a matter of discipline: review is a distinct role, invoked systematically, not a step that can be skipped on a day running behind schedule.

Once every task in a sub-project is closed, one last step remains before merge: a final branch review, deliberately adversarial. It does not re-read task by task — it takes the whole branch as one piece and actively looks for what might have slipped through the cracks of the local reviews: a regression introduced by task 7 against an invariant set in task 2, an inconsistency between two modules that each looked correct in isolation. It is the difference between proofreading a letter sentence by sentence and rereading it whole once finished: the two passes catch different things.

Test-first and enforcement guards at runtime

The review loop catches what a human (or an agent) knows to look for. It does not structurally catch drifts nobody thought to check on any given pass. That is why the project also relies on enforcement tests — tests that do not validate a business behavior, but a structural property of the code, by scanning it directly through its abstract syntax tree (AST) rather than eyeballing it.

Three concrete examples, all in tests/:

tests/test_spdx_headers.py checks that every first-party source file (core/, coordinator/, agents/, clients/, server.py) carries the SPDX-License-Identifier: AGPL-3.0-or-later header within its first three lines. The test walks the tree, collects the missing files, and fails with the precise list if even one is missing — no manual check to redo every time a subagent creates a new file.

tests/test_runtime_messages_english.py is more interesting because it does not look for a fixed string but for a structural pattern. The project requires that runtime operator-facing messages (whatever goes into a raise, into a logging call, into a print, or into the reason=/error= fields of a response) be in English. Rather than grepping for French words, the test parses each file as an AST, walks the Raise nodes and calls to logging methods (debug, info, warning, error, exception, etc.), and flags any string literal constant containing a French accented character in those specific positions. The file documents its own limit: French without accents (“timeout serveur”) is not detected — the test guards against regression of the common case, it does not replace the initial sweep that ensured completeness. It is an enforcement guard acknowledged as partial, not as total proof.

tests/test_lint_surface_consistency.py guards against a more insidious kind of regression: silent configuration drift. The list of paths covered by ruff check appears in three different places — ci.yml, release.yml, and [tool.mypy].files in pyproject.toml. Nothing mechanically stops someone (human or agent) from adding a module to core/ without adding it to all three. The test reads pyproject.toml as TOML and each workflow as YAML, extracts the ruff check command from each, and compares the set of paths against the mypy.files list. If any of the three drifts, the test fails — a silent path drop becomes a red test, not a coverage gap discovered six months later.

The common thread across these three guards: none of them wait for a human (or an agent) to remember to check a cross-cutting property. They turn it into an executable assertion, run on every pytest -q, including in CI. It is the mechanical complement to per-task review: review catches what someone thought to look for on that task; the enforcement test catches what nobody needs to re-think on every task, because the property is verified structurally across the whole repository.

What it buys

Taken together, these three mechanisms — spec → plan → execution sub-projects, an implementer/reviewer/fixer loop per task followed by an adversarial branch review, AST enforcement guards — aim at the same goal: high quality from the start, rather than a catch-up audit once the product is “done.” Three concrete effects follow.

First, a fresh context per task limits the drift that an overloaded context always eventually produces — less to hold in mind, fewer compromises made to move fast. Second, systematic review (per task and final) structurally separates writing from judging, rather than relying on the self-discipline of whoever just wrote the code. Finally, the AST guards make non-regression proven rather than hoped for on the handful of properties that matter most for this specific project: AGPL license respected file by file, English-only runtime surface, lint configuration consistency.

The limits, honestly

This method is not proof of correctness. It is a discipline that increases the probability of catching a problem before it reaches main, not a guarantee that no problem exists. Three limits deserve to be stated plainly rather than swept under the rug.

First, it costs iterations. A task that goes around the implementer → review → fix loop two or three times before being “clean” takes longer than a task coded once and judged good without verification. That cost is accepted — it is the price paid for not paying it later, as an incident — but it is real, and there is no reason it disappears with the project’s experience.

Second, the quality of the result depends directly on the quality of the starting plan and spec. A per-task review checks conformance to the plan; if the plan itself has a hole, a design blind spot, a false implicit assumption, the loop can produce code that is perfectly conformant to an imperfect plan. The adversarial final branch review exists in part to catch this case — looking at the whole rather than each brick — but it does not replace serious upfront spec work.

Finally, the AST guards only cover what someone thought to encode. The test on runtime messages says so itself in its docstring: it detects French accentuation, not the total absence of French phrasing. A guard protects one precise property against regression; it protects nothing against what nobody anticipated needed protecting. It is a method, not a guarantee — and saying so explicitly is part of the rigor it claims to bring.

This closes the series on cyber-agent-engine. The first article laid out the five principles of the trust boundary; the following articles detailed tokenization (article 2), the fail-closed policy (article 3), the off-box topology (article 4), and assembling it to operate with trust (article 5). This one closes the loop by showing how those five building blocks were themselves built.

Going further:

• Article 1 — an LLM with admin rights on your firewall — the general thesis this construction method had a mandate to hold end to end.
• Agents en Production — migration Qwen2.5 (fr) — the lineage of LoRA/agent experiments that preceded cyber-agent-engine, and where a good part of the upstream lessons in rigor were learned the hard way.