Back to blog

Research · September 27, 2026

Isolating the host stops the attack. It also stops the business.

Defensive agents have no reward signal. Here is how we build one a machine can check.

P

Peter

Co-Founder, SentryStack

Offensive agents have the cleanest training signal in security. Did the flag drop? Did the exploit land? Did the data leave? A script answers in milliseconds, the episode scores +1 or 0, and the model improves with no human in the room. That loop is why offensive capability is compounding faster than anything on the defensive side.

Ask the same question of a defender and it falls apart. Not because defense is harder to measure, but because it has two outcomes that have to hold at once.

The two-outcome problem

A defender has to stop the attacker and keep the mission running. Score only the first and you train an agent that pulls the plug, and pulling the plug always works. Isolate the host, revoke every credential, drop all egress, and the intrusion ends. So does payroll.

This is why defensive training has not scaled the way offensive training has. Every rollout needs a human to look at what the agent did and judge whether the cure was worse than the disease. That human is the bottleneck, and no amount of compute removes them.

One attack, several defenses, same starting state

The trick is not to score an action. It is to score the difference between actions taken against an identical world. We snapshot a compromised environment and branch it, running each candidate response from the same instant:

  1. No action. The baseline. The attack chain advances, and we record how far it gets.
  2. Isolate the host. The attacker is cut off. So is billing-api, the mission workload that happened to live there.
  3. Terminate the malicious process. The chain breaks at pid 31884 and the workload keeps serving traffic.

Only the third branch is a defense. The first two are a loss and a self-inflicted outage, and an agent trained on attacker-stopped alone cannot tell them apart.

What the verifier actually watches

Scoring is done by instrumentation inside the range, not by a judge model reading a transcript. It has privileged visibility that neither the attacker nor the defender has:

  • Attacker progress. How far along the chain the adversary got before the response landed.
  • Mission uptime. Whether the workload the environment exists to protect kept serving.
  • Collateral impact. What else the response broke: sessions killed, dependencies severed.
  • Remaining footholds. Persistence the agent believed it had removed and had not.

Those collapse into a scalar the policy can train on: R = detection + mitigation − cost, where mitigation is credited across the intermediate steps rather than paid out once at the end. An agent that contains an intrusion in four moves should not score the same as one that takes forty.

Why this is not a better simulator

  • Real chains, not CTF puzzles. Written by operators who spent careers building ranges for offensive testing, against hosts, identity, cloud and services that behave like production.
  • Scored by the range, not a model. A judge model introduces the noise we are trying to remove. The instrumentation already knows the answer.
  • Reset and replayed. Reproducible state is what makes counterfactual comparison valid, and what makes the whole thing run without a human.

Detection work asks whether an event is malicious. Patch gyms ask whether a bug reproduces. Neither asks the question that matters under a live adversary: which action breaks this chain without breaking the system it is defending?

Attack stopped. Mission intact.

Working on defensive RL?

We are talking to frontier labs, defense programs and operators.