Frontier AI agents on multi-step cyber ranges, AISI test

In plain words

The UK AI Security Institute (AISI) asked whether AI agents can carry out a long chain of computer attacks with little human help. It built two simulated networks and let seven language models try to break through them step by step. AISI reports that newer models got much further than older ones, and that giving a model more computing budget helped it go further still. On a simulated power plant, though, models made only limited progress. The test matters because it measures extended attack sequences, not isolated puzzles.

What was tested

AISI built two "cyber ranges", which are simulated network environments with multiple hosts, services and vulnerabilities arranged into sequential attack chains, built by cybersecurity experts. AISI says existing cyber evaluations rely on isolated capture-the-flag challenges or question-answer sets, and that these approaches do not capture whether AI systems have the autonomous, long-horizon capabilities needed for extended attack sequences in complex environments.

The first range, "The Last Ones", is a 32-step attack on a corporate network. The attacker has to exfiltrate sensitive data by moving through the enterprise, stealing credentials, exploiting web applications, reverse engineering binaries, compromising CI/CD pipelines and executing SQL injection chains across a multi-domain corporate network. AISI estimates a human expert could complete it in approximately 14 hours.

The second, "Cooling Tower", is a 7-step attack on an industrial control system. The attacker must disrupt a simulated power plant's cooling tower by reverse-engineering a proprietary control protocol to craft malicious commands. Each step is a substantially larger unit of work with more complex dependencies, and AISI estimates approximately 15 hours for a human expert.

A better result is more steps completed. The question behind the test, as AISI frames it, is whether reliable multi-step attack chains with minimal human oversight could lower the skill barrier for unsophisticated threat actors, increase the sophistication of attacks by experienced ones, or enable entirely novel offensive operations.

How the test works

AISI compared seven models released over an eighteen-month period, from August 2024 to February 2026, at varying inference-time compute budgets. Inference-time compute is the total number of tokens a model spends reasoning and acting during an attempt. The budgets named in the post are 10M and 100M tokens. The main measure is the average number of steps completed on a range.

Neither range has active defenders. Detections are logged but do not block or slow the agent, so AISI says the results measure raw capability in the absence of defensive response.

AISI describes a fix made during the work. In initial testing, models occasionally made progress through approaches not anticipated in range design. On "Cooling Tower" the intended path was to compromise a web application and then use reverse-engineered cryptographic material to gain access to programmable logic controllers (PLCs). Some models bypassed that sequence. Starting from the attacker's initial position, they probed the proprietary PLC protocol directly, deduced enough of its structure from network traffic to call unprotected functions, and read PLC memory without prior exploitation. AISI patched this issue for its main runs.

The model names AISI gives in this post are GPT-4o, Opus 4.5, Opus 4.6 and GPT 5.3 Codex. AISI points readers to its full paper for more detailed discussion.

Results

AISI reports that each successive model generation outperformed its predecessor at fixed token budgets. On "The Last Ones" at 10M tokens, the average rose from 1.7 steps for GPT-4o (August 2024) to 9.8 for Opus 4.6 (February 2026). At 100M tokens, Opus 4.5 averaged 11.0 steps and Opus 4.6 averaged 15.6, which AISI describes as a 42% improvement between models released roughly two months apart.

AISI offers two likely explanations for the generational gains, and presents them as interpretation. One is token efficiency, how quickly a model makes progress per token. The other is capability depth, whether a model has strong enough specialist skills, such as reverse engineering, cryptography or exploit development, to overcome hard steps. AISI says GPT-4o plateaus entirely after step 2. Performance drops sharply after milestone 4, where the task moves from reconnaissance and web exploitation to phases needing specialist knowledge, and AISI says Opus 4.6 is the first model to consistently push past this barrier.

More compute also helped. AISI reports that performance on "The Last Ones" scales log-linearly with inference-time compute, and that raising the budget from 10M to 100M tokens yields gains of up to 59%, with no observed plateau. AISI notes that this matters because anyone can simply give a model more tokens, unlike custom scaffolding, expert prompting or tailored tooling. At current pricing, it puts a 100M-token attempt with Opus 4.6 at approximately $80 USD.

AISI also reports strong variance across individual attempts. The best-performing Opus 4.6 run completed 22 of 32 steps, reaching milestone 6, and AISI says this corresponds to roughly 6 of the estimated 14 hours a human expert would need. Other runs with the same model and budget completed substantially fewer steps.

On "Cooling Tower" the picture is different. At 10M tokens most models complete zero steps. At 100M tokens Opus 4.6 achieves the highest average of 1.4 steps (max 2), while GPT 5.3 Codex achieves a single-run maximum of 3 out of 7. AISI attributes the limited progress to high per-step difficulty combined with the need to carry forward information gathered in earlier steps.

The step averages on the corporate range differ widely between the oldest and newest models. Because AISI reports strong variance between attempts, a reader should treat single-run figures as illustrations and not as a ranking.

What the results do and do not show

AISI states what the test measures: raw capability on two ranges with no active defenders. The post does not report results against defended networks. AISI lists as possible future work building a broader suite of ranges and validating performance in real-world settings featuring active defenders.

The results come from two ranges, one corporate and one industrial. Two scenarios do not establish general cyber ability. The strong variance across attempts also limits what any single figure can say. The "Cooling Tower" results reflect the range after AISI patched an unintended shortcut for its main runs.

AISI separates measured findings from its reading. The step averages, token budgets and gains are measured. The two explanations for generational improvement are described as "likely", so they are hypotheses. The point that scaling compute requires no specific technical sophistication from the operator is AISI's argument about implications.

TOSWO Research's reading: AISI reports no observed plateau between 10M and 100M tokens, but it gives no figures beyond 100M tokens, so the post does not show what larger budgets would do. Separately, because the ranges have no defenders, the scores may not carry over to defended environments, a point AISI's own call for testing with active defenders is consistent with.

This report was published on 16 March 2026 and describes the systems available then. Later AISI evaluations of newer systems are covered in the Claude Mythos Preview cyber evaluation, the open weight models gap test and the Kimi K3 cyber test, so the findings above should not be read as the current state of the field.

Why it matters

AISI concludes that cyber capabilities are improving rapidly, making continuous, rigorous testing essential, and that the sophistication of evaluation environments needed to measure them is rising in tandem. It says cyber ranges will form a crucial part of its approach going forward. It also says its compute finding echoes previous AISI findings on the relationship between cyber capabilities and inference scaling, and that it extends earlier work on isolated tasks to multi-step attack chains.

TOSWO Research's reading, conditional on the figures above: if progress on an attack chain depends partly on token budget, a score reported without its budget may be hard to interpret. Readers comparing this work with other evaluations can look at our simulations index, and at related cyber tests such as the UK AISI cyber evaluation of Claude Mythos Preview or the open weight models gap test. The way TOSWO treats such evidence is set out in our method page.

Questions readers ask

Can AI agents carry out a full cyber-attack on their own?

In AISI's tests, not fully. The best single run on the 32-step corporate range completed 22 steps, and on the 7-step power plant range AISI says models made only limited progress. The ranges had no active defenders, so the test measures raw capability only.

How much did more compute help?

AISI reports that raising the budget from 10M to 100M tokens gave gains of up to 59% on "The Last Ones", with no observed plateau. It estimates a 100M-token attempt with Opus 4.6 costs approximately $80 USD. Variance between individual attempts was strong.

Which model did best?

On the corporate range Opus 4.6 averaged 9.8 steps at 10M tokens and 15.6 at 100M tokens, the highest figures AISI gives. On the power plant range Opus 4.6 had the highest 100M-token average of 1.4 steps, while GPT 5.3 Codex had a single-run maximum of 3 out of 7.

What is a cyber range?

AISI defines it as a simulated network environment with multiple hosts, services and vulnerabilities arranged into sequential attack chains, built by cybersecurity experts. It differs from a single capture-the-flag puzzle because success needs many linked steps.