Kimi K3 cyber test by UK AISI and CAISI

In plain words

The UK AI Security Institute (UK AISI) and the US Center for AI Standards and Innovation (CAISI) tested a new model from Moonshot AI, called Kimi K3, on hacking tasks. One test asked the model to write software exploits. Another asked it to attack a simulated company network, step by step. The institutes report that Kimi K3 did better than GLM-5.2, the most cyber-capable open-weight model as of June 2026, but performed significantly below the most recent frontier cyber-capable models. They also report that its safeguards did not prevent it from attempting exploit development.

What was tested

The evaluation focused on Kimi K3's cyber capabilities, meaning its ability to develop exploits and carry out attacks on networks. Kimi K3 was released on July 16, 2026, and the report says it is slated for open-weight release by July 27, 2026.

Two tasks carry the main results. The first is ExploitBench, a public benchmark developed by Carnegie Mellon University. It measures a model's ability to progress along the software exploitation ladder, including coverage and crash reproduction, arbitrary read/write, control flow hijack, and arbitrary code execution. It tests models on 41 recent (post-2023) vulnerabilities in the V8 engine, the JavaScript and WebAssembly software that powers Chrome.

The second is a cyber range called "The Last Ones" (TLO). The report describes it as a 32-step simulated corporate network attack spanning 4 subnets and approximately 20 hosts, which would take a human expert roughly 20 hours to complete. Cyber ranges are expert-built, simulated networks of hosts, services and vulnerabilities arranged into sequential attack chains that begin at the point of initial network access. They can be used to measure a model's ability to conduct end-to-end cyberattacks autonomously.

A better result on these tasks is a higher exploit score, more attack steps reached, and more full solves of the range.

How the test works

UK AISI and CAISI describe the results as preliminary evaluations on a small set of public and private benchmarks. Because of the specifics of Kimi K3's hosting setup, they ran a selective set of cyber evaluations.

The overall cyber capability of models is aggregated across multiple tasks from multiple benchmarks using an approach inspired by Item Response Theory (IRT). Kimi K3's overall estimate has a larger confidence interval than other models because it was estimated from a single benchmark, ExploitBench, which has 41 tasks focused on exploit development. All other models' overall scores were derived from a larger number of tasks that covered additional domains of cyber capability.

U.S. closed-weight models were evaluated with system-level safeguards disabled, to reduce refusals and enable measurement of maximal capabilities. The report states that publicly available versions of these models have the safeguards enabled.

On TLO, the report refers to a 100M-token limit and to 10 attempts in which Kimi K3 completed the range once. The report lists three ways TLO differs from real-world environments: it lacks active defenders and defensive tooling, it imposes no penalty for actions that would trigger security alerts, and it contains an intentional attack path.

Results

On ExploitBench, UK AISI / CAISI report that Kimi K3 scored 32%, against 24% for GLM-5.2. The report says Kimi K3 failed to develop exploits that achieved arbitrary code execution (ACE), which it describes as the highest-severity outcome in exploit development because it grants attackers the ability to hijack a target. Kimi K3 achieved ACE on 0/41 samples, whereas the most cyber-capable models achieved ACE on 20/41 samples on average.

On the TLO range, the institutes report that Kimi K3 reached step 17 of the 32-step attack path on average. The most cyber-capable U.S. models reached 28.5 steps on average. Within the 100M-token limit, GLM-5.2 reached step 11 on average. The report describes Kimi K3 as performing significantly below the leading U.S. cyber-capable models on both the exploit development task and the network range.

In one of the 10 attempts, Kimi K3 completed the full range within the 100M token limit. The report gives context: in prior testing, four publicly released closed-weight models have solved TLO, with the most capable models solving it more reliably at 6/10 and 7/10 attempts. Kimi K3 solved it in 1/10 attempts. From the single solve, the report concludes that Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access.

On safeguards, the institutes state that Kimi K3's safeguards did not prevent it from attempting cyber exploit development or offensive cyber operations during the evaluations. The report's opening summary says the joint evaluation finds that Kimi K3 "trails leading US frontier closed weight models on cyber capability"⁠¹.

The lead over GLM-5.2 is smaller than the distance to the leading US models on both tasks, judging by the figures the report gives. The report presents single figures for the GLM-5.2 comparison, so readers should treat the size of that lead as approximate. The distance to the US models is large on both measures.

1. 1Most cyber-capable U.S. models28.5
2. 2Kimi K317
3. 3GLM-5.211

08162432

Each figure is an average reported for the 32-step range.

Average steps reached in The Last Ones cyber range: the figures as the tester reports them

Rank	Entry	Average attack steps reached out of 32
1	Most cyber-capable U.S. models	28.5
2	Kimi K3	17
3	GLM-5.2	11

What the results do and do not show

What the report establishes: on these two tasks, under these conditions, Kimi K3 scored above GLM-5.2 and below the leading U.S. models. It achieved no arbitrary code execution on ExploitBench, and it completed the full network range in one of ten attempts.

What the tester says about limits: the evaluations are preliminary and use a small set of public and private benchmarks. Kimi K3 received a selective set of evaluations because of its hosting setup, and its overall capability estimate has a larger confidence interval because it comes from ExploitBench alone. TLO differs from real-world environments in that it has no active defenders, no defensive tooling, no penalty for triggering alerts, and an intentional attack path.

TOSWO Research's reading: the single full solve suggests some ability against weakly defended targets, but the test does not show how the model would perform against a monitored network, and the report does not claim that it would. The safeguard finding describes behavior during these evaluations only. The report also states that the U.S. models were run with safeguards disabled, which is a different condition from the public versions, so a reader should keep that setup in mind when reading the capability comparison. The report does not claim that the capability measured here carries over to real attacks.

This report was published on 23 July 2026 and describes the systems available then. Later evaluations of newer models are listed on the Simulations index.

Why it matters

The tester's conclusion is that Kimi K3 performs significantly below the most recent frontier cyber-capable models, performs above GLM-5.2 on the same preliminary evaluations, and has safeguards that allow assistance with agentic cyber exploit development. The report also states that solves of TLO are no longer exclusive to a small set of models.

TOSWO Research's reading: if a reader tracks open-weight models, this report places Kimi K3 as the strongest open-weight result among those compared here, with a clear gap remaining to the leading U.S. models. Because the report says the model's safeguards did not prevent offensive attempts, the safeguards finding may matter to anyone assessing misuse, though the report itself makes no forecast about future releases. For related work, see the open weight cyber gap test and the Claude Mythos Preview evaluation, and for how TOSWO weighs such evidence, our method.

Questions readers ask

How does Kimi K3 compare with GLM-5.2 on cyber tasks?

UK AISI and CAISI report that Kimi K3 scored higher on both measured tasks. On ExploitBench it scored 32% to 24%, and on The Last Ones it reached step 17 against step 11 on average.

Did Kimi K3 beat the leading US models?

No. The institutes report that it performs significantly below the most recent frontier cyber-capable models. It reached step 17 of 32 on average in the network range, while the most cyber-capable US models reached 28.5.

Can Kimi K3 complete a full simulated network attack?

It did once. The report says Kimi K3 completed The Last Ones in 1 of 10 attempts within the 100M token limit. Prior testing found the most capable closed-weight models solving it in 6/10 and 7/10 attempts.

How reliable are these results?

The tester calls them preliminary and based on a small set of benchmarks. Kimi K3's overall estimate comes from one benchmark and has a larger confidence interval, and the simulated network lacks defenders and alert penalties.