Catastrophe through misuse
No misaligned AI is required: it is enough for an offensive capability to get cheaper. A person or group with access to a capable model skips steps that used to demand years of training, in biology or in network intrusion.
- Horizon
- 1–3 years
- Evidence
- Projected
- Consensus
- High
Assumptions that must hold
That the upliftUpliftHow much better a person does at something with an AI at their side, compared with what they would achieve alone. This observatory uses it mostly for dangerous capabilities.For exampleLike assembling furniture with a video tutorial beside you: it does not give you the tools, but you finish sooner and with fewer mistakes than with the manual alone. is real outside the test bench: that help measured in an evaluation translates into effective capability in the physical world.
That access controls on physical inputs —DNA synthesis screeningDNA synthesis screeningThe check that companies making DNA to order carry out to spot whether an order matches something dangerous, such as parts of a virus, before making and shipping it.For exampleLike the pharmacy that checks the prescription before handing over a controlled medicine., customer verification at remote labs— remain passable.
That the offensive advantage is not offset by defence accelerating at the same rate.
That hallucination in offensive tasks stops being the brake documented today by those who investigated the case.
Full entry →
What would refute it
That documented attempts keep hitting the same obstacle reported in the September 2025 campaign, fabricated results that force constant human validation.
That DNA synthesis screening and customer verification effectively close the physical step, leaving the knowledge with no exit into the material world.
That defence accelerates at least as much as attack, with automated patching measurably shrinking the exploitation window.
That dangerous-capability evaluations stop escalating across several model generations, and that the ones that do rise turn out to reflect better evaluation rather than greater capability.
Risks involved
Early signals
«Not observed» is not a clean bill of health: it means nobody has seen it yet, which is different from it not happening.
