Political and power concentration
Total surveillance and lock-in
Continuous identification of an entire population at near-zero marginal cost, and a regime that no longer depends on humans it must convince.
- Severity
- Irreversible
- Horizon
- 3–10 years
- Evidence
- Projected
- Consensus
- Low
This risk has two halves with very different qualities of evidence, and it is worth not mixing them up. The first has been observed and does not require superhuman AI. Human Rights Watch reverse-engineered the police application of Xinjiang’s Integrated Joint Operations Platform: it collects everything from the colour of a car to a person’s height to the centimetre, linked to their ID number; it flags 51 network tools as suspicious and classifies 36 types of person subject to special attention [524]China's Algorithms of Repression: Reverse Engineering a Xinjiang Police Mass Surveillance AppView source ↗. And it is not only an authoritarian regime: in June 2026 Statewatch documented official British figures —Essex police scanned 2.2 million faces in 2024-2025 with 117 arrests; the Metropolitan Police scanned 1.7 million in four months of 2026 with 44— plus 40 additional vans and £115 million over three years [998]England: Police use of facial recognition technology growing rapidlyView source ↗. What matters is not the error rate but the ratio between those scanned and those sought.
The second half is the lock-inLock-inA situation becoming fixed almost for good: a government, a set of values or a distribution of power that can no longer be changed.For exampleLike signing a hundred-year lease with clauses you did not write and nobody can change., and it is speculative: Finnveden, Riedel and Shulman argue that the route by which regimes have historically fallen —the defection of the security forces— closes if the regime does not depend on human support [418]AGI and Lock-inView source ↗.
What this does not demonstrate. The lock-in authors are explicit: they make claims about feasibility, not about probability, and that is what they are most often misquoted on [418]AGI and Lock-inView source ↗. On the observed side there is a nuance that cuts against the easy reading: HRW found Face++ facial recognition code inside the Xinjiang app and made clear that it independently confirmed that this code is inoperative [524]China's Algorithms of Repression: Reverse Engineering a Xinjiang Police Mass Surveillance AppView source ↗. Rose Hadshar gathers three reasons why power might remain distributed —that capabilities do not go that far, that the regulatory response is enough, that fear of misalignment reduces deployment— [495]Extreme power concentrationView source ↗.
Chain of materialisation
PreconditionObserved
Mass algorithmic surveillance is documented and needs no superhuman AI
Human Rights Watch reverse-engineered the police app of Xinjiang's Integrated Joint Operations Platform: it collects everything from car colour to height down to the centimetre, linked to the national ID number; it flags 51 network tools as suspicious and classifies 36 person types subject to special attention.
Precedents: Reverse engineering of the Xinjiang mass surveillance platform
TriggerObserved
The ratio of scanned to sought becomes absurd
In a consolidated democracy, Essex police scanned 2.2 million faces in 2024-2025 and made 117 arrests; London's Metropolitan Police scanned 1.7 million in the first four months of 2026 with 44 arrests. What matters is not the error rate but that identifying dozens requires biometrically processing millions who are suspected of nothing.
Observed and demonstrated evidence ends here. What follows is projection.
CascadeProjected
The route by which regimes fall is closed
Finnveden, Riedel and Shulman start from the fact that many states have been defeated by internal coups or revolutions, and that the dominant route is that essential supporters -military or security forces- launch the coup or allow it. If reliance on human supporters is eliminated, that possibility goes with it. The authors stress they make claims about feasibility, not likelihood.
ImpactSpeculative
A human dictator dies; a digital institution does not age
The second pillar is error correction: through redundancy and error-correcting codes, a value specification could be preserved intact at geological scales. It is an argument about technical possibility, and its only probabilistic figure is bounded and explicitly subjective.
Scenarios where it appears
See on the map →Report a mistake in this entry →
Sources
- [524] China's Algorithms of Repression: Reverse Engineering a Xinjiang Police Mass Surveillance App · Human Rights Watch 2019
- [998] England: Police use of facial recognition technology growing rapidly · Statewatch 2026
- [418] AGI and Lock-in · Forethought 2022
- [495] Extreme power concentration · 80,000 Hours 2025
- [420] AI-Enabled Coups: How a Small Group Could Use AI to Seize Power · Forethought 2025
- [73] Usage Policy · Anthropic 2025
- [290] Pentagon-Anthropic Dispute over Autonomous Weapon Systems: Potential Issues for Congress · Congressional Research Service 2026 archived copy only
- [538] International AI Safety Report 2026 · International AI Safety Report (panel con representantes nominados por más de 30 países) 2026