Gradual disempowerment

The mechanism has three steps and none of them requires an AI to want anything. First, AI systems replace human labour and cognition in the economy, culture and the state. In doing so they weaken two things at once: the explicit mechanisms of control —the vote, consumer choice— and something less visible, the implicit dependencies that forced those systems to keep serving people. A state needs people to pay taxes and to staff its bureaucracy; a company needs people to work and to buy. Nobody designed those dependencies as a safeguard, but they work as one. Second, wherever the system’s incentives already pointed away from human preferences, AI does not change them: it executes them better. Third, the three domains couple —economic power shapes cultural narratives and political decisions, and cultural change alters economic behaviour—, and the projected outcome is an effectively irreversible loss of human influence over crucial social systems ^[618]Gradual Disempowerment: Systemic Existential Risks from Incremental AI DevelopmentKulveit, Jan; Douglas, Raymond; Ammann, Nora et al. · 2025 · paperView source ↗Accessed on 9 September 2026.

The argument is made by Jan Kulveit, Raymond Douglas, Nora Ammann, Deger Turan, David Krueger and David Duvenaud. Two of them sit on the writing group of the International AI Safety Report, which is the strongest institutional backing any of these scenarios has ^[538]International AI Safety Report 2026Bengio, Yoshua · 2026 · reportView source ↗Accessed on 9 September 2026.

The most honest criticism is the benign-delegation one: we delegate all the time —maps, calculators, doctors— and nobody regrets having lost the habit of reading a paper map. An operational criterion is needed to tell disempowermentGradual disempowermentThe idea that humanity could lose control over its own future without any catastrophe, simply because more and more economic, political and cultural decisions pass to AI systems.For exampleLike a town that hands each of its services to an outside company until one day it notices it no longer decides anything of its own. apart from ordinary cognitive offloading, and the paper does not give one precisely. The empirical criticism has a name and a date: Zanna Iscenko and Fabien Curto Millet argue that the labour-market damage read as an early signal is the predictable consequence of the sharpest rate-hiking cycle in four decades ^[566]Looking for the Ladder: Is AI Impacting Entry-Level Jobs?Iscenko, Zanna; Curto Millet, Fabien · 2026 · reportView source ↗Accessed on 9 September 2026. And Arvind Narayanan and Sayash Kapoor dispute the prior step: capability does not become power without passing through institutions that take it up ^[773]AI as Normal TechnologyNarayanan, Arvind; Kapoor, Sayash · 2025 · paperView source ↗Accessed on 9 September 2026.

It is a slow scenario, and that is where its difficulty lies: it has no day on which it happens, and for that reason no day on which anyone decides to correct it either.