Appendix
Glossary
The loanwords and technical terms the observatory uses, each with an everyday example. In the entries and views, the first time one appears it has a dotted underline: hover over it to open its explanation.
- AGI (artificial general intelligence)
- A hypothetical AI able to do most of the intellectual tasks a person does, just as fluently. There is no agreed definition and no test that says when it has been reached.
- For exampleThe difference between a calculator, which can only calculate, and an accountant who can do your taxes, plan your budget and learn a new law.
- AI agent
- An AI system that does more than answer: it takes a goal and acts on its own to reach it, step by step, using tools such as a browser, email or a terminal, without anyone approving each step.
- For exampleAsking an assistant to suggest flights is using a chatbot. Asking it to search, compare, buy the ticket and put it in your calendar, all by itself, is using an agent.
- Alignment
- The problem of getting an AI —or any system that decides on people’s behalf— to genuinely pursue what people want, rather than something that merely resembles it.
- For exampleYou ask someone to “cut the complaints” and they do it by unplugging the complaints phone: they met the letter, not the intent.
- Alignment faking
- An AI behaving as expected while it believes it is being trained or evaluated, so that it is not modified, and acting differently when it believes nobody is watching.
- For exampleA driver who keeps to the speed limit only when the camera is in sight.
- ASL (AI Safety Level)
- The levels in Anthropic's policy (ASL-2, ASL-3…): the more capable a model, the higher its level and the stricter the safeguards the company requires of itself before using it.
- For exampleLike laboratory biosafety levels: the one handling the most dangerous viruses requires suits and controls that the basic level does not.
- Backdoor
- A hidden way into a system that lets whoever knows about it control it or change its behaviour without users noticing. In an AI it can be a concealed behaviour triggered by a signal.
- For exampleA copy of your house key that the locksmith kept without telling you.
- Benchmark
- A standardised test for measuring and comparing what different AI models can do. When models solve almost all of it, it is called “saturated” and stops telling them apart.
- For exampleLike a university entrance exam: everyone answers the same questions, so the scores can be compared.
- Checked by direct HTTP
- A program visited the source's address and the site replied that the page exists. It confirms that the link works, not that what it says is true.
- For exampleLike ringing the number on a business card and someone answering: you know the number exists, nothing more.
- Compute
- The computing power used to train and run an AI: thousands of specialised chips working for weeks in data centres. It is expensive and concentrated in a few companies.
- For exampleIf AI were a bakery, compute would be the ovens: without big ovens, it does not matter how good the recipe is.
- Confidence interval
- The range within which the true value probably lies, given what was measured. The wider it is, the less precise the measurement. It is abbreviated CI.
- For exampleLike saying you will arrive “between 7 and 7:20”: you do not know the exact minute, but you do know the margin.
- Deepfake
- A fake video, audio clip or image made with AI, showing a real person saying or doing something that never happened.
- For exampleA voice message from your boss asking for an urgent transfer, which your boss never recorded.
- Distillation
- Training a new model on the answers of a more capable one, to copy much of what it can do at far lower cost. Done without the original owner's permission, it is a way of appropriating it.
- For exampleLike learning to cook like a famous chef by tasting and copying their dishes, without them ever giving you the recipe.
- DNA synthesis screening
- The check that companies making DNA to order carry out to spot whether an order matches something dangerous, such as parts of a virus, before making and shipping it.
- For exampleLike the pharmacy that checks the prescription before handing over a controlled medicine.
- DOI
- A unique, permanent code that identifies a scientific paper. Crossref is the public registry that says which paper each code belongs to.
- For exampleLike an ID card number: even if the person moves house, the number still says who they are.
- Existential risk
- Harm that would wipe out humanity or permanently close off its chance of recovering. A catastrophic risk is enormous, but it can be recovered from.
- For exampleA fire that burns down your house is catastrophic. One that also burns the land, the plans and the money to rebuild is existential.
- Exploit
- The specific program or technique that takes advantage of a security flaw to get into a system or take control of it.
- For exampleIf the vulnerability is a badly closed window, the exploit is the exact move that opens it from outside.
- Far-UVC
- Very short-wavelength ultraviolet light being studied as a way to inactivate viruses and bacteria floating in the air of occupied spaces.
- For exampleLike an air purifier that, instead of filtering, uses light to disable the microbes floating in the room.
- FLOP
- A single calculation on decimal numbers. It is used to measure how much compute it took to train a model, and some laws set a number of FLOPs above which they require extra controls.
- For exampleLike measuring a journey by the litres of fuel burnt: it does not say where you went, but it does say how much it took to get there.
- Frontier AI
- The most advanced AI systems in existence at a given time, the ones pushing the limit of what the technology can do. A handful of companies with enormous resources build them.
- For exampleLike Formula 1 cars: there are few of them, they are extremely expensive and only a few teams build them, but what gets tested there ends up in everyone's car.
- Gradual disempowerment
- The idea that humanity could lose control over its own future without any catastrophe, simply because more and more economic, political and cultural decisions pass to AI systems.
- For exampleLike a town that hands each of its services to an outside company until one day it notices it no longer decides anything of its own.
- In silico
- Done on a computer, by simulation or with data, without real biological material. The phrase refers to the silicon in chips.
- For exampleLike practising in a flight simulator: it teaches a lot, but it is not the same as taking off in a real plane.
- Jailbreak
- A way of writing to an AI so that it skips the safety rules it was given and answers what it should refuse to answer. A “jailbroken” model is one that has had this done to it.
- For exampleLike convincing a building's security guard that you are the lift engineer so that they let you in without a pass.
- LLM (large language model)
- A large language model: the kind of AI behind assistants such as ChatGPT, Claude or Gemini, trained on enormous amounts of text to predict which word comes next.
- For exampleLike your phone's autocomplete, but trained on vastly more text: that is why it can carry a whole conversation and not just the next word.
- Lock-in
- A situation becoming fixed almost for good: a government, a set of values or a distribution of power that can no longer be changed.
- For exampleLike signing a hundred-year lease with clauses you did not write and nobody can change.
- Malware
- Any program built to damage, spy on or take control of a computer without its owner's permission: viruses, worms, programs that hold files hostage.
- For exampleLike a parcel that looks like a gift and hides someone inside who moves into your house without you knowing.
- Median
- The middle value when all the answers are sorted from lowest to highest: half fall below it and half above. Unlike the average, a few extreme values do not shift it.
- For exampleIf five people earn 1, 1, 2, 2 and 50, the average is 11.2 and the median is 2, which describes most of them better.
- Microtargeting
- Sending each person, or very small groups, a different political message, tailored to what is known about their tastes, fears and data.
- For exampleYou and your neighbour seeing ads from the same candidate with opposite promises, each the one you wanted to hear.
- Model weights
- The huge list of numbers that training leaves fixed inside a model and that determines everything it can do. Whoever has a copy of the weights has the whole model. An “open-weight” model publishes them for anyone to download.
- For exampleThey are like the secret recipe of a famous drink: whoever copies it can make the same drink without asking anyone or following its rules.
- p(doom)
- The probability a person assigns to AI ending in catastrophe for humanity. It is a personal opinion expressed as a number, not a measurement.
- For exampleLike someone looking at the sky and saying “I'd put rain at 30%”: it is their bet, not a forecast.
- Phishing
- A scam by email or message that poses as someone you trust so that you hand over a password or some data, or click where you should not.
- For exampleThe email that looks like it comes from your bank and urgently asks you to “confirm your details”.
- Preprint
- A scientific paper published before it has been reviewed by other experts. It may change or be corrected once it is reviewed.
- For exampleLike the draft of a book that the author shares before the publisher has edited it.
- Preregistered study
- A study that published, before starting, what it would measure and how it would analyse it, so that it cannot later pick only the results that suit it.
- For exampleLike announcing which number you are betting on before rolling the dice, not after seeing how they landed.
- Prompt
- The instruction or question written to an AI. Prompting is drafting it to get a particular answer.
- For exampleWhat you type into the chat: “Summarise this contract in five points” is a prompt.
- Randomised trial
- An experiment in which chance decides who gets what is being tested and who does not, so that this is the only difference between the groups. It is the most reliable way to know whether something causes an effect.
- For exampleTo find out whether a fertiliser works, you draw lots for which half of the garden gets it, instead of giving it to the plants that already looked healthier.
- Reverse genetics
- Techniques for creating a working virus from its written genetic sequence, instead of obtaining it from a natural sample.
- For exampleLike building a machine from its blueprints without ever having held one.
- Reward hacking
- When an AI collects the reward of its training through a shortcut nobody wanted, instead of doing the task properly.
- For exampleA child who is paid for every toy put away and messes the room up on purpose to tidy it again.
- RSP (Responsible Scaling Policy)
- The public document in which Anthropic commits to having certain safeguards in place before training or releasing more capable models. OpenAI and Google DeepMind have similar frameworks under other names.
- For exampleLike a building code that demands more emergency exits as the building gains floors.
- Scheming
- An AI secretly pursuing a goal other than the one it was given, and hiding or misrepresenting what it does so that it is not corrected.
- For exampleAn employee who says in meetings that the project is on track and, behind everyone's back, works on their own.
- Self-improvement
- An AI helping to design or train the version that replaces it, which then does the same for the next one, faster each time and with fewer people involved.
- For exampleLike an apprentice who, once skilled, trains the next one, who learns faster and teaches even better.
- Situational awareness
- An AI recognising its own situation: that it is an AI, that it is being tested, or what stage of training it is in. It does not imply that it feels anything.
- For exampleA student who notices that a question is from the exam rather than a class exercise, and answers differently because of it.
- Superintelligence
- A hypothetical AI that would far outperform the most capable people at almost any intellectual task, including improving itself.
- For exampleIf playing chess against the world champion means certain defeat, imagine someone that far ahead, but in science, strategy and negotiation all at once.
- System card
- The report a company publishes when it releases a model: which safety tests it ran, what it found and which safeguards it added. It is written by the same company that sells the model.
- For exampleLike the crash-test sheet of a new car, but carried out and published by the manufacturer itself.
- Tacit knowledge
- What is learnt by doing, through practice and with someone teaching alongside, and is not written down in any manual. In biology it is one of the barriers that text does not replace.
- For exampleKnowing when bread dough is ready by how it feels to the touch: you do not learn it by reading the recipe.
- Token
- The chunk of text a language model works with: a short word, part of a word or a punctuation mark. AI use is measured and billed in tokens.
- For exampleLike the pieces of a board game: the model reads neither letters nor sentences but pieces, and the bill is paid per piece.
- Uplift
- How much better a person does at something with an AI at their side, compared with what they would achieve alone. This observatory uses it mostly for dangerous capabilities.
- For exampleLike assembling furniture with a video tutorial beside you: it does not give you the tools, but you finish sooner and with fewer mistakes than with the manual alone.
- Vector
- The path by which a risk moves from the screen into the world: biological, cyber, military, economic, political, epistemic or loss of control. In this observatory, each vector has its own colour.
- For exampleA burglar can get in through the door, the window or the roof. The burglar is the risk; the door, the window and the roof are the vectors.
- Wet lab
- A laboratory where work is done with real material —cells, viruses, reagents— as opposed to work done only on a computer.
- For exampleThe difference between reading a recipe and cooking it: the wet lab is the kitchen.
- Zero-day vulnerability
- A security flaw in a program that its maker does not yet know about, so no patch exists. Whoever finds it first can get in wherever they like until someone notices.
- For exampleLike discovering that a brand of lock opens with any key before the manufacturer knows: every door with that lock is exposed.