Most AI risk frameworks sort harms into lanes. Catastrophic risks, such as CBRN uplift, cyber offense, and loss of control, get capability thresholds, dedicated evaluations, and a named place in frontier safety frameworks. Near-term harms, such as terrorist misuse and wartime misinformation, usually get a line in a usage policy and a spot in a broad testing net.
That sorting makes sense on paper. The people on the receiving end don’t experience harm in lanes, though, and the harms themselves don’t stay put.
Over the past year, an analyst’s AI chatbot misread a ship’s cargo and nearly led U.S. forces to board a Chinese vessel. Social media pages posted AI-generated portraits of supposed Holocaust victims, distorting the memory of real ones. Fake images of “captured” U.S. troops spread during the Iran war. And Tech Against Terrorism has tracked more than 30 public cases of AI acting as an operational assistant in terrorism or mass violence, linked to more than 70 deaths.
I’ve spent the past few months turning cases like these into a working paper and a talk at Oxford.1 Each one looks contained when you study it alone. Mapped together, they behave like a system.
Drawing the loops
Following Donella Meadows and the system dynamics tradition, I drew the cases as a causal loop diagram. Each arrow is a hypothesized causal link. A plus sign means two variables move in the same direction, and a minus sign means they move in opposite directions. A loop with an even number of minus signs reinforces itself, and a loop with an odd number pushes back.
In the governance erosion loop (R1), weak governance lets mis- and disinformation spread, disinformation wears down institutional resilience, and weaker institutions produce weaker governance. In the radicalization loop (R2), disinformation feeds radicalization and eventually attacks, and their propaganda feeds more disinformation. Interstate conflict produces deepfakes of prisoners of war, and those deepfakes feed the wartime disinformation that keeps the conflict going (R3). Wartime disinformation can also spill into revenge terrorism, which loops back into the wider information environment (R5).
The only loop here that pushes back is a governance response to attacks (B1). It works, but it kicks in after a delay, and only once someone has already been hurt.
What the numbers say
The full model also includes testimony archives and provenance tools. It has 14 variables, 26 signed links, and 43 feedback loops, 21 reinforcing and 22 balancing. Counting how many loops pass through each variable shows where the system concentrates.
| Variable | Feedback loops it lies on (of 43) |
|---|---|
| Mis- and disinformation | 37 |
| Governance capacity | 36 |
| Institutional resilience | 32 |
| TVE attacks | 27 |
| Wartime disinformation | 24 |
Two more results stood out. All 22 balancing loops pass through that delayed governance response, so every brake in the system engages after harm has occurred. And every near-term variable has a directed path to CBRN use, nuclear escalation, and large-scale cyberattacks. That fits what Atoosa Kasirzadeh calls the accumulative view of AI risk, in which smaller harms wear down the institutions a society would need to withstand a larger shock.
These are counts of structure in a qualitative model. They don’t measure how strong any effect is, and adding or removing a link changes them. Still, they point to the places where one intervention touches the most loops, which is the idea behind Meadows’ places to intervene in a system.
What this means for testing
For teams that red team AI systems, or have to act on the results, a few things follow.
First, test across lanes. A misinformation finding is also a governance finding and, a few links down the chain, a security finding. Programs that treat misinformation as “lower risk” can end up skipping the variable that sits on more loops than any other.
Second, shorten the delay. If the brakes only engage after harm, the most useful thing an evaluation program can do is move the response earlier. Recurring red teaming, shared threat intelligence on model misuse, and incident reporting built for conflict settings all help with that.
Third, write findings for the people who can act on them. A finding that sits in a report does nothing for governance capacity, and governance capacity sits on 36 of the 43 loops. That gap, between an evaluation result and an accountable decision, is where Syntony works.
What we don’t know yet
The model is a set of hypotheses. Its links are signed but unweighted, and they haven’t been reviewed by independent experts or tested against incident data. Our next step is an explicit adjacency matrix with an estimated strength, a confidence level, and sources for every link, followed by sensitivity analysis and the standard validation tests used in system dynamics. We’ll share that work as it develops.
For now, the structure supports a modest conclusion. The harms reaching people today are wearing down the same institutions we would rely on if something catastrophic happened, and they deserve a much larger share of evaluation and governance attention than they currently get.
-
The working paper, “Re-Evaluating Risk Conceptions of Near-Term AI Harms,” is available on request. Get in touch if you’d like a copy. ↩