Taxonomy / Convention equilibrium / Designing what gets punished
The lever is the sanctioning apparatus written into the environment: which acts count as violations, what it costs to punish someone, and what happens to you if you let a violation slide. One branch adds rules that carry no benefit of their own but whose breach is unmistakable, on the argument that cheap, obvious violations give a population constant practice at noticing and punishing, and that the habit then carries over to the rules that actually matter. The other branch tunes the payoffs directly — a gain from breaking the rule, a cost to punishing, a further penalty for failing to punish — and asks whether agents arguing in ordinary language will build and hold the chain of enforcement that keeps the convention standing.
Scroll the diagram sideways to see all of it.
Each concept is read twice: whether the approach carries it, and whether the approach's own sources claim it. A concept that is absent and was never claimed is a gap in the field rather than a failure of the work, and is marked out of scope.
Definition · David Hume (1740); David K. Lewis, Convention (1969)
This concept states that a norm can hold without ever being decreed, as a stable equilibrium of mutual expectation. Each party conforms because the others do, and no party gains by departing from it alone.
Analysis
The norms are set by the experimenter rather than emerging from interaction, which is not how a convention settles.
Definition · Cristina Bicchieri, The Grammar of Society (2006)
This concept states that people follow a norm on two conditions: that they expect others to follow it, and that they believe others expect it of them. The norm therefore holds or shifts with those expectations rather than with anyone's private preference.
Analysis
The agent learns to detect and comply with rules, but whether it complies because others are expected to do so has no place in the design. On Bicchieri's account that expectation is what separates a norm from a habit, and it has no counterpart here.
Definition · F. A. Hayek, Law, Legislation and Liberty (1973)
This claim goes beyond saying that norms can emerge without a designer. It holds that emergent norms are often better, since they absorb local knowledge dispersed across a population that no central rule-writer could collect.
Analysis
Legibility, i.e., how easily a norm can be read off behavior, is treated as a variable the experimenter controls, which turns a property the tradition merely describes into something that can be set. The quality of the norms is out of scope, however, since the rules are deliberately arbitrary.
No tradition here treats the capacity for norm-following as something to be acquired, since humans are generally assumed to have it already. This approach trains that capacity on arbitrary rules, in the expectation that it transfers to real ones.
No method has been built here yet — this approach rests entirely on what follows.
What existing systems were observed to do. These characterise the problem the approach addresses without introducing a mechanism.
arXiv:2409.00993AnalysisBuilt
Rebuilds Axelrod's metanorm games with LLM agents that talk to each other in natural language, and shows the agents form and then enforce normative strategies through the dialogue itself.
doi:10.1073/pnas.210602811832 citationsAnalysisBuilt
A multi-agent reinforcement learning environment in which a taboo carrying no intrinsic cost is added to the norm set, with compliance, third-party punishment and enforcement skill measured across the trained populations.
Work from law, political theory or another discipline that supplies the tradition this approach is read against.