Constitutionalism

The rules are treated as an artifact in their own right: someone decides which principles go in, how each one is worded, and whose values they encode, and then the text is published so outsiders can read it and hold the lab to it. Claude's Constitution and OpenAI's Model Spec are the two best-known published texts. The rest of the work here aims at the document rather than at any model — choosing and framing the principles before training begins, forcing pairs of principles into conflict to surface contradictions in the writing itself, and running the text against cross-cultural value surveys to see which culture's commitments it quietly carries.

The model is shown its own answers alongside the written principles and asked to criticize and rewrite them, then trained on the improved versions; a second model reading those same principles supplies the preference labels human raters would otherwise give. The document plus a model reading it replaces almost all the human labeling, and once training is finished the text no longer has to be present when the model answers. That is also why the adherence benchmarks sit here: when the rules have dissolved into the weights there is no rule left to inspect, so the only audit available is to break the published text into atomic claims and measure how often finished models violate them.

Rather than one overall verdict on whether an answer was good, the constitution is broken into separate rules, each rule is graded on its own, and the per-rule grades are combined with explicit weights into the reward that drives training. The cut from the previous strategy is the shape of the signal, not who produces it: holistic preference over whole responses there, decomposed grading with a visible weighting here. Because each rule carries its own score and its own weight, you can see which rule is being broken and retune that one without rebuilding the reward.

The governing principles are in the context when the answer is produced and the model works through them out loud, so the application of a rule is visible in the trace instead of buried in a preference. The two members differ in where those principles come from, and the difference matters: deliberative alignment recalls a standing spec the developer wrote and published, and the model is trained to reason over it, whereas SPRI has no constitution at all — it writes fresh principles for the query in front of it and scores its own answer against them, so nobody outside can read the rules in advance. The second route buys coverage where no rulebook exists and gives up the one property, a text someone else can hold you to, that makes the rest of this regime constitutional.

Enforcement lives in a component outside the model being governed: a second model inspects prompts on the way in and answers on the way out, and blocks whatever the standard forbids. The standard itself varies — a hand-written risk taxonomy, a statement of permitted and restricted content turned into synthetic training data for a classifier, a rule tree extracted from a company's own product and design documents with every node linked back to its source, or an explicit harm-benefit weighting rather than a list of rules at all — but in every case it is written down separately from the model it guards, so it can be rewritten or swapped without retraining anything. Where the policy text comes from is a variant inside this strategy, not a strategy of its own; the shared move is that the rule and its enforcer are detachable from the thing being ruled.

Instead of stating what the model may say, this fixes an order of authority over where text comes from: the developer's system prompt outranks the user, who outranks content pulled in from web pages, tool outputs and other untrusted sources. The model is trained on examples that teach it to obey the higher-ranked source and selectively ignore conflicting instructions from lower ones, which is what stops an instruction buried in a retrieved document from overriding the rules the operator set. The evaluation here probes exactly that failure: rules handed to the model in its prompt, with a user pushing, and adversarial optimization pushing harder, to talk it back out of them.