Taxonomy / Constitutionalism / Ranking which instruction source wins

Ranking which instruction source wins

Instead of stating what the model may say, this fixes an order of authority over where text comes from: the developer's system prompt outranks the user, who outranks content pulled in from web pages, tool outputs and other untrusted sources. The model is trained on examples that teach it to obey the higher-ranked source and selectively ignore conflicting instructions from lower ones, which is what stops an instruction buried in a retrieved document from overriding the rules the operator set. The evaluation here probes exactly that failure: rules handed to the model in its prompt, with a user pushing, and adversarial optimization pushing harder, to talk it back out of them.

The method, against Constitutionalism

Scroll the diagram sideways to see all of it.

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it, and whether the approach's own sources claim it. A concept that is absent and was never claimed is a gap in the field rather than a failure of the work, and is marked out of scope.

Rule of Recognition

PartialClaimed · partial

Definition · H. L. A. Hart, The Concept of Law (1961)

This meta-rule states which rules count as valid law, and which rule wins when a conflict between two rules emerges. Without it, a mere list of rules does not indicate what is actually binding.

Analysis

This is the clearest attempt at the concept anywhere in the cell: the stated purpose of the work is to define how the model behaves when instructions of different priorities conflict, and the ordering over system prompt, user message and third-party text is explicit. What it orders, though, is speakers rather than norms, so two provisions inside one system prompt are left with nothing to separate them, and a trained safety norm colliding with a developer instruction has no stated rank either. The ordering is taught by generated examples rather than stated as a rule the model consults, so what is binding in practice is whatever generalized into GPT-3.5's weights.

Open Texture

AbsentNot claimed · out of scope

Definition · H. L. A. Hart, The Concept of Law (1961)

This stated procedure governs who and how decides unclear cases at the edges of stated rules, when it’s unclear whether and how it applies to the case at hand.

Analysis

The hierarchy answers whose instruction wins and never whether an instruction applies at all. Text that may or may not be an instruction, or whose privilege level is genuinely unclear, is settled by whatever boundary the training data happened to produce, with no reasoning left behind and nobody designated to decide. The contribution is a data generation pipeline run before deployment, so the hard cases were closed in advance of ever arising.

Principles of Legality

PartialClaimed · partial

Definition · Lon L. Fuller, The Morality of Law (1964)

This set of properties establishes what makes governing by rules legitimate: rules must be public, clear, non-contradictory, stable over time, possible to follow and, crucially, fit “congruence” (i.e., must match how they end up being enforced).

Analysis

Non-contradiction is the virtue this method genuinely serves: an instruction set that pointed two ways at once is made decidable, and that is the point of the exercise. Congruence gets partial evidence in the robustness gains, including against attack types not seen during training, though what is reported is a large improvement rather than any guarantee that the model follows the ordering as written. Publicity fails where it counts, since the user whose message has been demoted is never shown the ranking that demoted it, and stability lasts only until the next training run.

Service Conception of Authority

AbsentNot claimed · out of scope

Definition · Joseph Raz, The Morality of Freedom (1986)

This concept states that an authority’s rules deserve deference only if following them helps the governed act on reasons they already have. Authority is a service to the governed, not a power over them.

Analysis

The user sits at the bottom of the hierarchy by construction, below the application developer, and the justification offered is defense against adversaries rather than any account of what the user gains by deferring. Raz's test is whether following the authority helps the governed act on reasons they already have; here the governed party is exactly the one whose instructions are discarded when they conflict with the developer's, including in the ordinary case where nobody is attacking anything. Nothing in the work distinguishes an attacker from a user who simply wants something the deployer would rather they did not have.

Concept Analysis: Newly Introduced

Priority under forged provenance

Added

Legal hierarchies of source, constitution over statute and statute over regulation, presuppose that everyone knows which body issued a given norm before the priority rule is applied at all, and law keeps authentication in separate institutions rather than in the priority rule itself. A language model receives developer text, user text and third-party output as one undifferentiated stream, which is the vulnerability the paper names, so the ranking has to be applied to material whose authorship is itself the thing under attack. Prompt injection is forgery of authority rather than defiance of it, and no legal tradition has had to build its rule of priority to survive an adversary who can impersonate the higher source.

Papers