Taxonomy / Polycentric proceduralism / Training locally, sharing only model updates

Training locally, sharing only model updates

Each group trains on its own members' feedback where that feedback already sits and sends out only the trained thing, whether model updates, raw parameters, or compact selectors standing in for preferences, so no group's data leaves it and no pooled corpus is ever assembled. What separates this from fitting a standard per group is which part is partitioned: here it is the pipeline, the data and the computation, and not necessarily the number of standards at the end. That distinction cuts both ways and is worth stating plainly rather than burying. One member trains a personalized policy for each client and so meets the previous strategy's test as well, and is placed here because its defining move is the federated architecture; another keeps data local but converges by averaging on a single shared preference predictor, so its plurality is entirely in the process, in who holds the data, who computes, and who can withhold or fork, and not in the outcome. That process-level plurality is a real lever a member of a pooled corpus does not hold, which is why these belong in a procedural regime, but the single-model case is the thinnest claim to partition in the set.

The method, against Polycentric proceduralism

Scroll the diagram sideways to see all of it.

Concept Analysis: Theoretical Foundations

Each concept is read twice: whether the approach carries it, and whether the approach's own sources claim it. A concept that is absent and was never claimed is a gap in the field rather than a failure of the work, and is marked out of scope.

Many centers under shared meta-rules

PartialClaimed · partial

Definition · Michael Polanyi, The Logic of Liberty (1951)

Many independent decision centers, each acting on its own — but under one common framework of rules that all of them acknowledge. The shared framework is what makes it a system rather than a scatter.

Analysis

Both halves of Polanyi's definition are closer here than anywhere else in the cell: each group is a genuine decision center, training its own preference model on its own data, and the federated protocol is a common framework every center acknowledges and operates under. The framework is procedural rather than normative — it governs how updates are combined, not what any center may decide — so what the centers share is a transport layer, not meta-rules.

Overlapping jurisdiction

AbsentNot claimed · out of scope

Definition · Vincent Ostrom, Tiebout & Warren (1961)

The centers' authority overlaps on purpose — the same case can fall under more than one — and the friction between them is treated as a feature, worked out through interaction rather than prevented by a clean division of turf.

Analysis

Clients are partitioned, and where preferences conflict the response is to group similar clients together so that heterogeneity is reduced. That is the opposite of the tradition's move: overlap is treated as a problem to be engineered away rather than as friction worth having.

Subsidiarity and nesting

PartialNot claimed

Definition · Johannes Althusius (1603); Elinor Ostrom, Governing the Commons (1990)

Decisions are made at the most local level capable of making them, and local units nest inside larger ones that handle what the local level cannot. Authority is layered — neither centralized nor flatly fragmented.

Analysis

There is a real two-layer structure — local training under a global aggregation step — which is more nesting than the rest of the cell has. But subsidiarity is a rule about which decisions belong at which level, and here the allocation is fixed by the algorithm: everything local is data, everything global is the merge, and nothing can be escalated because a group could not settle it.

Concept Analysis: Newly Introduced

Jurisdiction as a privacy consequence

Added

The centers here exist because raw preference data must not leave its group, not because any group claimed authority over its own affairs. Polycentricity arrives as a by-product of a confidentiality constraint — which delivers the architecture the tradition describes while supplying none of the authority that is supposed to justify it.

Papers

PluralLLM: Pluralistic Alignment in LLMs via Federated Learning

Mahmoud Srewa et al., Mar 2025

arXiv:2503.0992517 citationsMethodBuilt

Lets several user groups train a shared transformer preference predictor by federated averaging without any group's feedback leaving it, converging 46% faster than centralized training with a 4% higher alignment score and near-identical group fairness.

Towards Federated RLHF with Aggregated Client Preference for LLMs

Feijie Wu et al., Jul 2024

arXiv:2407.03038MethodBuilt

Encodes each client's preferences as binary selectors and aggregates the selectors rather than the data, grouping clients with similar preferences to handle heterogeneity and using several selectors at once to resist reward hacking.

FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF

Flint Xiaofeng Fan et al., Dec 2024

arXiv:2412.15538MethodBuilt

Each client folds its own human feedback into a local reward function and updates its own policy through a personalized RLHF loop, with no raw data or human feedback leaving the client, reaching performance on a par with centralized RLHF on the MovieLens and IMDb datasets while improving personalization across client environments, and carrying convergence guarantees and sample complexity bounds that scale efficiently with the number of clients.

Collaborative Content Moderation in the Fediverse

Haris Bin Zia et al., Jan 2025

arXiv:2501.05871MethodBuilt

Lets Fediverse servers exchange the parameters of their partially trained local moderation models with similar servers to form a federated model shared among the collaborating servers, reaching average per-server macro-F1 of 0.71 on harmful content detection, 0.73 on bot content detection and 0.58 on content warning assignment.