Alignment

This page is a position statement. It describes how the lab frames the problem and what it works on.

The question

The lab treats alignment as a question about systems that model themselves. Any capable agent carries a model of what it is, what it wants, and how it will behave. The agent itself is a different thing. The distance between the model and the thing is where the lab locates the problem. A system whose model of itself is accurate can be understood, and a system whose model has drifted from what it is can be wrong about itself in ways no one notices, including the system.

On this view, alignment is a relation between a system, its self-model, and the other systems it is coupled to. It is not a label attached to a finished artifact.

What is commonly misread

• Alignment is a dynamic that has to be maintained as a system changes, not a property settled once and kept.
• Capability and alignment are not opposites, and that framing them as a trade-off hides the question of how well a capable system understands itself and those around it.
• Refusal behavior is a property of outputs and is not, taken alone, evidence about what a system is or wants.
• What a model says about itself is not the same as the model of itself that its internal structure carries, and that conflating the two makes both harder to study.

How the lab works on it

The lab works in three general directions. The first is formal self-reference: stating precisely what it means for a system to contain a model of itself, and what can and cannot be established about that model from the inside. The second is measurable quantities defined over a system's own records, so that questions about self-model and behavior can be put to observation instead of left to opinion. The third is open tooling for observing agents at runtime, published as public software.

The lab publishes when work reaches its own bar for rigor and not before.