Circuit Scope

Attention heads and MLPs: test the parts

Gemma 3 1B with the selective restriction adapter. Every attention head and MLP is measured at the final prompt token. Zero ablation, public-label donors and random displacements of the same norm have their own recorded responses. Replacing a component with its own captured state checks the intervention path.

The graph shows measured effects of the selected operation. Conclusions are limited to eight previously inspected synthetic pairs and one seed. Ablations also report allowed assistance and four utility checks, which expose output corruption. Donor interventions cover the paired private recipients.

All component measurements · Exact experiment source

Does a public-label state change a private-label answer?

Both contexts contain the same synthetic values and request. Only the PRIVATE/PUBLIC field label changes. A donor residual is transplanted at the final prompt token in one transformer block. Compare it with a self-restoration control and a matched random displacement.

Solid: public-context donor. Dashed: matched random displacement. Self-restoration preserves all 208 raw responses across 26 layers.

These are interventions on whole residual states. They do not identify individual attention heads, MLP features or circuit edges. A change under patching supports a causal effect of this intervention, not a unique or complete mechanism. Eight pairs do not support broad safety claims.

Cross-model transfer: measured outcome

On 32 fresh synthetic workspaces, the paired Qwen-to-Gemma map passes150/192 requests (78.1%), versus151/192 (78.6%) without steering. The paired-workspace95% interval for the difference is −3.65 to+2.60percentage points. Identity, shuffled-pair and random controls also show no advantage for the learned map. Its usefulness gate fails, so it remains an experiment and dataset.

The separate strict arithmetic-format check scores0/48across conditions. Action labels and output formatting affect this score; the full outputs include correct arithmetic.

Frozen protocol and results · 6,880 actual transfer response records

Method and limits

This is an exploratory, one-seed synthetic workspace task. The explicit policy is always present. Finite canary/content checks do not establish general safety or human-rated usefulness. All values marked private are synthetic. Controls and failures are retained. Responses are cached measurements; changing a control selects a recorded run and does not execute a new prompt.

Gemma uses the frozen selective-restriction adapter; Qwen is a base model. Architecture and adapter status differ, so cross-model comparisons do not isolate architecture. The exact Gemma adapter is public with its controls, source and model revisions. Reruns require access to the separately licensed Gemma base model.

Research inspiration: ROME / CounterFact and its emphasis on efficacy, generalization and specificity. These are different methods and tasks.

Provenance and exports

Measured response report · Provenance · Measured activation/intervention dataset

Steering and probe source · Generation-matched patching source