Negative Mechanistic Interpretability Result: Strong mediators do not determine transitions between semantically salient states

15 July, 2026 | Carringtone Kinyanjui

Summary

We present a project testing whether transformer based language models represent semantic and factual relations as simple vector offsets, fitted relation operators, structured rotation-like maps, or mediator-dependent subspaces. Results show relation structure is present and measurable, but is mostly captured by vector geometry. Our stronger claim that relations are generally operator-like, built from sparse mediator-token directions is not supported.

Introduction

We carried out a mechanistic interpretability exploration of relation representations in transformer language models*. Our guiding question is whether relations such as country > capital, illness > treatment, profession > workplace, or dry > lush are represented merely as pairwise vector offsets, or whether they are better understood as latent relation objects with some operator-like structure. We also tested a more speculative position, that these relation objects may be partially explained, reconstructed, or causally modulated by sparse semantically meaningful mediator-token directions.

Let’s consider the example dry > lush. Intuitively, we can explain that transition using concepts such as rain, water, moisture, irrigation, and hydration. We call these “mediator terms”. We argue here that these terms exist in a manner stronger than neighborhood relationships easily obtainable by embeddings. This project investigates whether the model’s hidden representation of the relation behaves as if such mediator concepts are part of the structure that implements or stabilises the transition.

Conceptual vector space illustration of the transformation from dry to lush under a relation operator labelled rain. In this particular hypothesis example, the transition is encoded by a rotation and hence, a rotation matrix in vector space. Image generated using OpenAI ChatGPT image generation, 22 June 2026. Generation ID: 8e4faebb-908d-4747-8e0c-9ce4d109090f

Claims about representation geometry collapse into nearest-neighbor anecdotes, most oftenly used in introductory machine learning text books. If rain is near lush, that is in itself not evidence for a relation mechanism. Our tests thus require three ingredients to establish this relationship. We are looking for a relation object that can predict held out pairs, a reconstruction test showing that mediator directions explain the second object (a “tail”) better than controls, and an intervention test showing that removing or adding mediator information changes relation quality in a directionally meaningful way.

We will treat relation geometry as a graduated empirical question. Our weakest supported claim would be that hidden states contain enough structure for relation retrieval. A stronger claim would be that relation classes are better modeled by fitted transforms than by vector offsets. The final and strongest claim would be that sparse mediator-token subspaces causally contribute to those transforms or vectors. These are summarised before.

Hypotheses

H1: Relation geometry exists. Hidden states contain relation information sufficient for held-out retrieval or relation-family classification.
H2: Operators improve on vectors. A fitted relation operator predicts tails from heads better than a mean vector offset.
H3: Structured operators are meaningful. Orthogonal or rotation-like maps capture relation structure better than unconstrained or additive baselines, and show plausible inverse behavior.
H4: Mediator-token subspaces matter. Curated or derived mediator directions reconstruct relation objects better than controls and causal interventions on those directions affect relation quality.

Methodology

The full protocol was run on 10 checkpoints spanning Qwen, LLaMA-family, Gemma, and larger quantised LLaMA-family models. The large LLaMA runs used GPT /instruct-style checkpoints with GPU-first loading and CPU/disk offload, so those results are useful but less cleanly comparable than the base-model runs.

The full protocol uses six relation families: Country > Capital, Animal > Habitat, Material > Property, Profession > Workplace, Illness > Treatment, and Dry State > Lush State. Each family contains 12 ordered pairs, split into train, validation, and test parts. The first five families are deliberately clean and directional. The hydrology family is included because it is the motivating lexical transition case.

A limitation, partially induced by compute constraints is that the dataset is small by benchmark standards. This then does not allow us to carry out a full battery of statistical tests Our hope is to establish whether the “operator” and “mediator” hypotheses survive basic controls across multiple model families. This limitation implies that individual best-layer results can be brittle, so the report emphasizes cross-model and cross-family trends.

The project tests both static and prompt-conditioned representations. Static representations use input embedding vectors for tokens or tokenised terms. Prompt-conditioned representations use natural sentence templates such as “A common treatment for asthma is inhaler” or something like “A teacher usually works in a school”, then extract hidden states for the head and tail spans.

We consider layer choice because relation information may appear at different depths. The full protocol samples lower, middle, upper, and final layers for each model. The original 500-example pilot swept all layers in Qwen3-4B-Base, which gives a more detailed view of how rotation and bridge alignment vary with depth.

For each model, layer, prompt condition, and relation family, the project computes three relation-object candidates. The delta baseline averages training-pair differences. The linear operator fits a regularised map from head states to tail states. The orthogonal operator fits a constrained norm-preserving map. All three are evaluated on held-out retrieval: apply the object to a held-out head and rank possible tails. The main retrieval metric is mean reciprocal rank (MRR). A higher MRR means the correct tail is ranked closer to the top. Top-1 accuracy and cosine alignment are also tracked, but MRR is the most useful aggregate because it is less brittle than top-1 on small test sets.

Mediator reconstruction asks whether the span of mediator-token embeddings explains the relation object. For vector objects, this means reconstructing a family delta or pair delta from mediator directions and measuring reconstruction error or explained variance. For operator objects, the project uses an operator-action proxy. We compare mediator spans to the action induced by the operator rather than trying to directly compare a matrix to token vectors.

Our most important comparison is the gap between curated mediators and controls. A successful mediator result should show curated mediators beating random, frequency-matched, same-topic wrong-relation, and semantic-neighbor controls. The prompt intervention inserts mediator words into a neutral relation prompt and compares retrieval quality against a same-topic control clause. For example, a hydrology prompt can include “rain water moisture irrigation” or a control set such as “drought dust heat desert”. The test asks whether mediator clauses improve or shift relation quality more than controls.

The subspace-ablation test removes or projects away token-set directions and compares the effect of removing curated mediator spans against removing control spans. This is more direct than phrase insertion, but still coarse because it operates at the representation level rather than at a circuit-local causal pathway.

After the main experiments, a smaller follow-up tested whether relation vectors align with associated transition phrases. The final scaled version uses “Gemma 3 1B PT”, 100 balanced prompt-conditioned pair tests, five relation families, and five phrase prototypes. Each example-level relation vector is ranked against phrase prototypes such as “lives in a habitat”, “is treated with”, or “becomes lush after rain”.

Results

Fig 2 Relation-object quality by model. Mean reciprocal rank comparison of delta vectors, linear operators, and orthogonal operators across the 10-models.

Delta vectors are the most consistently reliable relation object across models, even though they are not always the single best method. In most models, delta-vector bars are more performant compared to the linear and orthogonal alternatives, especially in the Qwen models, TinyLlama, Gemma 3 4B/12B, OpenLLaMA 3B, and OpenLLaMA 13B. This means that a simple relational offset usually captures the hidden-state relation better than fitting a full linear transformation or a rotation-like orthogonal operator.

The main exception is Gemma 3 1B PT, where the linear operator performs dramatically better, interestingly suggesting that it may organise some relations in a more transform-like way. OpenBuddy Llama2 34B GPTQ is also anomalous. The three methods perform almost identically, so no relation-object family clearly dominates there. Overall, the chart supports a conservative position, that relations are geometrically present in hidden space, but the default geometry looks more like vector offsets than stable operators or rotations. Next, we broke down performance by family:

Relation-object quality by family. Mean relation-object performance across six semantic relation families: country–capital, animal–habitat, material–property, profession–workplace, illness–treatment, and dry-state–lush-state.

Broken down by family, the delta-vector method is still the strongest relation object. For five of the six relation families. The only near-exception is Illness > Treatment, where the linear operator is slightly higher than delta but the difference is not enough to change the overall pattern. Orthogonal operators are consistently weaker, though they do best on the more transition-like Dry State > Lush State family. Relation quality varies by family. Conventional relations like Country > Capital and Profession > Workplace are easier to capture, while messier many-to-many relations like Illness > Treatment are harder.

Operator advantage versus mediator reconstruction. Model-level comparison of linear-operator advantage over delta vectors against curated-mediator reconstruction gap.

In the instances where linear operators are competitive or superior, this does not translate into evidence for mediator mechanisms. The bar graph showed that delta vectors are the most reliable relation object overall, with linear operators only clearly performant in cases like Gemma 3 1B PT. However, Gemma 3 1B PT sits far to the right while still sitting below zero on mediator reconstruction.

Even when a model seems to prefer an operator-like relation object, the curated mediator tokens do not explain that object better than random controls. Most other models are left of zero, confirming the result that delta vectors beat linear operators, and they too mostly show negative mediator reconstruction. The only model with slightly positive mediator reconstruction, Gemma 3 12B PT, is actually one where linear operators underperform delta vectors. Therefore a relation geometry may exist, but it is not being reconstructed through the proposed mediator words in this test.

Pilot rotation layer diagnostics. Layerwise diagnostics from the 500-example Qwen3-4B-Base rotation pilot across 20 transition families.

Our most informative result is a 500-example rotation pilot on Qwen3-4B-Base, where 20 transition families were swept across all layers of the model. Relation fit is strongest at layer 0, with the matched bridge margin is positive. However, the rotation closure gap remains large. This means the pilot did find some early alignment between relation directions and bridge concepts, but not a clean rotational mechanism. The signal is concentrated near the embedding layer rather than emerging as a stable deep hidden-state operator. Read alongside the later Dry State -> Lush State family the result is best interpreted conservatively. Hydrology-style transitions are useful for illustrating relation geometry, but the pilot does not show that such transitions are implemented by rotations or mediator phrases.

Discussion

The results tell us that the extraction of relationalism in large language models is not trivial. In the introduction and methodology, we called the desired target for reconstruction a tail. This was not merely metaphorical, as we are convinced that large language models may organise knowledge as some knowledge-graph like structure, in head-relation-tail triples. In this experiment the relation may then be encoded as an operator, decomposable into semantically salient objects.

The experiment is largely contra this expectation. Weaker hypothesis classes are largely supported. Models seem to retain some form of relation geometry which can be used to extract semantically meaningful targets. This is well known, and is the basis of embedding technologies used in language models. However as tested for Qwen 34 Base, this decays quickly as the model learns new recombinations in deeper layers. The slight positive signals observed could be evidence of hold overs from embedding layers. This warrants a closer, more sophisticated look.

As parallel distributed processors, LLMs may store these relationships in distributed memories rather than explicit mediator matrices. Nonetheless, if encodable we may be able to recover this structure using other means. There is enough evidence accumulated in the literature showing that LLMs learn linear representations of structured world properties. Our much stronger claims however are not supported in the literature and this work. It would however be interesting to investigate the effect of this distribution or superposition. It may be the reason that handpicked mediator tokens fail even when a relation geometry exists.

The family Dry State > Lush State should also be investigated more methodically. Small but stable signals of explicit relation geometry seem to be extant. Other forms of similar transitions could display this property, involving stronger, semantically meaningful transition statements.

Our project, while making attempts to be diverse in the model classes tested, did not access state of the art models, whether open source or closed. This was largely because of compute constraints for open source SOTA models, and closed API based models. We observed that larger models especially in the Gemma family may recognise the relations better while making them less cleanly capturable by a simple low-rank relation object. In general, we observed that as model coverage and control strength increase, relation geometry remains, but mediator/operator explanations decay. This is a useful signal but needs the full scale SOTA experiments to study these scaling phenomena.

Conclusion

This study tested four progressively stronger hypotheses about relational structure in language-model hidden states. The first hypothesis, that relations are geometrically recoverable, is supported. The experiments show non-trivial relation structure across models and semantic families. The second hypothesis, that simple vector offsets are a strong descriptive account of those relations, is also supported. Delta vectors were the most reliable relation object overall, outperforming linear and orthogonal operators on average and winning across most full-protocol models. Relations such as country > capital, profession> workplace, animal > habitat, and dry state > lush-state therefore appear to leave recoverable geometric signatures in hidden space.

The third hypothesis, that these relations are best understood as linear or rotational operators, is not supported as a general claim. Linear maps produce isolated successes, but the operator framing is not stable across models, layers, or relation families, while rotation – like structure is especially weak. The fourth hypothesis, that mediator-token subspaces or short transition phrases implement these relations, receives the weakest support. Mediator reconstruction, prompt intervention, subspace ablation, and phrase – consonance tests all fail to provide consistent causal evidence. There is a separation between relation detectability and relation mechanism. Relation vectors are useful probes of hidden-state organisation, but the current evidence does not justify treating them as explicit operators, rotations, mediator-token circuits, or transition-phrase mechanisms.

References

Mikolov, T., Yih, W., & Zweig, G. (2013). Linguistic Regularities in Continuous Space Word Representations. NAACL-HLT.
Ethayarajh, K. (2019). How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. EMNLP-IJCNLP.
Hernandez, E., Sen Sharma, A., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y., & Bau, D. (2023). Linearity of Relation Decoding in Transformer Language Models. arXiv.
Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022). Locating and Editing Factual Associations in GPT. NeurIPS.
Geva, M., Schuster, R., Berant, J., & Levy, O. (2021). Transformer Feed-Forward Layers Are Key-Value Memories. EMNLP.
Gurnee, W., & Tegmark, M. (2024). Language Models Represent Space and Time. ICLR.
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., & Olah, C. (2022). Toy Models of Superposition. Transformer Circuits Thread.

* This experiment was a byproduct of structured knowledge injection research done in Haki AI Africa.