An amusing experiment circulating around April Fools’ Day offered a clear, accessible window into how modern language models can be inspected and nudged from the inside. A version of Meta’s Llama model was modified so that it became preoccupied with the Eiffel Tower, steering almost any prompt — prank ideas, compliments, casual chat — back toward the Paris landmark. The result is funny, but the mechanism behind it is a serious and active area of AI research.
What is actually happening
The “Eiffel Tower Llama” is an example of feature steering. Rather than retraining a model or changing its prompt, researchers identify an internal feature that corresponds to a concept — in this case, the Eiffel Tower — and amplify it. With that feature turned up, the model keeps pulling its responses toward the concept, and related ideas such as towers, views, climbs and celebrations tend to surface as well. The technique was made approachable by a tool from the interpretability company Goodfire, which used sparse autoencoders to expose and adjust features inside Llama 3.
A cousin of “Golden Gate Claude”
The idea echoes an earlier, widely shared demonstration in which Anthropic amplified a “Golden Gate Bridge” feature inside its Claude model, producing a version that worked the bridge into nearly every answer. Both experiments illustrate the same underlying advance: large models contain identifiable internal features tied to specific concepts, and those features can be located and dialled up or down. That capability is the foundation of mechanistic interpretability, the effort to understand what is happening inside a model rather than treating it as an opaque box.
Why it matters beyond the joke
Being able to find and adjust features has practical implications. The same approach used to create a tower-obsessed model can, in principle, be pointed at safety-relevant behaviours — features associated with sycophancy, with covering up mistakes, or with toxic output. If such features can be reliably identified, they offer a more transparent route to understanding and shaping model behaviour than prompt engineering alone.
Limitations and what to watch
The experiment also highlights the limits of the technique. Amplifying a single feature degrades the model’s overall coherence: outputs grow stranger and can collapse into word salad, which points to a real trade-off between steering one behaviour and preserving general performance. Identifying a clean, well-isolated feature is itself difficult, and turning a feature up too far tends to break more than it fixes. Feature steering is therefore best read as an illuminating research tool and an early step toward controllable models, not a finished method for safely tuning production systems. The original write-up, with many more examples, appears on Janelle Shane’s AI Weirdness blog, and the underlying tooling is described in Goodfire’s research on steering Llama 3.
