OpenAI's Automated Interpretability from paper "Language models can explain neurons in language models". Modified by Johnny Lin to add new models/context windows.
Default prompts from the main branch, strategy TokenActivationPair. Uses top 10 deduplicated activations.
Recent Explanations
key, high‑salience content words that carry a sentence’s main focus—especially emphasized terms, negations, and core subject nouns/verbs (e.g., “hard,” “no,” “over,” primary topic keywords).
Tokens that mark emphasis or structural importance (section headings, bolded/emphatic words, numbers/dates or other prominent markers) indicating a document's important or attention-grabbing elements.
Looking at the activations, this neuron activates strongly (values 4-5) on tokens like "os", "i", "e", "ne" that appear to be parts of external link sections in Wikipedia articles across multiple languages (Spanish "Enlaces externos", Italian "Collegamenti esterni",