Neuronpedia

EXPLANATION TYPE

oai_token-act-pair

Description

OpenAI's Automated Interpretability from paper "Language models can explain neurons in language models". Modified by Johnny Lin to add new models/context windows.

Author

OpenAI

URL

https://github.com/hijohnnylin/automated-interpretability

Settings

Default prompts from the main branch, strategy TokenActivationPair. Uses top 10 deduplicated activations.

Recent Explanations

text tokenization or segmentation artifacts, particularly instances where the letter "l" appears as an isolated token boundary.

claude-4-5-haiku

GEMMA-4-31B-IT

INDEX 6

language expressing moral judgment or criticism of someone's actions or behavior.

claude-4-5-sonnet

Name-calling is never appropriate and I should have known

OLMO-3-1025-7B

16-RES-MATRYOSHKA-65K

INDEX 8493

technical and academic text, particularly scientific papers, patents, and formal documentation with structured formatting.

claude-4-5-sonnet

access to healthcare by 2015.[@R1

OLMO-3-1025-7B

16-RES-MATRYOSHKA-65K

INDEX 1106

colons that introduce examples, formulas, equations, or code snippets.

claude-4-5-sonnet

?↵Consider the plane equation↵Z=(-a * X

OLMO-3-1025-7B

16-RES-MATRYOSHKA-65K

INDEX 61595

academic or research text discussing psychological studies, particularly those examining cognitive processes, behavioral predictors, and mental health factors.

claude-4-5-sonnet

effects on alcohol cognitions.↵This article summarizes a

OLMO-3-1025-7B

16-RES-MATRYOSHKA-65K

INDEX 1152

the neuron detects technical developer/AI documentation language — i.e., mentions of software, models, services, and other engineering concepts.

gpt-5-mini

are incredibly skilled at generating text that *resembles*

GEMMA-3-4B-IT

22-GEMMASCOPE-2-RES-16K

INDEX 70

file input/output operations and code related to reading from or writing to files.

claude-4-5-haiku

want to print to a file to gen dataframe

GEMMA-2-9B-IT

20-AXBENCH-REFT-R1-RES-16K

INDEX 500

phrases describing physical locations or settings.

gemini-2.5-flash-lite

slowly. It stopped at the shop by the temple and

LLAMA3.1-8B

11-LLAMASCOPE-RES-32K

INDEX 18496

references to religious themes and moral judgment (e.g., God, judgment, morality).

gpt-5-nano

slowly. It stopped at the shop by the temple and

LLAMA3.1-8B

11-LLAMASCOPE-RES-32K

INDEX 18496

pirate-style speech patterns or dialectical language (words like "mate", "savvy", "be" as auxiliary verb, "Arrr").

claude-4-5-sonnet

matey! Monotheism be a belief in one

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 34737

German text or technical/formal language with prepositions and relational words.

claude-4-5-sonnet

Es ist nicht identisch mit NAME_1, das

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 54319

numbered name placeholders, particularly "NAME_2" and "NAME_3".

claude-4-5-sonnet

assistant<|end_header_id|>\n\nNAME_2. Please try again.

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 1505

definitional or explanatory clauses that describe what something is or means.

claude-4-5-sonnet

ism, which be a belief in many gods. Mono

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 33234

informal conversational language, particularly casual speech patterns and direct address to "you".

claude-4-5-sonnet

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 9981

word suffixes, particularly "-ained", "-ate", "-hood", and "-able".

claude-4-5-sonnet

Soil type: Well-drained, fertile\n* Climate

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 18012

transitions between conversational turns or speakers, particularly in chat or dialogue formats.

claude-4-5-sonnet

also act like a friend. Without further ado, let

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 39556

the beginning of assistant responses in multilingual conversational contexts.

claude-4-5-sonnet

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 28412

tokens that appear immediately before special end-of-turn markers or system prompts in conversational AI training data.

claude-4-5-sonnet

2 is from Austin, Texas, Answer in German.

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 8662

line breaks and code formatting characters in technical or programming contexts.

claude-4-5-sonnet

type f | wc -l\n```\nThe `

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 46787

sentence-ending punctuation followed by line breaks, particularly at the end of messages or sections.

claude-4-5-sonnet

es Neuigkeiten gibt.\n\nVielen Dank.\n"""\n

LLAMA3.3-70B-IT

50-RESID-POST-GF

INDEX 35449