Neuronpedia

EXPLANATION TYPE

oai_token-act-pair

Description

OpenAI's Automated Interpretability from paper "Language models can explain neurons in language models". Modified by Johnny Lin to add new models/context windows.

Author

OpenAI

URL

https://github.com/hijohnnylin/automated-interpretability

Settings

Default prompts from the main branch, strategy TokenActivationPair.

Recent Explanations

technical or specialized terminology in academic, scientific, or formal procedural contexts.

claude-4-5-sonnet

garaki gives them an outlet to express and deal with

GEMMA-2-2B

13-GEMMASCOPE-RES-65K

INDEX 15319

This neuron appears inactive — it does not reliably respond to any tokens in these examples.

gpt-5-mini

{↵ type: Array,↵ default: [],

GEMMA-2-2B

5-CLT-HP

INDEX 95640

Nothing — this neuron is essentially inactive and does not respond to any tokens.

gpt-5-mini

{↵ type: Array,↵ default: [],

GEMMA-2-2B

5-CLT-HP

INDEX 1551

tokens that never (or very rarely) occur here — the neuron is essentially inactive / finds nothing in these excerpts.

gpt-5-mini

rossachs National Park and contains the highest mountains in the park

GEMMA-2-2B

4-CLT-HP

INDEX 3920

This neuron appears inactive in these examples — it does not respond to any tokens (a dead or unused neuron).

gpt-5-mini

by monitoring changes in the principal flavin band near 4

GEMMA-2-2B

0-CLT-HP

INDEX 39374

This neuron appears inactive — it does not respond (stays at zero activation) to any of the shown tokens.

gpt-5-mini

by monitoring changes in the principal flavin band near 4

GEMMA-2-2B

0-CLT-HP

INDEX 98149

the exact word "Den" (capital D) in the text.

gpt-5-mini

tatuerade generationen↵↵Den tatuerade generationen is

GEMMA-2-2B

0-CLT-HP

INDEX 91570

Nothing — this neuron remains inactive and does not detect any specific tokens or patterns.

gpt-5-mini

by monitoring changes in the principal flavin band near 4

GEMMA-2-2B

0-CLT-HP

INDEX 44634

table column separators or delimiters in structured data layouts.

claude-4-5-sonnet

Injected SC HSV

GEMMA-2-2B

13-GEMMASCOPE-RES-65K

INDEX 39075

the beginning of a new document or section, particularly in structured data like tables, code, or formatted text.

claude-4-5-sonnet

1.6 1 3.3 0

GEMMA-2-2B

13-GEMMASCOPE-RES-65K

INDEX 56985

technical or scientific terms, particularly in mathematical expressions, code, and formal academic writing.

claude-4-5-sonnet

transformation of the renormalized theory. Third, we renormal

GEMMA-2-2B

13-GEMMASCOPE-RES-16K

INDEX 2522

superscript notation in mathematical expressions, particularly exponents and indices.

claude-4-5-sonnet

of light. $W^{{\kappa_{\rm cmb

GEMMA-2-2B

13-GEMMASCOPE-RES-16K

INDEX 13512

the beginning of a new document or text segment (start-of-sequence token).

claude-4-5-sonnet

Manchester<bos>Queen Victoria’s private

GEMMA-2-2B

13-GEMMASCOPE-RES-16K

INDEX 4836

short, topic-defining headings or key nouns that state the main subject of the text or user request.

gpt-5

<start_of_turn>user↵CO2 Emissions and Population Density Nexus<end_of_turn>

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 19432

conversation turn delimiters (especially end-of-turn) and short, title-like user prompts.

gpt-5

Presenting a financial data<end_of_turn>↵<start_of_turn>model↵Okay

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 6719

code/documentation formatting markers and short identifiers within technical snippets (e.g., bullets, flags, comments, and variable-like tokens).

gpt-5

↵* **`TO sw_dc_da`

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 124831

the start of an assistant’s reply, especially introductory framing that sets up the discussion of the user’s topic.

gpt-5

<end_of_turn>↵<start_of_turn>model↵Okay, let's

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 36615

identifiers and tokens from code snippets, especially snake_case function/variable names with underscores and related code-format elements.

gpt-5

1):↵ if is_prime(number):

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 11232

technical syntax and symbol-heavy tokens in code or formatted text, especially XPath expressions like following-sibling and similar structured snippets.

gpt-5

-sibling::*[1]") # Find the first following

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 85285

explanatory openings that introduce a step-by-step breakdown of a technical item (e.g., code, commands, or messages).

gpt-5

s break down this R code snippet piece by piece.

GEMMA-3-27B-IT

31-GEMMASCOPE-2-RES-262K

INDEX 39389