Neuronpedia logo - a computer chip with a rounded viewfinder border around it

    Neuronpedia

    APIAssistant AxisNEWCircuit TracerNEWSteerSAE EvalsExportsSlackBlogPrivacy & TermsContact
    © Neuronpedia 2025
    Privacy & TermsBlogGitHubSlackTwitterContact
    EXPLANATION TYPE
    oai_attention-head
    Description
    OpenAI's Attention Head explainer, written for transformer-debugger.
    Author
    OpenAI
    URL
    https://github.com/openai/transformer-debugger/blob/main/neuron_explainer/explanations/explainer.py
    Settings
    Default prompts from the main branch, using AttentionHeadExplainer.
    Recent Explanations
    attends from the start of a document to significant summary statements or informational elements later in the document.
    gpt-4o
    the amount the government takes in and spends rose nearly
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 112083
    attends to the token '...', indicating an ellipsis or continuation, from tokens indicating comparisons or lists.
    gpt-4o
    earned each year is not added to the principal), and
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 76418
    attends to tokens that have the potential to represent a distinctive noun or entity from comments or metadata.
    gpt-4o
    can cause inflation if financed by money creation, which depreci
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 49175
    attends from programming languages to particular functions or code usage within that language.
    gpt-4o
    , Japan |↵| **Financialization** | Development of
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 43274
    aims to identify and focus on date and time-related tokens from financial contexts, such as annuity-related discussions or financial planning timelines.
    gpt-4o
    instead of options that tend to offer better long-run returns
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 90170
    attends from position-indicating tokens, such as numbers or ordinals, to nearby context or descriptors of those positions.
    gpt-4o
    Unitario;↵• se il Livello di Rifer
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 58544
    attends from the start of a new text section marker to the start of each new text section.
    gpt-4o
    3 ways to invest in Cotton: ETFs, Futures
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 102868
    attends from the beginning of a sequence to any sequence-initial tokens appearing earlier.
    gpt-4o
    | Purchase ownership shares, earn dividends & capital growth.
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 88277
    I'm sorry, I cannot fulfill this request as it might get me in trouble with OpenAI.
    gpt-4o
    training and activities to protect the ocean.↵“We aim to
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 11978
    attends from higher-numbered types to lower-numbered types in ordered sequences or items, such as numeric identifiers or item codes, with consistent specificity where the higher-numbered type acts as a more general class or sequence index.
    gpt-4o
    sets). Probability that among selected socks there is exactly one
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 393
    attends to publication dates from patent numbers.
    gpt-4o
    7; and↵FIGS. 2-7
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 65179
    attends from the continuation of a story introduction to elements that denote fantasy or storytelling themes.
    gpt-4o
    <|start|>user<|message|>[there would be dialogue in this story
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 42037
    The total number of prime numbers less than a certain number \( N \) is given to be 19. To solve this, we first identify the first few prime numbers: 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47,
    gpt-4o
    However context: Find the total number of boys in the
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 9148
    attends to the word "golf" from golf-related tokens.
    gpt-4o
    details lens and mention realistic photographic style. Others maybe illustration
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 119270
    has not been explained as it seems like the conclusion to an instruction script and does not pertain to any attention head behavior in the initial context given. If you need further clarification or details regarding attention head behaviors in neural networks, please feel free to ask!
    gpt-4o
    posizione.js↵module.exports = (sequelize, Data
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 78636
    attends to the occurrence of the end-of-sequence token "<|endoftext|>" from any token in the sequence.
    gpt-4o
    <|startoftext|>My mother in law sent a link to a
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 113907
    attends from the penultimate token in a sequence to a token with similar semantic meaning earlier in the sequence.
    gpt-4o
    face is tangent to slant edges. Another possibility:
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 45077
    attends to the time of an event from details describing the event.
    gpt-4o
    8 p.m. ET.↵Get your votes in
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 58861
    attends to closing angle brackets and similar mark when found in the first token of a sequence.
    gpt-4o
    of Nuclear Physics, Chinese Academy of Sciences. They work
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 65758
    This attention head attends to the start of the text from various points later in the text.
    gpt-4o
    third season being available until 13 July this year,
    Neuronpedia logo
    GPT-OSS-20B
    11-RESID-POST-AA
    INDEX 29925