© Neuronpedia 2026
    Privacy & TermsBlogGitHubSlackTwitterContact
    Neuronpedia logo - a computer chip with a rounded viewfinder border around it

    Neuronpedia

    Natural Language
    Autoencoders
    NEW
    Assistant AxisNEWCircuit TracerUPDATESteerSAE EvalsExportsAPI Community BlogPrivacy & TermsContact
    1. Home
    2. Gemma-3-12B
    3. 24-GEMMASCOPE-2-RES-16K
    4. 12111
    Prev
    Next
    INDEX
    Explanations

    The neuron seems to be capturing instances where a specific token (like 'over', 'a', 'typically', 'later', 'for', 'i') is followed by a token or concept related to a description, an item, or a stage in a process.Let's look at the sequence:MAX_ACTIVATING_TOKENS -> TOKENS_AFTER_MAX_ACTIVATING_TOKEN- `over` -> `Carrot` (as in "served over 1" or "Carrot & Chickpea Curry served over 1")- `a` -> `server` (as in "using the server's default timezone")- `provided` -> `instructions` (as in "provided instructions")- `a` -> `REST` (as in "a RESTful API")- `a` -> `big` (as in "a big sign")- `typically` -> `broken` (as in "typically broken down into three stages")- `later` -> `on` (as in "later on, please choose one of the")- `for` -> `human` (as in "primarily for human readability")- `generally` -> `based` (as in "generally based on the federal short-term rate")The common thread is a preceding word often acting as a preposition or article, followed by a noun or descriptive adjective that starts a new concept or phrase related to instructions, descriptions, or definitions.The `TOP_POSITIVE_LOGITS` are single letters or punctuation. This neuron might be more sensitive to positional information or very short words.Considering the examples:- "served over 1" - preposition + number/item- "using the server's" - article + noun- "provided instructions" - adjective + noun- "a RESTful API" - article + noun phrase- "a big sign" - article + adjective + noun- "typically broken down" - adverb + adjective + preposition- "later on" - adverb + preposition- "for human readability" - preposition + adjective + noun- "generally based on" - adverb + adjective + prepositionThe pattern is often a general connector word followed by a specific detail. "Typically broken down" and "generally based on" point to definitions or explanations of processes. "A RESTful API" and "a big sign" are descriptive examples. "Carrot" and "server" are specific items/concepts.Let's try to find a concise phrase. The pattern seems to be relating general connectors with specific details that define or describe.The `TOP_POSITIVE_LOGITS` are very abstract (ii, iz, h, ig, ":, vv, as, qu, be, ih) which makes them hard to directly infer meaning from.However, the `TOKENS_AFTER_MAX_ACTIVATING_TOKEN` and `TOP_ACTIVATING_TEXTS` strongly suggest the neuron is firing when it sees certain initial words ('over', 'a', 'typically', 'later', 'for', 'generally') followed by details, descriptions, or stages of a process/concept.Examples:'typically broken down' -> stages of a process (money laundering)'later on' -> continuation of a narrative (storytelling prompt)'for human readability' -> explanation of a feature (JSON comments)'generally based on' -> explanation of a rule (IRS interest)The pattern is an introductory word followed by descriptive or explanatory content.Let's re-examine the MAX_ACTIVATING_TOKENS:`over`, `the`, `provided`, `a`, `pursuing`, `**`, `typically`, `later`, `for`, `generally`And TOKENS_AFTER_MAX_ACTIVATING_TOKEN:`Carrot`, `server`, `instructions`, `REST`, `a`, `big`, `broken`, `on`, `human`, `based`Common pattern: Article/Preposition/Adverb -> Noun/Adjective/Verb(a/the/over/for/later/typically/generally) + (Carrot/server/instructions/REST/a/big/broken/on/human/based)This feels like it's about establishing context or introducing a detail."served over 1" (context for meal)"using the server's" (context for software)"provided instructions" (context for task)"a RESTful API" (definition of a concept)"a big sign" (description)"typically broken down" (explanation of stages)"later on" (temporal progression)"for human readability" (explanation of purpose)"generally based on" (explanation of rule)The neuron is focused on connecting these initial words to subsequent descriptive or defining elements. The output `TOP_POSITIVE_LOGITS` are very abstract, so we must rely on the text.If we look at the structure, it's often `[connector] [specific item/description/stage]`.The `TOP_POSITIVE_LOGITS` could be interpreted as part of very short words or grammatical markers.Could it be about identifying the start of a descriptive clause or phrase following a common introductory word?Examples suggest it's about initiating a description or listing details."Carrot & Chickpea Curry (recipe below) served over 1" - `over` + `1` (item)"using the server's default timezone" - `the` + `server's` (entity)"provided instructions" - `provided` + `instructions` (concept)"a RESTful API" - `a` + `RESTful API` (concept)"a big sign" - `a` + `big sign` (object description)"typically broken down" - `typically` + `broken` (process stage)"later on, please choose" - `later` + `on` (temporal continuation)"for human readability" - `for` + `human` (purpose)"generally based on" - `generally` + `based` (rule description)The pattern is quite consistent: a common introductory word followed by a term that specifies, describes, or defines something.The neuron detects when a general connector word is followed by a specific detail.Let's try to make it concise:"connector followed by specific detail" (5 words)"introducing specific details" (3 words)"contextualizing specific information" (3 words)"general word followed by specific" (5 words)"initiating descriptive phrases" (3 words)"preposition followed by noun/adj" (5 words) - this is too grammaticalThe focus is on introducing the *specific* detail after a general word.From `TOP_POSITIVE_LOGITS`, `as` is common. Is it looking for "as [something]"? Not really in the `TOKENS_AFTER_MAX_ACTIVATING_TOKEN`.Let's check `TOP_ACTIVATING_TEXTS` for "as"."nutritional yeast) * **Lunch:** Leftover Carrot & Chickpea Curry * **Dinner:** Carrot & Chickpea Curry (recipe below) served over 1"No immediate "as"."using the server's default timezone, which could be different from Celery Beat. * **Solution:** Ensure"No immediate "as"."chef and create recipes based on your machine's capabilities and the provided instructions.""based on" is there. That matches `generally based on`."learning concepts as you need them." - `as you`"resist impulsive behaviors. It's about feeling satisfied and

    np_acts-logits-general · gemini-2.5-flash-lite

    The neuron flags tokens representing large or out‐of‐place numeric values—especially floating‐point numbers—embedded in the text.

    oai_token-act-pair · o4-miniTriggered by @jyhe0408
    New Auto-Interp
    Top Features by Cosine Similarity
    Configuration
    google/gemma-scope-2-12b-pt/resid_post/layer_24_width_16k_l0_medium
    Prompts (Dashboard)
    392,802 prompts, 256 tokens each
    Dataset (Dashboard)
    monology/pile-uncopyrighted
    No Configuration Found
    Embeds
    IFrame
    Link
    Not in Any Lists

    No Comments

    Negative Logits
    𝚘
    0.94
     Tasma
    0.91
     Realtors
    0.87
     âm
    0.86
     DMR
    0.84
     immobilier
    0.82
     près
    0.82
     dea
    0.81
    伥
    0.81
     Surety
    0.80
    POSITIVE LOGITS
    ii
    0.94
    vv
    0.86
    iz
    0.83
    ih
    0.83
    h
    0.83
    as
    0.83
    ig
    0.83
    be
    0.82
    qu
    0.82
    ":
    0.80
    Activations Density 0.000%

    No Known Activations