OpenAI's Automated Interpretability from paper "Language models can explain neurons in language models". Modified by Johnny Lin to add new models/context windows.
Default prompts from the main branch, strategy TokenActivationPair. Uses top 10 deduplicated activations.
Recent Explanations
Chinese/Japanese tokens that complete a fixed term tied to the year-end or New Year period or to annual events, such as the final character of words for New Year goods, year-end meals, or yearly chronicles.
Predicting the word "stockings" (Christmas stockings) or related Christmas decoration contexts, firing on tokens right before or on the stocking/tree placement, such as "under the tree," "ornaments," and "stockings," while being suppressed by "Santa."
the "osa" token completing the surname "Espinosa" (strongest on "osa" after "Esp" + "in"), with weaker activation on the "ino"/"in" pieces of related name fragments.
the word "moon" when it refers to a special lunar event, especially the "moon" token completing "supermoon" or named moons like blood, blue, or harvest moon.
the word for a calendar month or period (e.g., "month," a named holiday month, or the date/year unit) in a sentence about a holiday or dated occasion, firing on the token that names or completes the time period, right before the specific date or event is given.
the tokens just before a predicted mention of the Epiphany/Three Kings holiday (January 6th, King Cake, the Magi's visit), firing on connecting words and numbers in texts about that celebration where the next word would continue the Epiphany context.
Tokens in text about Christmas and the holiday season, especially function words and connecting words (like "the," "to," "of," "it") that come right before or after a Christmas-related concept and anticipate or follow holiday content.
Words in Christmas or holiday-tradition text that come right after a point where a list of activities or customs is being set up or continued, such as verbs and commas or "and" introducing the next practice (e.g., "brought," "includes," "enjoyed," "put away," "take down"), so the neuron seems to predict that a holiday-related item or action comes next.
Words in gift-giving or holiday/celebration contexts (Christmas, birthdays, weddings), especially function words like "a", "of", "for" where a gift or present is expected to come next.
the noun that completes an event or celebration phrase, especially holiday or festive occasions (e.g., the final word of a "Christmas ___", "birthday ___", or "National Flower ___" phrase).
Christmas (Natale/Noël/Рождество) and Christmas-related holiday context in non-English text, especially the word for Christmas itself and the words just before it, such as "Christmas Eve", "Christmas day", or "New Year".
Weekday names (especially Saturday, Sunday, Monday, Friday) that begin event dates or schedules, along with a scattering of other unpredictable words at lower strength (e.g., "bread", "lighting", "disguise", "chocolate").
the Brazilian state name "Santa Catarina" in Portuguese text, with the strongest activation on the "atar"/"ina" tokens of "Catarina" and weaker activation on related Santa Catarina terms (e.g., "catarinense").
the token "Santa" (and "santa") when it begins a California place name such as Santa Rosa, Santa Cruz, Santa Monica or Santa Barbara, with weaker activation on the following city-name tokens.
tokens in Christmas-themed text, especially holiday traditions and gift-giving, where the next word is likely to continue a festive or holiday-related phrase (e.g., "the" before gifts or lights, "it" before a Christmas-related verb).
words and phrases describing what Santa Claus does (his actions, tools, and activities), especially the continuation after his name, such as tracking, delivering gifts, his sleigh, and watching or keeping tabs on who's naughty or nice.
the final token of place names in the Santa Clara / San Francisco Bay Area and similar proper nouns (e.g., "Clara" after "Santa", "ar" in "Goohar", "oric" in "Armorique", "COM" in "SOUTHCOM"), firing on the word that completes a specific named entity, especially "Clara" in "Santa Clara".
This neuron fires on tokens within Brazilian Portuguese text (especially news about Rio de Janeiro, football, and city life), most strongly on function words, prepositions, and names that mid-sentence continue a Portuguese phrase, while staying near zero on English text.
