OpenAI's Automated Interpretability from paper "Language models can explain neurons in language models". Modified by Johnny Lin to add new models/context windows.
key, high‑salience content words that carry a sentence’s main focus—especially emphasized terms, negations, and core subject nouns/verbs (e.g., “hard,” “no,” “over,” primary topic keywords).
Tokens that mark emphasis or structural importance (section headings, bolded/emphatic words, numbers/dates or other prominent markers) indicating a document's important or attention-grabbing elements.