Neuronpedia

Google DeepMind x Neuronpedia

Introducing Gemma Scope, a new tool for understanding the internals of AI models.

Launch

Neuronpedia is an open platform for interpretability research.

Explore, steer, and experiment on AI models.

Getting Started

Explore Visually

Best viewed on a tablet or larger screen.

Each dot in the visualization represents an idea or concept that the model has learned. The dots, or features, are arranged in a fully interactive UMAP graph: you can zoom, pan, and select dots to see details. You can also filter features by specific terms.

Docs: Features Demo

Run Example Search

Test Instantly

Neuronpedia lets you search and test models by doing inference over millions of SAE features. You can use your own custom text, and sort and filter results by layers, tokens, or both.

Docs: Search View Demo

Infrastructure For All

Neuronpedia hosts interpretability data, tools, visualizations, and APIs for all researchers. We eliminate the engineering obstacles in order to move interpretability further, faster. Upload your SAEs and we take care of the rest.

Docs: Intro API Playground Upload SAEs

Gemma Scope - Exploring the Inner Workings of Gemma 2

Language Model Interpretability Team, Google DeepMind

Llama Scope: SAEs for Llama-3.1-8B

OpenMOSS Team, Fudan University

Feature Splitting for GPT2-Small

Joseph Bloom

Gated SAE for Llama3-8B-Instruct

Julius Han

Multi TopK SAE for Llama3.1-8B

EleutherAI

Sparse Autoencoder for GPT2-Small - v5

OpenAI

Collaborate and Share

Every SAE feature has a unique link with link previews, can be compiled into sharable lists, and supports iframe embedding (as demonstrated here).

Docs: Lists Docs: Embed

Who We Are

Neuronpedia was created by Johnny Lin, an ex-Apple engineer who previously founded a privacy startup. Neuronpedia is supported by the Long Term Future Fund and AISTOF.

Get Involved

Citation

@misc{neuronpedia,
    title = {Neuronpedia: Interactive Reference and Tooling for Analyzing Neural Networks},
    year = {2023},
    note = {Software available from neuronpedia.org},
    url = {https://www.neuronpedia.org},
    author = {Lin, Johnny}
}