The neuron isn’t picking out any linguistic pattern at all but simply spikes on token embeddings with very large id‐values (i.e. rare or out-of-vocabulary wordpiece tokens).