In a new research paper, OpenAI details a technique that uses its GPT-4 language model to write explanations for the behavior of neurons in its older GPT-2 model. This step forward for interpretability aims to explain why neural networks create the outputs they do. While AI researchers are not sure of their functionality and capabilities, OpenAI hopes to automate the interpretation process to overcome the limitations of traditional manual human inspection. OpenAIs technique explains what patterns in text cause a neuron to activate through neuron, circuit, and attention head explanations.
