transformer-lens-interpretability

SKILL.md

TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation.

The main class that wraps transformer models with HookPoints on every activation:

| GPT-2 | gpt2, gpt2-medium, gpt2-large, gpt2-xl | | LLaMA | llama-7b, llama-13b, llama-2-7b, llama-2-13b | | EleutherAI | pythia-70m to pythia-12b, gpt-neo, gpt-j-6b | | Mistral | mistral-7b, mixtral-8x7b | | Others | phi, qwen, opt, gemma |

為使用 TransformerLens 透過 HookPoint 和啟動快取檢查和操作變壓器內部結構的機械可解釋性研究提供指導。逆向工程模型演算法、研究注意力模式或執行活化修補實驗時使用。 來源：orchestra-research/ai-research-skills。

查看原文