transformer-lens-interpretability

SKILL.md

TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation.

The main class that wraps transformer models with HookPoints on every activation:

| GPT-2 | gpt2, gpt2-medium, gpt2-large, gpt2-xl | | LLaMA | llama-7b, llama-13b, llama-2-7b, llama-2-13b | | EleutherAI | pythia-70m to pythia-12b, gpt-neo, gpt-j-6b | | Mistral | mistral-7b, mixtral-8x7b | | Others | phi, qwen, opt, gemma |

为使用 TransformerLens 通过 HookPoint 和激活缓存检查和操作变压器内部结构的机械可解释性研究提供指导。在逆向工程模型算法、研究注意力模式或执行激活修补实验时使用。 来源：orchestra-research/ai-research-skills。

查看原文