transformer-lens-interpretability

SKILL.md

TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation.

The main class that wraps transformer models with HookPoints on every activation:

| GPT-2 | gpt2, gpt2-medium, gpt2-large, gpt2-xl | | LLaMA | llama-7b, llama-13b, llama-2-7b, llama-2-13b | | EleutherAI | pythia-70m to pythia-12b, gpt-neo, gpt-j-6b | | Mistral | mistral-7b, mixtral-8x7b | | Others | phi, qwen, opt, gemma |

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments. Source: orchestra-research/ai-research-skills.

View raw