[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6FC6QAXPBP2612E3ERK607B":3,"skill-same:01M6FC6QAXPBP2612E3ERK607B":94},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":42,"linkOnly":43,"markdownContent":44,"markdownBody":45,"frontMatter":46,"refName":57,"commitSha":58,"skillDirectory":59,"forksCount":60,"repositoryHtmlUrl":61,"repositoryHomepageUrl":6,"ownerHtmlUrl":62,"ownerBlogUrl":63,"ownerDescription":64,"updatedAt":65,"install":66,"resources":79},"01M6FC6QAXPBP2612E3ERK607B","nnsight-remote-interpretability",null,"Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.","Dicklesworthstone\u002Fpi_agent_rust","Dicklesworthstone","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F35050222?v=4","01M6CG26AR942CSVMN1M5E3DSE","Jeff Emanuel","01M6CG26EPP71PSHW1VDS71121",1992,"MIT","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"remote","Interpretability","nnsight remote interpretability","nnsight","models","when","for","orchestra","with","resources","neural network","local","research","Neural","use","model","execution","network","gpu","run","pytorch","architecture","experiments","2026-11-10 10:12:30",false,"---\nname: nnsight-remote-interpretability\ndescription: Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [nnsight, NDIF, Remote Execution, Mechanistic Interpretability, Model Internals]\ndependencies: [nnsight>=0.5.0, torch>=2.0.0]\n---\n\n# nnsight: Transparent Access to Neural Network Internals\n\nnnsight (\u002Fɛn.saɪt\u002F) enables researchers to interpret and manipulate the internals of any PyTorch model, with the unique capability of running the same code locally on small models or remotely on massive models (70B+) via NDIF.\n\n**GitHub**: [ndif-team\u002Fnnsight](https:\u002F\u002Fgithub.com\u002Fndif-team\u002Fnnsight) (730+ stars)\n**Paper**: [NNsight and NDIF: Democratizing Access to Foundation Model Internals](https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.14561) (ICLR 2025)\n\n## Key Value Proposition\n\n**Write once, run anywhere**: The same interpretability code works on GPT-2 locally or Llama-3.1-405B remotely. Just toggle `remote=True`.\n\n```python\n# Local execution (small model)\nwith model.trace(\"Hello world\"):\n    hidden = model.transformer.h[5].output[0].save()\n\n# Remote execution (massive model) - same code!\nwith model.trace(\"Hello world\", remote=True):\n    hidden = model.model.layers[40].output[0].save()\n```\n\n## When to Use nnsight\n\n**Use nnsight when you need to:**\n- Run interpretability experiments on models too large for local GPUs (70B, 405B)\n- Work with any PyTorch architecture (transformers, Mamba, custom models)\n- Perform multi-token generation interventions\n- Share activations between different prompts\n- Access full model internals without reimplementation\n\n**Consider alternatives when:**\n- You want consistent API across models → Use **TransformerLens**\n- You need declarative, shareable interventions → Use **pyvene**\n- You're training SAEs → Use **SAELens**\n- You only work with small models locally → **TransformerLens** may be simpler\n\n## Installation\n\n```bash\n# Basic installation\npip install nnsight\n\n# For vLLM support\npip install \"nnsight[vllm]\"\n```\n\nFor remote NDIF execution, sign up at [login.ndif.us](https:\u002F\u002Flogin.ndif.us) for an API key.\n\n## Core Concepts\n\n### LanguageModel Wrapper\n\n```python\nfrom nnsight import LanguageModel\n\n# Load model (uses HuggingFace under the hood)\nmodel = LanguageModel(\"openai-community\u002Fgpt2\", device_map=\"auto\")\n\n# For larger models\nmodel = LanguageModel(\"meta-llama\u002FLlama-3.1-8B\", device_map=\"auto\")\n```\n\n### Tracing Context\n\nThe `trace` context manager enables deferred execution - operations are collected into a computation graph:\n\n```python\nfrom nnsight import LanguageModel\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace(\"The Eiffel Tower is in\") as tracer:\n    # Access any module's output\n    hidden_states = model.transformer.h[5].output[0].save()\n\n    # Access attention patterns\n    attn = model.transformer.h[5].attn.attn_dropout.input[0][0].save()\n\n    # Modify activations\n    model.transformer.h[8].output[0][:] = 0  # Zero out layer 8\n\n    # Get final output\n    logits = model.output.save()\n\n# After context exits, access saved values\nprint(hidden_states.shape)  # [batch, seq, hidden]\n```\n\n### Proxy Objects\n\nInside `trace`, module accesses return Proxy objects that record operations:\n\n```python\nwith model.trace(\"Hello\"):\n    # These are all Proxy objects - operations are deferred\n    h5_out = model.transformer.h[5].output[0]  # Proxy\n    h5_mean = h5_out.mean(dim=-1)              # Proxy\n    h5_saved = h5_mean.save()                   # Save for later access\n```\n\n## Workflow 1: Activation Analysis\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nprompt = \"The capital of France is\"\n\nwith model.trace(prompt) as tracer:\n    # 1. Collect activations from multiple layers\n    layer_outputs = []\n    for i in range(12):  # GPT-2 has 12 layers\n        layer_out = model.transformer.h[i].output[0].save()\n        layer_outputs.append(layer_out)\n\n    # 2. Get attention patterns\n    attn_patterns = []\n    for i in range(12):\n        # Access attention weights (after softmax)\n        attn = model.transformer.h[i].attn.attn_dropout.input[0][0].save()\n        attn_patterns.append(attn)\n\n    # 3. Get final logits\n    logits = model.output.save()\n\n# 4. Analyze outside context\nfor i, layer_out in enumerate(layer_outputs):\n    print(f\"Layer {i} output shape: {layer_out.shape}\")\n    print(f\"Layer {i} norm: {layer_out.norm().item():.3f}\")\n\n# 5. Find top predictions\nprobs = torch.softmax(logits[0, -1], dim=-1)\ntop_tokens = probs.topk(5)\nfor token, prob in zip(top_tokens.indices, top_tokens.values):\n    print(f\"{model.tokenizer.decode(token)}: {prob.item():.3f}\")\n```\n\n### Checklist\n- [ ] Load model with LanguageModel wrapper\n- [ ] Use trace context for operations\n- [ ] Call `.save()` on values you need after context\n- [ ] Access saved values outside context\n- [ ] Use `.shape`, `.norm()`, etc. for analysis\n\n## Workflow 2: Activation Patching\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nclean_prompt = \"The Eiffel Tower is in\"\ncorrupted_prompt = \"The Colosseum is in\"\n\n# 1. Get clean activations\nwith model.trace(clean_prompt) as tracer:\n    clean_hidden = model.transformer.h[8].output[0].save()\n\n# 2. Patch clean into corrupted run\nwith model.trace(corrupted_prompt) as tracer:\n    # Replace layer 8 output with clean activations\n    model.transformer.h[8].output[0][:] = clean_hidden\n\n    patched_logits = model.output.save()\n\n# 3. Compare predictions\nparis_token = model.tokenizer.encode(\" Paris\")[0]\nrome_token = model.tokenizer.encode(\" Rome\")[0]\n\npatched_probs = torch.softmax(patched_logits[0, -1], dim=-1)\nprint(f\"Paris prob: {patched_probs[paris_token].item():.3f}\")\nprint(f\"Rome prob: {patched_probs[rome_token].item():.3f}\")\n```\n\n### Systematic Patching Sweep\n\n```python\ndef patch_layer_position(layer, position, clean_cache, corrupted_prompt):\n    \"\"\"Patch single layer\u002Fposition from clean to corrupted.\"\"\"\n    with model.trace(corrupted_prompt) as tracer:\n        # Get current activation\n        current = model.transformer.h[layer].output[0]\n\n        # Patch only specific position\n        current[:, position, :] = clean_cache[layer][:, position, :]\n\n        logits = model.output.save()\n\n    return logits\n\n# Sweep over all layers and positions\nresults = torch.zeros(12, seq_len)\nfor layer in range(12):\n    for pos in range(seq_len):\n        logits = patch_layer_position(layer, pos, clean_hidden, corrupted)\n        results[layer, pos] = compute_metric(logits)\n```\n\n## Workflow 3: Remote Execution with NDIF\n\nRun the same experiments on massive models without local GPUs.\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\n\n# 1. Load large model (will run remotely)\nmodel = LanguageModel(\"meta-llama\u002FLlama-3.1-70B\")\n\n# 2. Same code, just add remote=True\nwith model.trace(\"The meaning of life is\", remote=True) as tracer:\n    # Access internals of 70B model!\n    layer_40_out = model.model.layers[40].output[0].save()\n    logits = model.output.save()\n\n# 3. Results returned from NDIF\nprint(f\"Layer 40 shape: {layer_40_out.shape}\")\n\n# 4. Generation with interventions\nwith model.trace(remote=True) as tracer:\n    with tracer.invoke(\"What is 2+2?\"):\n        # Intervene during generation\n        model.model.layers[20].output[0][:, -1, :] *= 1.5\n\n    output = model.generate(max_new_tokens=50)\n```\n\n### NDIF Setup\n\n1. Sign up at [login.ndif.us](https:\u002F\u002Flogin.ndif.us)\n2. Get API key\n3. Set environment variable or pass to nnsight:\n\n```python\nimport os\nos.environ[\"NDIF_API_KEY\"] = \"your_key\"\n\n# Or configure directly\nfrom nnsight import CONFIG\nCONFIG.API_KEY = \"your_key\"\n```\n\n### Available Models on NDIF\n\n- Llama-3.1-8B, 70B, 405B\n- DeepSeek-R1 models\n- Various open-weight models (check [ndif.us](https:\u002F\u002Fndif.us) for current list)\n\n## Workflow 4: Cross-Prompt Activation Sharing\n\nShare activations between different inputs in a single trace.\n\n```python\nfrom nnsight import LanguageModel\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace() as tracer:\n    # First prompt\n    with tracer.invoke(\"The cat sat on the\"):\n        cat_hidden = model.transformer.h[6].output[0].save()\n\n    # Second prompt - inject cat's activations\n    with tracer.invoke(\"The dog ran through the\"):\n        # Replace with cat's activations at layer 6\n        model.transformer.h[6].output[0][:] = cat_hidden\n        dog_with_cat = model.output.save()\n\n# The dog prompt now has cat's internal representations\n```\n\n## Workflow 5: Gradient-Based Analysis\n\nAccess gradients during backward pass.\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace(\"The quick brown fox\") as tracer:\n    # Save activations and enable gradient\n    hidden = model.transformer.h[5].output[0].save()\n    hidden.retain_grad()\n\n    logits = model.output\n\n    # Compute loss on specific token\n    target_token = model.tokenizer.encode(\" jumps\")[0]\n    loss = -logits[0, -1, target_token]\n\n    # Backward pass\n    loss.backward()\n\n# Access gradients\ngrad = hidden.grad\nprint(f\"Gradient shape: {grad.shape}\")\nprint(f\"Gradient norm: {grad.norm().item():.3f}\")\n```\n\n**Note**: Gradient access not supported for vLLM or remote execution.\n\n## Common Issues & Solutions\n\n### Issue: Module path differs between models\n```python\n# GPT-2 structure\nmodel.transformer.h[5].output[0]\n\n# LLaMA structure\nmodel.model.layers[5].output[0]\n\n# Solution: Check model structure\nprint(model._model)  # See actual module names\n```\n\n### Issue: Forgetting to save\n```python\n# WRONG: Value not accessible outside trace\nwith model.trace(\"Hello\"):\n    hidden = model.transformer.h[5].output[0]  # Not saved!\n\nprint(hidden)  # Error or wrong value\n\n# RIGHT: Call .save()\nwith model.trace(\"Hello\"):\n    hidden = model.transformer.h[5].output[0].save()\n\nprint(hidden)  # Works!\n```\n\n### Issue: Remote timeout\n```python\n# For long operations, increase timeout\nwith model.trace(\"prompt\", remote=True, timeout=300) as tracer:\n    # Long operation...\n```\n\n### Issue: Memory with many saved activations\n```python\n# Only save what you need\nwith model.trace(\"prompt\"):\n    # Don't save everything\n    for i in range(100):\n        model.transformer.h[i].output[0].save()  # Memory heavy!\n\n    # Better: save specific layers\n    key_layers = [0, 5, 11]\n    for i in key_layers:\n        model.transformer.h[i].output[0].save()\n```\n\n### Issue: vLLM gradient limitation\n```python\n# vLLM doesn't support gradients\n# Use standard execution for gradient analysis\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")  # Not vLLM\n```\n\n## Key API Reference\n\n| Method\u002FProperty | Purpose |\n|-----------------|---------|\n| `model.trace(prompt, remote=False)` | Start tracing context |\n| `proxy.save()` | Save value for access after trace |\n| `proxy[:]` | Slice\u002Findex proxy (assignment patches) |\n| `tracer.invoke(prompt)` | Add prompt within trace |\n| `model.generate(...)` | Generate with interventions |\n| `model.output` | Final model output logits |\n| `model._model` | Underlying HuggingFace model |\n\n## Comparison with Other Tools\n\n| Feature | nnsight | TransformerLens | pyvene |\n|---------|---------|-----------------|--------|\n| Any architecture | Yes | Transformers only | Yes |\n| Remote execution | Yes (NDIF) | No | No |\n| Consistent API | No | Yes | Yes |\n| Deferred execution | Yes | No | No |\n| HuggingFace native | Yes | Reimplemented | Yes |\n| Shareable configs | No | No | Yes |\n\n## Reference Documentation\n\nFor detailed API documentation, tutorials, and advanced usage, see the `references\u002F` folder:\n\n| File | Contents |\n|------|----------|\n| [references\u002FREADME.md](references\u002FREADME.md) | Overview and quick start guide |\n| [references\u002Fapi.md](references\u002Fapi.md) | Complete API reference for LanguageModel, tracing, proxy objects |\n| [references\u002Ftutorials.md](references\u002Ftutorials.md) | Step-by-step tutorials for local and remote interpretability |\n\n## External Resources\n\n### Tutorials\n- [Getting Started](https:\u002F\u002Fnnsight.net\u002Fstart\u002F)\n- [Features Overview](https:\u002F\u002Fnnsight.net\u002Ffeatures\u002F)\n- [Remote Execution](https:\u002F\u002Fnnsight.net\u002Fnotebooks\u002Ffeatures\u002Fremote_execution\u002F)\n- [Applied Tutorials](https:\u002F\u002Fnnsight.net\u002Fapplied_tutorials\u002F)\n\n### Official Documentation\n- [Official Docs](https:\u002F\u002Fnnsight.net\u002Fdocumentation\u002F)\n- [NDIF Info](https:\u002F\u002Fndif.us\u002F)\n- [Community Forum](https:\u002F\u002Fdiscuss.ndif.us\u002F)\n\n### Papers\n- [NNsight and NDIF Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.14561) - Fiotto-Kaufman et al. (ICLR 2025)\n\n## Architecture Support\n\nnnsight works with any PyTorch model:\n- **Transformers**: GPT-2, LLaMA, Mistral, etc.\n- **State Space Models**: Mamba\n- **Vision Models**: ViT, CLIP\n- **Custom architectures**: Any nn.Module\n\nThe key is knowing the module structure to access the right components.\n","# nnsight: Transparent Access to Neural Network Internals\n\nnnsight (\u002Fɛn.saɪt\u002F) enables researchers to interpret and manipulate the internals of any PyTorch model, with the unique capability of running the same code locally on small models or remotely on massive models (70B+) via NDIF.\n\n**GitHub**: [ndif-team\u002Fnnsight](https:\u002F\u002Fgithub.com\u002Fndif-team\u002Fnnsight) (730+ stars)\n**Paper**: [NNsight and NDIF: Democratizing Access to Foundation Model Internals](https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.14561) (ICLR 2025)\n\n## Key Value Proposition\n\n**Write once, run anywhere**: The same interpretability code works on GPT-2 locally or Llama-3.1-405B remotely. Just toggle `remote=True`.\n\n```python\n# Local execution (small model)\nwith model.trace(\"Hello world\"):\n    hidden = model.transformer.h[5].output[0].save()\n\n# Remote execution (massive model) - same code!\nwith model.trace(\"Hello world\", remote=True):\n    hidden = model.model.layers[40].output[0].save()\n```\n\n## When to Use nnsight\n\n**Use nnsight when you need to:**\n- Run interpretability experiments on models too large for local GPUs (70B, 405B)\n- Work with any PyTorch architecture (transformers, Mamba, custom models)\n- Perform multi-token generation interventions\n- Share activations between different prompts\n- Access full model internals without reimplementation\n\n**Consider alternatives when:**\n- You want consistent API across models → Use **TransformerLens**\n- You need declarative, shareable interventions → Use **pyvene**\n- You're training SAEs → Use **SAELens**\n- You only work with small models locally → **TransformerLens** may be simpler\n\n## Installation\n\n```bash\n# Basic installation\npip install nnsight\n\n# For vLLM support\npip install \"nnsight[vllm]\"\n```\n\nFor remote NDIF execution, sign up at [login.ndif.us](https:\u002F\u002Flogin.ndif.us) for an API key.\n\n## Core Concepts\n\n### LanguageModel Wrapper\n\n```python\nfrom nnsight import LanguageModel\n\n# Load model (uses HuggingFace under the hood)\nmodel = LanguageModel(\"openai-community\u002Fgpt2\", device_map=\"auto\")\n\n# For larger models\nmodel = LanguageModel(\"meta-llama\u002FLlama-3.1-8B\", device_map=\"auto\")\n```\n\n### Tracing Context\n\nThe `trace` context manager enables deferred execution - operations are collected into a computation graph:\n\n```python\nfrom nnsight import LanguageModel\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace(\"The Eiffel Tower is in\") as tracer:\n    # Access any module's output\n    hidden_states = model.transformer.h[5].output[0].save()\n\n    # Access attention patterns\n    attn = model.transformer.h[5].attn.attn_dropout.input[0][0].save()\n\n    # Modify activations\n    model.transformer.h[8].output[0][:] = 0  # Zero out layer 8\n\n    # Get final output\n    logits = model.output.save()\n\n# After context exits, access saved values\nprint(hidden_states.shape)  # [batch, seq, hidden]\n```\n\n### Proxy Objects\n\nInside `trace`, module accesses return Proxy objects that record operations:\n\n```python\nwith model.trace(\"Hello\"):\n    # These are all Proxy objects - operations are deferred\n    h5_out = model.transformer.h[5].output[0]  # Proxy\n    h5_mean = h5_out.mean(dim=-1)              # Proxy\n    h5_saved = h5_mean.save()                   # Save for later access\n```\n\n## Workflow 1: Activation Analysis\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nprompt = \"The capital of France is\"\n\nwith model.trace(prompt) as tracer:\n    # 1. Collect activations from multiple layers\n    layer_outputs = []\n    for i in range(12):  # GPT-2 has 12 layers\n        layer_out = model.transformer.h[i].output[0].save()\n        layer_outputs.append(layer_out)\n\n    # 2. Get attention patterns\n    attn_patterns = []\n    for i in range(12):\n        # Access attention weights (after softmax)\n        attn = model.transformer.h[i].attn.attn_dropout.input[0][0].save()\n        attn_patterns.append(attn)\n\n    # 3. Get final logits\n    logits = model.output.save()\n\n# 4. Analyze outside context\nfor i, layer_out in enumerate(layer_outputs):\n    print(f\"Layer {i} output shape: {layer_out.shape}\")\n    print(f\"Layer {i} norm: {layer_out.norm().item():.3f}\")\n\n# 5. Find top predictions\nprobs = torch.softmax(logits[0, -1], dim=-1)\ntop_tokens = probs.topk(5)\nfor token, prob in zip(top_tokens.indices, top_tokens.values):\n    print(f\"{model.tokenizer.decode(token)}: {prob.item():.3f}\")\n```\n\n### Checklist\n- [ ] Load model with LanguageModel wrapper\n- [ ] Use trace context for operations\n- [ ] Call `.save()` on values you need after context\n- [ ] Access saved values outside context\n- [ ] Use `.shape`, `.norm()`, etc. for analysis\n\n## Workflow 2: Activation Patching\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nclean_prompt = \"The Eiffel Tower is in\"\ncorrupted_prompt = \"The Colosseum is in\"\n\n# 1. Get clean activations\nwith model.trace(clean_prompt) as tracer:\n    clean_hidden = model.transformer.h[8].output[0].save()\n\n# 2. Patch clean into corrupted run\nwith model.trace(corrupted_prompt) as tracer:\n    # Replace layer 8 output with clean activations\n    model.transformer.h[8].output[0][:] = clean_hidden\n\n    patched_logits = model.output.save()\n\n# 3. Compare predictions\nparis_token = model.tokenizer.encode(\" Paris\")[0]\nrome_token = model.tokenizer.encode(\" Rome\")[0]\n\npatched_probs = torch.softmax(patched_logits[0, -1], dim=-1)\nprint(f\"Paris prob: {patched_probs[paris_token].item():.3f}\")\nprint(f\"Rome prob: {patched_probs[rome_token].item():.3f}\")\n```\n\n### Systematic Patching Sweep\n\n```python\ndef patch_layer_position(layer, position, clean_cache, corrupted_prompt):\n    \"\"\"Patch single layer\u002Fposition from clean to corrupted.\"\"\"\n    with model.trace(corrupted_prompt) as tracer:\n        # Get current activation\n        current = model.transformer.h[layer].output[0]\n\n        # Patch only specific position\n        current[:, position, :] = clean_cache[layer][:, position, :]\n\n        logits = model.output.save()\n\n    return logits\n\n# Sweep over all layers and positions\nresults = torch.zeros(12, seq_len)\nfor layer in range(12):\n    for pos in range(seq_len):\n        logits = patch_layer_position(layer, pos, clean_hidden, corrupted)\n        results[layer, pos] = compute_metric(logits)\n```\n\n## Workflow 3: Remote Execution with NDIF\n\nRun the same experiments on massive models without local GPUs.\n\n### Step-by-Step\n\n```python\nfrom nnsight import LanguageModel\n\n# 1. Load large model (will run remotely)\nmodel = LanguageModel(\"meta-llama\u002FLlama-3.1-70B\")\n\n# 2. Same code, just add remote=True\nwith model.trace(\"The meaning of life is\", remote=True) as tracer:\n    # Access internals of 70B model!\n    layer_40_out = model.model.layers[40].output[0].save()\n    logits = model.output.save()\n\n# 3. Results returned from NDIF\nprint(f\"Layer 40 shape: {layer_40_out.shape}\")\n\n# 4. Generation with interventions\nwith model.trace(remote=True) as tracer:\n    with tracer.invoke(\"What is 2+2?\"):\n        # Intervene during generation\n        model.model.layers[20].output[0][:, -1, :] *= 1.5\n\n    output = model.generate(max_new_tokens=50)\n```\n\n### NDIF Setup\n\n1. Sign up at [login.ndif.us](https:\u002F\u002Flogin.ndif.us)\n2. Get API key\n3. Set environment variable or pass to nnsight:\n\n```python\nimport os\nos.environ[\"NDIF_API_KEY\"] = \"your_key\"\n\n# Or configure directly\nfrom nnsight import CONFIG\nCONFIG.API_KEY = \"your_key\"\n```\n\n### Available Models on NDIF\n\n- Llama-3.1-8B, 70B, 405B\n- DeepSeek-R1 models\n- Various open-weight models (check [ndif.us](https:\u002F\u002Fndif.us) for current list)\n\n## Workflow 4: Cross-Prompt Activation Sharing\n\nShare activations between different inputs in a single trace.\n\n```python\nfrom nnsight import LanguageModel\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace() as tracer:\n    # First prompt\n    with tracer.invoke(\"The cat sat on the\"):\n        cat_hidden = model.transformer.h[6].output[0].save()\n\n    # Second prompt - inject cat's activations\n    with tracer.invoke(\"The dog ran through the\"):\n        # Replace with cat's activations at layer 6\n        model.transformer.h[6].output[0][:] = cat_hidden\n        dog_with_cat = model.output.save()\n\n# The dog prompt now has cat's internal representations\n```\n\n## Workflow 5: Gradient-Based Analysis\n\nAccess gradients during backward pass.\n\n```python\nfrom nnsight import LanguageModel\nimport torch\n\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")\n\nwith model.trace(\"The quick brown fox\") as tracer:\n    # Save activations and enable gradient\n    hidden = model.transformer.h[5].output[0].save()\n    hidden.retain_grad()\n\n    logits = model.output\n\n    # Compute loss on specific token\n    target_token = model.tokenizer.encode(\" jumps\")[0]\n    loss = -logits[0, -1, target_token]\n\n    # Backward pass\n    loss.backward()\n\n# Access gradients\ngrad = hidden.grad\nprint(f\"Gradient shape: {grad.shape}\")\nprint(f\"Gradient norm: {grad.norm().item():.3f}\")\n```\n\n**Note**: Gradient access not supported for vLLM or remote execution.\n\n## Common Issues & Solutions\n\n### Issue: Module path differs between models\n```python\n# GPT-2 structure\nmodel.transformer.h[5].output[0]\n\n# LLaMA structure\nmodel.model.layers[5].output[0]\n\n# Solution: Check model structure\nprint(model._model)  # See actual module names\n```\n\n### Issue: Forgetting to save\n```python\n# WRONG: Value not accessible outside trace\nwith model.trace(\"Hello\"):\n    hidden = model.transformer.h[5].output[0]  # Not saved!\n\nprint(hidden)  # Error or wrong value\n\n# RIGHT: Call .save()\nwith model.trace(\"Hello\"):\n    hidden = model.transformer.h[5].output[0].save()\n\nprint(hidden)  # Works!\n```\n\n### Issue: Remote timeout\n```python\n# For long operations, increase timeout\nwith model.trace(\"prompt\", remote=True, timeout=300) as tracer:\n    # Long operation...\n```\n\n### Issue: Memory with many saved activations\n```python\n# Only save what you need\nwith model.trace(\"prompt\"):\n    # Don't save everything\n    for i in range(100):\n        model.transformer.h[i].output[0].save()  # Memory heavy!\n\n    # Better: save specific layers\n    key_layers = [0, 5, 11]\n    for i in key_layers:\n        model.transformer.h[i].output[0].save()\n```\n\n### Issue: vLLM gradient limitation\n```python\n# vLLM doesn't support gradients\n# Use standard execution for gradient analysis\nmodel = LanguageModel(\"gpt2\", device_map=\"auto\")  # Not vLLM\n```\n\n## Key API Reference\n\n| Method\u002FProperty | Purpose |\n|-----------------|---------|\n| `model.trace(prompt, remote=False)` | Start tracing context |\n| `proxy.save()` | Save value for access after trace |\n| `proxy[:]` | Slice\u002Findex proxy (assignment patches) |\n| `tracer.invoke(prompt)` | Add prompt within trace |\n| `model.generate(...)` | Generate with interventions |\n| `model.output` | Final model output logits |\n| `model._model` | Underlying HuggingFace model |\n\n## Comparison with Other Tools\n\n| Feature | nnsight | TransformerLens | pyvene |\n|---------|---------|-----------------|--------|\n| Any architecture | Yes | Transformers only | Yes |\n| Remote execution | Yes (NDIF) | No | No |\n| Consistent API | No | Yes | Yes |\n| Deferred execution | Yes | No | No |\n| HuggingFace native | Yes | Reimplemented | Yes |\n| Shareable configs | No | No | Yes |\n\n## Reference Documentation\n\nFor detailed API documentation, tutorials, and advanced usage, see the `references\u002F` folder:\n\n| File | Contents |\n|------|----------|\n| [references\u002FREADME.md](references\u002FREADME.md) | Overview and quick start guide |\n| [references\u002Fapi.md](references\u002Fapi.md) | Complete API reference for LanguageModel, tracing, proxy objects |\n| [references\u002Ftutorials.md](references\u002Ftutorials.md) | Step-by-step tutorials for local and remote interpretability |\n\n## External Resources\n\n### Tutorials\n- [Getting Started](https:\u002F\u002Fnnsight.net\u002Fstart\u002F)\n- [Features Overview](https:\u002F\u002Fnnsight.net\u002Ffeatures\u002F)\n- [Remote Execution](https:\u002F\u002Fnnsight.net\u002Fnotebooks\u002Ffeatures\u002Fremote_execution\u002F)\n- [Applied Tutorials](https:\u002F\u002Fnnsight.net\u002Fapplied_tutorials\u002F)\n\n### Official Documentation\n- [Official Docs](https:\u002F\u002Fnnsight.net\u002Fdocumentation\u002F)\n- [NDIF Info](https:\u002F\u002Fndif.us\u002F)\n- [Community Forum](https:\u002F\u002Fdiscuss.ndif.us\u002F)\n\n### Papers\n- [NNsight and NDIF Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.14561) - Fiotto-Kaufman et al. (ICLR 2025)\n\n## Architecture Support\n\nnnsight works with any PyTorch model:\n- **Transformers**: GPT-2, LLaMA, Mistral, etc.\n- **State Space Models**: Mamba\n- **Vision Models**: ViT, CLIP\n- **Custom architectures**: Any nn.Module\n\nThe key is knowing the module structure to access the right components.\n",{"name":5,"tags":47,"author":52,"license":15,"version":53,"description":7,"dependencies":54},[22,48,49,50,51],"NDIF","Remote Execution","Mechanistic Interpretability","Model Internals","Orchestra Research","1.0.0",[55,56],"nnsight>=0.5.0","torch>=2.0.0","main","90b50ed0778745a80fd5bfb13b1843d45b9e82f7","tests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight",233,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust","https:\u002F\u002Fgithub.com\u002FDicklesworthstone","https:\u002F\u002Fwww.jeffreyemanuel.com\u002F","Building in NY","2026-11-15 01:54:15",{"source":8,"skillName":5,"commands":67,"prompt":77,"zipAvailable":78},[68,71,74],{"tool":69,"command":70},"npx","npx skills add Dicklesworthstone\u002Fpi_agent_rust --skill nnsight-remote-interpretability",{"tool":72,"command":73},"bunx","bunx skills add Dicklesworthstone\u002Fpi_agent_rust --skill nnsight-remote-interpretability",{"tool":75,"command":76},"pnpm","pnpm dlx skills add Dicklesworthstone\u002Fpi_agent_rust --skill nnsight-remote-interpretability","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight\u002FSKILL.md",true,[80,86,90],{"path":81,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fapi.md","REFERENCE","参考文档",7160,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight\u002Freferences\u002Fapi.md",{"path":87,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002FREADME.md",1913,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight\u002Freferences\u002FREADME.md",{"path":91,"resourceType":82,"resourceTypeDesc":83,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Ftutorials.md",7970,"https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmechanistic-interpretability-nnsight\u002Freferences\u002Ftutorials.md",[95,128,162,184,209,236],{"skillNo":96,"name":97,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":99,"licenseFileUrl":100,"sourceHtmlUrl":101,"categories":102,"keywords":108,"publishedAt":127},"01M6FCR7G5A005P364Z6ZWCJDP","theme-factory","Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors\u002Ffonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.","Apache-2.0","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fcreative-design\u002Ftheme-factory\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fcreative-design\u002Ftheme-factory\u002FSKILL.md",[103],{"categoryNo":104,"categoryName":105,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000015","视觉设计","SCENARIO","使用场景",[109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,27,125,126],"theme factory","factory","theme","docs","toolkit","generate","artifacts","SET","fly","slides","landing","html","styling","artifact","pages","themes","you","fonts","2026-11-11 15:11:14",{"skillNo":129,"name":130,"nameZh":6,"description":131,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":99,"licenseFileUrl":132,"sourceHtmlUrl":133,"categories":134,"keywords":138,"publishedAt":42},"01M6FEGPPK0ZTKCCM0R3E3KS33","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FSKILL.md",[135],{"categoryNo":136,"categoryName":137,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000009","前端开发",[139,140,115,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,33,120,156,157,158,159,25,160,161],"web artifacts builder","web","artifacts builder","builder","frontend","css","components","react","component","routing","state management","state","shadcn","file","ui","Tools","Tailwind","single-file","simple","ui components","multi","jsx","suite",{"skillNo":163,"name":164,"nameZh":6,"description":165,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":99,"licenseFileUrl":166,"sourceHtmlUrl":167,"categories":168,"keywords":173,"publishedAt":42},"01M6FEE9VTK1W4GY21AKREJXRG","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FSKILL.md",[169,172],{"categoryNo":170,"categoryName":171,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":136,"categoryName":137,"taxonomyType":106,"taxonomyTypeDesc":107},[174,175,176,143,177,178,140,179,113,180,153,181,182,183,30,27,25],"testing","webapp testing","webapp","screenshots","playwright","debugging","browser","logs","applications","behavior",{"skillNo":185,"name":186,"nameZh":6,"description":187,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":188,"sourceHtmlUrl":189,"categories":190,"keywords":191,"publishedAt":42},"01M6FE20F7ZTQBFWKF4MPJ2SBX","neuropixels-analysis","Neuropixels neural recording analysis. Load SpikeGLX\u002FOpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen\u002FIBL curation, AI-assisted visual analysis, for Neuropixels 1.0\u002F2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fscientific\u002Fneuropixels-analysis\u002FSKILL.md",[],[192,193,194,195,196,197,198,199,33,200,32,201,202,203,204,205,27,206,207,208,25,24],"analysis","neuropixels analysis","spike","metrics","data","open","quality","user","recordings","motion","visual","quality metrics","unit","sorting","curation","recording","load",{"skillNo":210,"name":211,"nameZh":6,"description":212,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":99,"licenseFileUrl":213,"sourceHtmlUrl":214,"categories":215,"keywords":219,"publishedAt":42},"01M6FDP0PDK6F1GGX395C7625Z","skill-creator","Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fskill-creator\u002FSKILL.md",[216],{"categoryNo":217,"categoryName":218,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000023","Skill与智能体开发",[220,221,222,223,224,225,226,227,228,229,230,231,232,233,27,25,234,24,235],"skill creator","creator","workflows","create","update","capabilities","knowledge","tool","integrations","specialized","guide","new skill","existing skill","users","Existing","new",{"skillNo":237,"name":238,"nameZh":6,"description":239,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":240,"sourceHtmlUrl":241,"categories":242,"keywords":244,"publishedAt":42},"01M6FDM8Y7VK7NYG9SM7QPA3PW","notebooklm","Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FLICENSE","https:\u002F\u002Fgithub.com\u002FDicklesworthstone\u002Fpi_agent_rust\u002Fblob\u002Fmain\u002Ftests\u002Fext_conformance\u002Fartifacts\u002Ftemplates-davila7\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fnotebooklm\u002FSKILL.md",[243],{"categoryNo":217,"categoryName":218,"taxonomyType":106,"taxonomyTypeDesc":107},[238,245,246,247,248,249,250,180,251,252,33,253,254,255,256,25,257],"google","browser automation","gemini","automation","auth","code","library","management","only","document","query","Citation","source"]