[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XGCBKX711PC4RGG57DAHYA":3,"skill-same:01M6XGCBKX711PC4RGG57DAHYA":77},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":51,"commitSha":52,"skillDirectory":53,"forksCount":14,"repositoryHtmlUrl":54,"repositoryHomepageUrl":6,"ownerHtmlUrl":55,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":56,"install":57,"resources":70},"01M6XGCBKX711PC4RGG57DAHYA","model-pruning",null,"Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fmodel-pruning\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"model","Model Pruning","Wanda","when","reduce","unstructured","Unstructured Pruning","Structured Pruning","One-Shot Pruning","N:M Sparsity","Sparsity","SparseGPT","training","Emerging Techniques","Model Compression","Accelerate","synthetic","methods","Transformers","Fast Inference","2026-11-10 10:12:30",false,"---\nname: model-pruning\ndescription: Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.\ncategory: ml-training\nversion: 1.0.0\nauthor: Synthetic Sciences\nlicense: MIT\ntags: [Emerging Techniques, Model Pruning, Wanda, SparseGPT, Sparsity, Model Compression, N:M Sparsity, One-Shot Pruning, Structured Pruning, Unstructured Pruning, Fast Inference]\ndependencies: [transformers, torch]\n---\n\n# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\n### Complete Pruning Pipeline\n\n```python\nfrom transformers import Trainer, TrainingArguments\n\ndef production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\",  # or \"sparsegpt\"\n):\n    # 1. Load model\n    model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)\n    tokenizer = AutoTokenizer.from_pretrained(model_name)\n\n    # 2. Load calibration data\n    calib_dataset = load_dataset(\"wikitext\", \"wikitext-2-raw-v1\", split=\"train[:1000]\")\n\n    # 3. Apply pruning\n    if method == \"wanda\":\n        pruned_model = wanda_prune(model, calib_dataset, sparsity=target_sparsity)\n    elif method == \"sparsegpt\":\n        pruner = SparseGPT(model)\n        pruned_model = pruner.prune(calib_dataset, sparsity=target_sparsity)\n\n    # 4. (Optional) Fine-tune to recover accuracy\n    training_args = TrainingArguments(\n        output_dir=\".\u002Fpruned-model\",\n        num_train_epochs=1,\n        per_device_train_batch_size=4,\n        learning_rate=1e-5,\n        bf16=True,\n    )\n\n    trainer = Trainer(\n        model=pruned_model,\n        args=training_args,\n        train_dataset=finetune_dataset,\n    )\n\n    trainer.train()\n\n    # 5. Save\n    pruned_model.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n    tokenizer.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n\n    return pruned_model\n\n# Usage\npruned_model = production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\"\n)\n```\n\n### Evaluation\n\n```python\nfrom lm_eval import evaluator\n\n# Evaluate pruned vs original model\noriginal_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=meta-llama\u002FLlama-2-7b-hf\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\npruned_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=.\u002Fpruned-llama-7b-50\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\n# Compare\nprint(f\"Original: {original_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Pruned:   {pruned_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Degradation: {(original_results - pruned_results):.3f}\")\n\n# Typical results at 50% sparsity:\n# - Wanda: \u003C1% accuracy loss\n# - SparseGPT: \u003C0.5% accuracy loss\n# - Magnitude: 2-3% accuracy loss\n```\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n\n","# Model Pruning: Compressing LLMs\n\n## When to Use This Skill\n\nUse Model Pruning when you need to:\n- **Reduce model size** by 40-60% with \u003C1% accuracy loss\n- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)\n- **Deploy on constrained hardware** (mobile, edge devices)\n- **Compress without retraining** using one-shot methods\n- **Enable efficient serving** with reduced memory footprint\n\n**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity\n\n**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)\n\n## Installation\n\n```bash\n# Wanda implementation\ngit clone https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\ncd wanda\npip install -r requirements.txt\n\n# Optional: SparseGPT\ngit clone https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\ncd sparsegpt\npip install -e .\n\n# Dependencies\npip install torch transformers accelerate\n```\n\n## Quick Start\n\n### Wanda Pruning (One-Shot, No Retraining)\n\n**Source**: ICLR 2024 (arXiv 2306.11695)\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"meta-llama\u002FLlama-2-7b-hf\",\n    torch_dtype=torch.float16,\n    device_map=\"cuda\"\n)\ntokenizer = AutoTokenizer.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Calibration data (small dataset for activation statistics)\ncalib_data = [\n    \"The quick brown fox jumps over the lazy dog.\",\n    \"Machine learning is transforming the world.\",\n    \"Artificial intelligence powers modern applications.\",\n]\n\n# Wanda pruning function\ndef wanda_prune(model, calib_data, sparsity=0.5):\n    \"\"\"\n    Wanda: Prune by weight magnitude × input activation.\n\n    Args:\n        sparsity: Fraction of weights to prune (0.5 = 50%)\n    \"\"\"\n    # 1. Collect activation statistics\n    activations = {}\n\n    def hook_fn(name):\n        def hook(module, input, output):\n            # Store input activation norms\n            activations[name] = input[0].detach().abs().mean(dim=0)\n        return hook\n\n    # Register hooks for all linear layers\n    hooks = []\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            hooks.append(module.register_forward_hook(hook_fn(name)))\n\n    # Run calibration data\n    model.eval()\n    with torch.no_grad():\n        for text in calib_data:\n            inputs = tokenizer(text, return_tensors=\"pt\").to(model.device)\n            model(**inputs)\n\n    # Remove hooks\n    for hook in hooks:\n        hook.remove()\n\n    # 2. Prune weights based on |weight| × activation\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear) and name in activations:\n            W = module.weight.data\n            act = activations[name]\n\n            # Compute importance: |weight| × activation\n            importance = W.abs() * act.unsqueeze(0)\n\n            # Flatten and find threshold\n            threshold = torch.quantile(importance.flatten(), sparsity)\n\n            # Create mask\n            mask = importance >= threshold\n\n            # Apply mask (prune)\n            W *= mask.float()\n\n    return model\n\n# Apply Wanda pruning (50% sparsity, one-shot, no retraining)\npruned_model = wanda_prune(model, calib_data, sparsity=0.5)\n\n# Save\npruned_model.save_pretrained(\".\u002Fllama-2-7b-wanda-50\")\n```\n\n### SparseGPT (Second-Order Pruning)\n\n**Source**: arXiv 2301.00774\n\n```python\nfrom sparsegpt import SparseGPT\n\n# Load model\nmodel = AutoModelForCausalLM.from_pretrained(\"meta-llama\u002FLlama-2-7b-hf\")\n\n# Initialize SparseGPT\npruner = SparseGPT(model)\n\n# Calibration data\ncalib_data = load_calibration_data()  # ~128 samples\n\n# Prune (one-shot, layer-wise reconstruction)\npruned_model = pruner.prune(\n    calib_data=calib_data,\n    sparsity=0.5,           # 50% sparsity\n    prunen=0,               # Unstructured (0) or N:M structured\n    prunem=0,\n    percdamp=0.01,          # Damping for Hessian inverse\n)\n\n# Results: Near-lossless pruning at 50% sparsity\n```\n\n### N:M Structured Pruning (Hardware Accelerator)\n\n```python\ndef nm_prune(weight, n=2, m=4):\n    \"\"\"\n    N:M pruning: Keep N weights per M consecutive weights.\n    Example: 2:4 = keep 2 out of every 4 weights.\n\n    Compatible with NVIDIA sparse tensor cores (2:4, 4:8).\n    \"\"\"\n    # Reshape weight into groups of M\n    shape = weight.shape\n    weight_flat = weight.flatten()\n\n    # Pad to multiple of M\n    pad_size = (m - weight_flat.numel() % m) % m\n    weight_padded = F.pad(weight_flat, (0, pad_size))\n\n    # Reshape into (num_groups, m)\n    weight_grouped = weight_padded.reshape(-1, m)\n\n    # Find top-N in each group\n    _, indices = torch.topk(weight_grouped.abs(), n, dim=-1)\n\n    # Create mask\n    mask = torch.zeros_like(weight_grouped)\n    mask.scatter_(1, indices, 1.0)\n\n    # Apply mask\n    weight_pruned = weight_grouped * mask\n\n    # Reshape back\n    weight_pruned = weight_pruned.flatten()[:weight_flat.numel()]\n    return weight_pruned.reshape(shape)\n\n# Apply 2:4 sparsity (NVIDIA hardware)\nfor name, module in model.named_modules():\n    if isinstance(module, torch.nn.Linear):\n        module.weight.data = nm_prune(module.weight.data, n=2, m=4)\n\n# 50% sparsity, 2× speedup on A100 with sparse tensor cores\n```\n\n## Core Concepts\n\n### 1. Pruning Criteria\n\n**Magnitude Pruning** (baseline):\n```python\n# Prune weights with smallest absolute values\nimportance = weight.abs()\nthreshold = torch.quantile(importance, sparsity)\nmask = importance >= threshold\n```\n\n**Wanda** (weights × activations):\n```python\n# Importance = |weight| × input_activation\nimportance = weight.abs() * activation\n# Better than magnitude alone (considers usage)\n```\n\n**SparseGPT** (second-order):\n```python\n# Uses Hessian (second derivative) for importance\n# More accurate but computationally expensive\nimportance = weight^2 \u002F diag(Hessian)\n```\n\n### 2. Structured vs Unstructured\n\n**Unstructured** (fine-grained):\n- Prune individual weights\n- Higher quality (better accuracy)\n- No hardware speedup (irregular sparsity)\n\n**Structured** (coarse-grained):\n- Prune entire neurons, heads, or layers\n- Lower quality (more accuracy loss)\n- Hardware speedup (regular sparsity)\n\n**Semi-structured (N:M)**:\n- Best of both worlds\n- 50% sparsity (2:4) → 2× speedup on NVIDIA GPUs\n- Minimal accuracy loss\n\n### 3. Sparsity Patterns\n\n```python\n# Unstructured (random)\n# [1, 0, 1, 0, 1, 1, 0, 0]\n# Pros: Flexible, high quality\n# Cons: No speedup\n\n# Structured (block)\n# [1, 1, 0, 0, 1, 1, 0, 0]\n# Pros: Hardware friendly\n# Cons: More accuracy loss\n\n# N:M (semi-structured)\n# [1, 0, 1, 0] [1, 1, 0, 0]  (2:4 pattern)\n# Pros: Hardware speedup + good quality\n# Cons: Requires specific hardware (NVIDIA)\n```\n\n## Pruning Strategies\n\n### Strategy 1: Gradual Magnitude Pruning\n\n```python\ndef gradual_prune(model, initial_sparsity=0.0, final_sparsity=0.5, num_steps=100):\n    \"\"\"Gradually increase sparsity during training.\"\"\"\n    for step in range(num_steps):\n        # Current sparsity\n        current_sparsity = initial_sparsity + (final_sparsity - initial_sparsity) * (step \u002F num_steps)\n\n        # Prune at current sparsity\n        for module in model.modules():\n            if isinstance(module, torch.nn.Linear):\n                weight = module.weight.data\n                threshold = torch.quantile(weight.abs().flatten(), current_sparsity)\n                mask = weight.abs() >= threshold\n                weight *= mask.float()\n\n        # Train one step\n        train_step(model)\n\n    return model\n```\n\n### Strategy 2: Layer-wise Pruning\n\n```python\ndef layer_wise_prune(model, sparsity_per_layer):\n    \"\"\"Different sparsity for different layers.\"\"\"\n    # Early layers: Less pruning (more important)\n    # Late layers: More pruning (less critical)\n\n    sparsity_schedule = {\n        \"layer.0\": 0.3,   # 30% sparsity\n        \"layer.1\": 0.4,\n        \"layer.2\": 0.5,\n        \"layer.3\": 0.6,   # 60% sparsity\n    }\n\n    for name, module in model.named_modules():\n        if isinstance(module, torch.nn.Linear):\n            # Find layer index\n            for layer_name, sparsity in sparsity_schedule.items():\n                if layer_name in name:\n                    # Prune at layer-specific sparsity\n                    prune_layer(module, sparsity)\n                    break\n\n    return model\n```\n\n### Strategy 3: Iterative Pruning + Fine-tuning\n\n```python\ndef iterative_prune_finetune(model, target_sparsity=0.5, iterations=5):\n    \"\"\"Prune gradually with fine-tuning between iterations.\"\"\"\n    current_sparsity = 0.0\n    sparsity_increment = target_sparsity \u002F iterations\n\n    for i in range(iterations):\n        # Increase sparsity\n        current_sparsity += sparsity_increment\n\n        # Prune\n        prune_model(model, sparsity=current_sparsity)\n\n        # Fine-tune (recover accuracy)\n        fine_tune(model, epochs=2, lr=1e-5)\n\n    return model\n\n# Results: Better accuracy than one-shot at high sparsity\n```\n\n## Production Deployment\n\n### Complete Pruning Pipeline\n\n```python\nfrom transformers import Trainer, TrainingArguments\n\ndef production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\",  # or \"sparsegpt\"\n):\n    # 1. Load model\n    model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)\n    tokenizer = AutoTokenizer.from_pretrained(model_name)\n\n    # 2. Load calibration data\n    calib_dataset = load_dataset(\"wikitext\", \"wikitext-2-raw-v1\", split=\"train[:1000]\")\n\n    # 3. Apply pruning\n    if method == \"wanda\":\n        pruned_model = wanda_prune(model, calib_dataset, sparsity=target_sparsity)\n    elif method == \"sparsegpt\":\n        pruner = SparseGPT(model)\n        pruned_model = pruner.prune(calib_dataset, sparsity=target_sparsity)\n\n    # 4. (Optional) Fine-tune to recover accuracy\n    training_args = TrainingArguments(\n        output_dir=\".\u002Fpruned-model\",\n        num_train_epochs=1,\n        per_device_train_batch_size=4,\n        learning_rate=1e-5,\n        bf16=True,\n    )\n\n    trainer = Trainer(\n        model=pruned_model,\n        args=training_args,\n        train_dataset=finetune_dataset,\n    )\n\n    trainer.train()\n\n    # 5. Save\n    pruned_model.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n    tokenizer.save_pretrained(\".\u002Fpruned-llama-7b-50\")\n\n    return pruned_model\n\n# Usage\npruned_model = production_pruning_pipeline(\n    model_name=\"meta-llama\u002FLlama-2-7b-hf\",\n    target_sparsity=0.5,\n    method=\"wanda\"\n)\n```\n\n### Evaluation\n\n```python\nfrom lm_eval import evaluator\n\n# Evaluate pruned vs original model\noriginal_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=meta-llama\u002FLlama-2-7b-hf\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\npruned_results = evaluator.simple_evaluate(\n    model=\"hf\",\n    model_args=\"pretrained=.\u002Fpruned-llama-7b-50\",\n    tasks=[\"arc_easy\", \"hellaswag\", \"winogrande\"],\n)\n\n# Compare\nprint(f\"Original: {original_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Pruned:   {pruned_results['results']['arc_easy']['acc']:.3f}\")\nprint(f\"Degradation: {(original_results - pruned_results):.3f}\")\n\n# Typical results at 50% sparsity:\n# - Wanda: \u003C1% accuracy loss\n# - SparseGPT: \u003C0.5% accuracy loss\n# - Magnitude: 2-3% accuracy loss\n```\n\n## Best Practices\n\n### 1. Sparsity Selection\n\n```python\n# Conservative (safe)\nsparsity = 0.3  # 30%, \u003C0.5% loss\n\n# Balanced (recommended)\nsparsity = 0.5  # 50%, ~1% loss\n\n# Aggressive (risky)\nsparsity = 0.7  # 70%, 2-5% loss\n\n# Extreme (model-dependent)\nsparsity = 0.9  # 90%, significant degradation\n```\n\n### 2. Method Selection\n\n```python\n# One-shot, no retraining → Wanda or SparseGPT\nif no_retraining_budget:\n    use_method = \"wanda\"  # Faster\n\n# Best quality → SparseGPT\nif need_best_quality:\n    use_method = \"sparsegpt\"  # More accurate\n\n# Hardware speedup → N:M structured\nif need_speedup:\n    use_method = \"nm_prune\"  # 2:4 or 4:8\n```\n\n### 3. Avoid Common Pitfalls\n\n```python\n# ❌ Bad: Pruning without calibration data\nprune_random(model)  # No activation statistics\n\n# ✅ Good: Use calibration data\nprune_wanda(model, calib_data)\n\n# ❌ Bad: Too high sparsity in one shot\nprune(model, sparsity=0.9)  # Massive accuracy loss\n\n# ✅ Good: Gradual or iterative\niterative_prune(model, target=0.9, steps=10)\n```\n\n## Performance Comparison\n\n**Pruning methods at 50% sparsity** (LLaMA-7B):\n\n| Method | Accuracy Loss | Speed | Memory | Retraining Needed |\n|--------|---------------|-------|---------|-------------------|\n| **Magnitude** | -2.5% | 1.0× | -50% | No |\n| **Wanda** | -0.8% | 1.0× | -50% | No |\n| **SparseGPT** | -0.4% | 1.0× | -50% | No |\n| **N:M (2:4)** | -1.0% | 2.0× | -50% | No |\n| **Structured** | -3.0% | 2.0× | -50% | No |\n\n**Source**: Wanda paper (ICLR 2024), SparseGPT paper\n\n## Resources\n\n- **Wanda Paper (ICLR 2024)**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2306.11695\n- **Wanda GitHub**: https:\u002F\u002Fgithub.com\u002Flocuslab\u002Fwanda\n- **SparseGPT Paper**: https:\u002F\u002Farxiv.org\u002Fabs\u002F2301.00774\n- **SparseGPT GitHub**: https:\u002F\u002Fgithub.com\u002FIST-DASLab\u002Fsparsegpt\n- **NVIDIA Sparse Tensor Cores**: https:\u002F\u002Fdeveloper.nvidia.com\u002Fblog\u002Faccelerating-inference-with-sparsity-using-ampere-and-tensorrt\u002F\n\n\n",{"name":5,"tags":44,"author":45,"license":15,"version":46,"category":47,"description":7,"dependencies":48},[32,20,21,30,29,33,28,27,26,25,38],"Synthetic Sciences","1.0.0","ml-training",[49,50],"transformers","torch","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fmodel-pruning","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":58,"prompt":68,"zipAvailable":69},[59,62,65],{"tool":60,"command":61},"npx","npx skills add fadrienne\u002FMyGitHub --skill model-pruning",{"tool":63,"command":64},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill model-pruning",{"tool":66,"command":67},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill model-pruning","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fmodel-pruning\u002FSKILL.md",true,[71],{"path":72,"resourceType":73,"resourceTypeDesc":74,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fwanda.md","REFERENCE","参考文档",9492,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fmodel-pruning\u002Freferences\u002Fwanda.md",[78,109,136,170,197,224],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":89,"publishedAt":39},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[84],{"categoryNo":85,"categoryName":86,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[90,91,92,93,94,95,96,97,22,98,99,100,101,102,103,104,105,106,107,108],"power","statistical power","effect","scipy","regression","choose","design","uv","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":118,"publishedAt":39},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[115],{"categoryNo":116,"categoryName":117,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000008","测试与质量",[100,119,90,120,121,122,22,123,124,125,126,127,128,129,130,131,132,133,134,135],"statistical analysis","academic research","implementing","selection","you","reporting","data","for","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":151,"publishedAt":39},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[142,145,146],{"categoryNo":143,"categoryName":144,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":85,"categoryName":86,"taxonomyType":87,"taxonomyTypeDesc":88},{"categoryNo":147,"categoryName":148,"taxonomyType":149,"taxonomyTypeDesc":150},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[152,153,154,155,156,157,158,159,160,161,98,162,163,164,165,166,167,168,124,169],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":171,"name":172,"nameZh":6,"description":173,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":174,"categories":175,"keywords":180,"publishedAt":39},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[176,179],{"categoryNo":177,"categoryName":178,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":85,"categoryName":86,"taxonomyType":87,"taxonomyTypeDesc":88},[181,153,182,183,184,185,186,187,188,160,22,189,190,191,192,193,194,195,126,196],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":198,"name":199,"nameZh":6,"description":200,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":201,"categories":202,"keywords":209,"publishedAt":39},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[203,206],{"categoryNo":204,"categoryName":205,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":207,"categoryName":208,"taxonomyType":149,"taxonomyTypeDesc":150},"01JBSEEDCAT000000000000030","开发工程师",[210,211,131,212,213,214,215,97,216,22,98,100,217,218,219,220,221,222,168,223],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":225,"name":226,"nameZh":6,"description":227,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":228,"licenseFileUrl":6,"sourceHtmlUrl":229,"categories":230,"keywords":237,"publishedAt":39},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[231,234,235,236],{"categoryNo":232,"categoryName":233,"taxonomyType":87,"taxonomyTypeDesc":88},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":143,"categoryName":144,"taxonomyType":87,"taxonomyTypeDesc":88},{"categoryNo":85,"categoryName":86,"taxonomyType":87,"taxonomyTypeDesc":88},{"categoryNo":147,"categoryName":148,"taxonomyType":149,"taxonomyTypeDesc":150},[152,238,239,240,241,242,181,243,244,22,98,100,245,246,247,248,249,250,153,168],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]