[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6ADWS8GZAYZS4Z1KM8M4W28":3,"skill-same:01M6ADWS8GZAYZS4Z1KM8M4W28":181},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":49,"linkOnly":50,"markdownContent":51,"markdownBody":52,"frontMatter":53,"refName":67,"commitSha":68,"skillDirectory":69,"forksCount":70,"repositoryHtmlUrl":71,"repositoryHomepageUrl":72,"ownerHtmlUrl":73,"ownerBlogUrl":74,"ownerDescription":75,"updatedAt":76,"install":77,"resources":90},"01M6ADWS8GZAYZS4Z1KM8M4W28","post-training-grpo-rl-training",null,"Expert guidance for GRPO\u002FRL fine-tuning with TRL for reasoning and task-specific model training","davila7\u002Fclaude-code-templates","davila7","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6216945?v=4","01M6ADEWWFPS191NFM0Y1EX1JH","Daniel Avila","01M6ADJHGM78PHQRD3XZTFDVKK",33481,"MIT","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48],"training","GRPO","post training grpo rl training","post","grpo rl training","rl","for","research","orchestra","Reward Modeling","TRL","structured output","modeling","expert","model","learning","tuning","Transformers","PPO","RLHF","Post-Training","DPO","Fine-Tuning","reinforcement learning","reasoning","2026-11-10 10:12:30",false,"---\nname: post-training-grpo-rl-training\ndescription: Expert guidance for GRPO\u002FRL fine-tuning with TRL for reasoning and task-specific model training\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  author: Orchestra Research\n  tags: [Post-Training, Reinforcement Learning, GRPO, TRL, RLHF, Reward Modeling, Reasoning, DPO, PPO, Structured Output]\n  dependencies: [transformers>=4.47.0, trl>=0.14.0, datasets>=3.2.0, peft>=0.14.0, torch]\n---\n\n# GRPO\u002FRL Training with TRL\n\nExpert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.\n\n## When to Use This Skill\n\nUse GRPO training when you need to:\n- **Enforce specific output formats** (e.g., XML tags, JSON, structured reasoning)\n- **Teach verifiable tasks** with objective correctness metrics (math, coding, fact-checking)\n- **Improve reasoning capabilities** by rewarding chain-of-thought patterns\n- **Align models to domain-specific behaviors** without labeled preference data\n- **Optimize for multiple objectives** simultaneously (format + correctness + style)\n\n**Do NOT use GRPO for:**\n- Simple supervised fine-tuning tasks (use SFT instead)\n- Tasks without clear reward signals\n- When you already have high-quality preference pairs (use DPO\u002FPPO instead)\n\n---\n\n## Core Concepts\n\n### 1. GRPO Algorithm Fundamentals\n\n**Key Mechanism:**\n- Generates **multiple completions** for each prompt (group size: 4-16)\n- Compares completions within each group using reward functions\n- Updates policy to favor higher-rewarded responses relative to the group\n\n**Critical Difference from PPO:**\n- No separate reward model needed\n- More sample-efficient (learns from within-group comparisons)\n- Simpler to implement and debug\n\n**Mathematical Intuition:**\n```\nFor each prompt p:\n  1. Generate N completions: {c₁, c₂, ..., cₙ}\n  2. Compute rewards: {r₁, r₂, ..., rₙ}\n  3. Learn to increase probability of high-reward completions\n     relative to low-reward ones in the same group\n```\n\n### 2. Reward Function Design Philosophy\n\n**Golden Rules:**\n1. **Compose multiple reward functions** - Each handles one aspect (format, correctness, style)\n2. **Scale rewards appropriately** - Higher weight = stronger signal\n3. **Use incremental rewards** - Partial credit for partial compliance\n4. **Test rewards independently** - Debug each reward function in isolation\n\n**Reward Function Types:**\n\n| Type | Use Case | Example Weight |\n|------|----------|----------------|\n| **Correctness** | Verifiable tasks (math, code) | 2.0 (highest) |\n| **Format** | Strict structure enforcement | 0.5-1.0 |\n| **Length** | Encourage verbosity\u002Fconciseness | 0.1-0.5 |\n| **Style** | Penalize unwanted patterns | -0.5 to 0.5 |\n\n---\n\n## Implementation Workflow\n\n### Step 1: Dataset Preparation\n\n**Critical Requirements:**\n- Prompts in chat format (list of dicts with 'role' and 'content')\n- Include system prompts to set expectations\n- For verifiable tasks, include ground truth answers as additional columns\n\n**Example Structure:**\n```python\nfrom datasets import load_dataset, Dataset\n\nSYSTEM_PROMPT = \"\"\"\nRespond in the following format:\n\u003Creasoning>\n[Your step-by-step thinking]\n\u003C\u002Freasoning>\n\u003Canswer>\n[Final answer]\n\u003C\u002Fanswer>\n\"\"\"\n\ndef prepare_dataset(raw_data):\n    \"\"\"\n    Transform raw data into GRPO-compatible format.\n\n    Returns: Dataset with columns:\n    - 'prompt': List[Dict] with role\u002Fcontent (system + user messages)\n    - 'answer': str (ground truth, optional but recommended)\n    \"\"\"\n    return raw_data.map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': extract_answer(x['raw_answer'])\n    })\n```\n\n**Pro Tips:**\n- Use one-shot or few-shot examples in system prompt for complex formats\n- Keep prompts concise (max_prompt_length: 256-512 tokens)\n- Validate data quality before training (garbage in = garbage out)\n\n### Step 2: Reward Function Implementation\n\n**Template Structure:**\n```python\ndef reward_function_name(\n    prompts,        # List[List[Dict]]: Original prompts\n    completions,    # List[List[Dict]]: Model generations\n    answer=None,    # Optional: Ground truth from dataset\n    **kwargs        # Additional dataset columns\n) -> list[float]:\n    \"\"\"\n    Evaluate completions and return rewards.\n\n    Returns: List of floats (one per completion)\n    \"\"\"\n    # Extract completion text\n    responses = [comp[0]['content'] for comp in completions]\n\n    # Compute rewards\n    rewards = []\n    for response in responses:\n        score = compute_score(response)\n        rewards.append(score)\n\n    return rewards\n```\n\n**Example 1: Correctness Reward (Math\u002FCoding)**\n```python\ndef correctness_reward(prompts, completions, answer, **kwargs):\n    \"\"\"Reward correct answers with high score.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    extracted = [extract_final_answer(r) for r in responses]\n    return [2.0 if ans == gt else 0.0\n            for ans, gt in zip(extracted, answer)]\n```\n\n**Example 2: Format Reward (Structured Output)**\n```python\nimport re\n\ndef format_reward(completions, **kwargs):\n    \"\"\"Reward XML-like structured format.\"\"\"\n    pattern = r'\u003Creasoning>.*?\u003C\u002Freasoning>\\s*\u003Canswer>.*?\u003C\u002Fanswer>'\n    responses = [comp[0]['content'] for comp in completions]\n    return [1.0 if re.search(pattern, r, re.DOTALL) else 0.0\n            for r in responses]\n```\n\n**Example 3: Incremental Format Reward (Partial Credit)**\n```python\ndef incremental_format_reward(completions, **kwargs):\n    \"\"\"Award partial credit for format compliance.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    rewards = []\n\n    for r in responses:\n        score = 0.0\n        if '\u003Creasoning>' in r:\n            score += 0.25\n        if '\u003C\u002Freasoning>' in r:\n            score += 0.25\n        if '\u003Canswer>' in r:\n            score += 0.25\n        if '\u003C\u002Fanswer>' in r:\n            score += 0.25\n        # Penalize extra text after closing tag\n        if r.count('\u003C\u002Fanswer>') == 1:\n            extra_text = r.split('\u003C\u002Fanswer>')[-1].strip()\n            score -= len(extra_text) * 0.001\n        rewards.append(score)\n\n    return rewards\n```\n\n**Critical Insight:**\nCombine 3-5 reward functions for robust training. Order matters less than diversity of signals.\n\n### Step 3: Training Configuration\n\n**Memory-Optimized Config (Small GPU)**\n```python\nfrom trl import GRPOConfig\n\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n\n    # Learning rate\n    learning_rate=5e-6,          # Lower = more stable\n    adam_beta1=0.9,\n    adam_beta2=0.99,\n    weight_decay=0.1,\n    warmup_ratio=0.1,\n    lr_scheduler_type='cosine',\n\n    # Batch settings\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=4,  # Effective batch = 4\n\n    # GRPO-specific\n    num_generations=8,            # Group size: 8-16 recommended\n    max_prompt_length=256,\n    max_completion_length=512,\n\n    # Training duration\n    num_train_epochs=1,\n    max_steps=None,               # Or set fixed steps (e.g., 500)\n\n    # Optimization\n    bf16=True,                    # Faster on A100\u002FH100\n    optim=\"adamw_8bit\",          # Memory-efficient optimizer\n    max_grad_norm=0.1,\n\n    # Logging\n    logging_steps=1,\n    save_steps=100,\n    report_to=\"wandb\",            # Or \"none\" for no logging\n)\n```\n\n**High-Performance Config (Large GPU)**\n```python\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n    learning_rate=1e-5,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=2,\n    num_generations=16,           # Larger groups = better signal\n    max_prompt_length=512,\n    max_completion_length=1024,\n    num_train_epochs=1,\n    bf16=True,\n    use_vllm=True,                # Fast generation with vLLM\n    logging_steps=10,\n)\n```\n\n**Critical Hyperparameters:**\n\n| Parameter | Impact | Tuning Advice |\n|-----------|--------|---------------|\n| `num_generations` | Group size for comparison | Start with 8, increase to 16 if GPU allows |\n| `learning_rate` | Convergence speed\u002Fstability | 5e-6 (safe), 1e-5 (faster, riskier) |\n| `max_completion_length` | Output verbosity | Match your task (512 for reasoning, 256 for short answers) |\n| `gradient_accumulation_steps` | Effective batch size | Increase if GPU memory limited |\n\n### Step 4: Model Setup and Training\n\n**Standard Setup (Transformers)**\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\nfrom trl import GRPOTrainer\n\n# Load model\nmodel_name = \"Qwen\u002FQwen2.5-1.5B-Instruct\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",  # 2-3x faster\n    device_map=\"auto\"\n)\n\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# Optional: LoRA for parameter-efficient training\npeft_config = LoraConfig(\n    r=16,                         # Rank (higher = more capacity)\n    lora_alpha=32,               # Scaling factor (typically 2*r)\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\"\n    ],\n    task_type=\"CAUSAL_LM\",\n    lora_dropout=0.05,\n)\n\n# Initialize trainer\ntrainer = GRPOTrainer(\n    model=model,\n    processing_class=tokenizer,\n    reward_funcs=[\n        incremental_format_reward,\n        format_reward,\n        correctness_reward,\n    ],\n    args=training_args,\n    train_dataset=dataset,\n    peft_config=peft_config,      # Remove for full fine-tuning\n)\n\n# Train\ntrainer.train()\n\n# Save\ntrainer.save_model(\"final_model\")\n```\n\n**Unsloth Setup (2-3x Faster)**\n```python\nfrom unsloth import FastLanguageModel\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"google\u002Fgemma-3-1b-it\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,\n    max_lora_rank=32,\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=32,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=32,\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Rest is identical to standard setup\ntrainer = GRPOTrainer(model=model, ...)\ntrainer.train()\n```\n\n---\n\n## Critical Training Insights\n\n### 1. Loss Behavior (EXPECTED PATTERN)\n- **Loss starts near 0 and INCREASES during training**\n- This is CORRECT - loss measures KL divergence from initial policy\n- Model is learning (diverging from original behavior to optimize rewards)\n- Monitor reward metrics instead of loss for progress\n\n### 2. Reward Tracking\nKey metrics to watch:\n- `reward`: Average across all completions\n- `reward_std`: Diversity within groups (should remain > 0)\n- `kl`: KL divergence from reference (should grow moderately)\n\n**Healthy Training Pattern:**\n```\nStep   Reward    Reward_Std   KL\n100    0.5       0.3          0.02\n200    0.8       0.25         0.05\n300    1.2       0.2          0.08  ← Good progression\n400    1.5       0.15         0.12\n```\n\n**Warning Signs:**\n- Reward std → 0 (model collapsing to single response)\n- KL exploding (> 0.5) (diverging too much, reduce LR)\n- Reward stuck (reward functions too harsh or model capacity issue)\n\n### 3. Common Pitfalls and Solutions\n\n| Problem | Symptom | Solution |\n|---------|---------|----------|\n| **Mode collapse** | All completions identical | Increase `num_generations`, add diversity penalty |\n| **No learning** | Flat rewards | Check reward function logic, increase LR |\n| **OOM errors** | GPU memory exceeded | Reduce `num_generations`, enable gradient checkpointing |\n| **Slow training** | \u003C 1 it\u002Fs | Enable `use_vllm=True`, use Unsloth, reduce seq length |\n| **Format ignored** | Model doesn't follow structure | Increase format reward weight, add incremental rewards |\n\n---\n\n## Advanced Patterns\n\n### 1. Multi-Stage Training\nFor complex tasks, train in stages:\n\n```python\n# Stage 1: Format compliance (epochs=1)\ntrainer_stage1 = GRPOTrainer(\n    model=model,\n    reward_funcs=[incremental_format_reward, format_reward],\n    ...\n)\ntrainer_stage1.train()\n\n# Stage 2: Correctness (epochs=1)\ntrainer_stage2 = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    ...\n)\ntrainer_stage2.train()\n```\n\n### 2. Adaptive Reward Scaling\n```python\nclass AdaptiveReward:\n    def __init__(self, base_reward_func, initial_weight=1.0):\n        self.func = base_reward_func\n        self.weight = initial_weight\n\n    def __call__(self, *args, **kwargs):\n        rewards = self.func(*args, **kwargs)\n        return [r * self.weight for r in rewards]\n\n    def adjust_weight(self, success_rate):\n        \"\"\"Increase weight if model struggling, decrease if succeeding.\"\"\"\n        if success_rate \u003C 0.3:\n            self.weight *= 1.2\n        elif success_rate > 0.8:\n            self.weight *= 0.9\n```\n\n### 3. Custom Dataset Integration\n```python\ndef load_custom_knowledge_base(csv_path):\n    \"\"\"Example: School communication platform docs.\"\"\"\n    import pandas as pd\n    df = pd.read_csv(csv_path)\n\n    dataset = Dataset.from_pandas(df).map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': CUSTOM_SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': x['expert_answer']\n    })\n    return dataset\n```\n\n---\n\n## Deployment and Inference\n\n### Save and Merge LoRA\n```python\n# Merge LoRA adapters into base model\nif hasattr(trainer.model, 'merge_and_unload'):\n    merged_model = trainer.model.merge_and_unload()\n    merged_model.save_pretrained(\"production_model\")\n    tokenizer.save_pretrained(\"production_model\")\n```\n\n### Inference Example\n```python\nfrom transformers import pipeline\n\ngenerator = pipeline(\n    \"text-generation\",\n    model=\"production_model\",\n    tokenizer=tokenizer\n)\n\nresult = generator(\n    [\n        {'role': 'system', 'content': SYSTEM_PROMPT},\n        {'role': 'user', 'content': \"What is 15 + 27?\"}\n    ],\n    max_new_tokens=256,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.9\n)\nprint(result[0]['generated_text'])\n```\n\n---\n\n## Best Practices Checklist\n\n**Before Training:**\n- [ ] Validate dataset format (prompts as List[Dict])\n- [ ] Test reward functions on sample data\n- [ ] Calculate expected max_prompt_length from data\n- [ ] Choose appropriate num_generations based on GPU memory\n- [ ] Set up logging (wandb recommended)\n\n**During Training:**\n- [ ] Monitor reward progression (should increase)\n- [ ] Check reward_std (should stay > 0.1)\n- [ ] Watch for OOM errors (reduce batch size if needed)\n- [ ] Sample generations every 50-100 steps\n- [ ] Validate format compliance on holdout set\n\n**After Training:**\n- [ ] Merge LoRA weights if using PEFT\n- [ ] Test on diverse prompts\n- [ ] Compare to baseline model\n- [ ] Document reward weights and hyperparameters\n- [ ] Save reproducibility config\n\n---\n\n## Troubleshooting Guide\n\n### Debugging Workflow\n1. **Isolate reward functions** - Test each independently\n2. **Check data distribution** - Ensure diversity in prompts\n3. **Reduce complexity** - Start with single reward, add gradually\n4. **Monitor generations** - Print samples every N steps\n5. **Validate extraction logic** - Ensure answer parsing works\n\n### Quick Fixes\n```python\n# Debug reward function\ndef debug_reward(completions, **kwargs):\n    responses = [comp[0]['content'] for comp in completions]\n    for i, r in enumerate(responses[:2]):  # Print first 2\n        print(f\"Response {i}: {r[:200]}...\")\n    return [1.0] * len(responses)  # Dummy rewards\n\n# Test without training\ntrainer = GRPOTrainer(..., reward_funcs=[debug_reward])\ntrainer.generate_completions(dataset[:1])  # Generate without updating\n```\n\n---\n\n## References and Resources\n\n**Official Documentation:**\n- TRL GRPO Trainer: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftrl\u002Fgrpo_trainer\n- DeepSeek R1 Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.12948\n- Unsloth Docs: https:\u002F\u002Fdocs.unsloth.ai\u002F\n\n**Example Repositories:**\n- Open R1 Implementation: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fopen-r1\n- TRL Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Ftree\u002Fmain\u002Fexamples\n\n**Recommended Reading:**\n- Progressive Disclosure Pattern for agent instructions\n- Reward shaping in RL (Ng et al.)\n- LoRA paper (Hu et al., 2021)\n\n---\n\n## Usage Instructions for Agents\n\nWhen this skill is loaded:\n\n1. **Read this entire file** before implementing GRPO training\n2. **Start with the simplest reward function** (e.g., length-based) to validate setup\n3. **Use the templates** in `templates\u002F` directory as starting points\n4. **Reference examples** in `examples\u002F` for task-specific implementations\n5. **Follow the workflow** sequentially (don't skip steps)\n6. **Debug incrementally** - add one reward function at a time\n\n**Critical Reminders:**\n- Always use multiple reward functions (3-5 is optimal)\n- Monitor reward metrics, not loss\n- Test reward functions before training\n- Start small (num_generations=4), scale up gradually\n- Save checkpoints frequently (every 100 steps)\n\nThis skill is designed for **expert-level implementation**. Beginners should start with supervised fine-tuning before attempting GRPO.\n\n\n\n","# GRPO\u002FRL Training with TRL\n\nExpert-level guidance for implementing Group Relative Policy Optimization (GRPO) using the Transformer Reinforcement Learning (TRL) library. This skill provides battle-tested patterns, critical insights, and production-ready workflows for fine-tuning language models with custom reward functions.\n\n## When to Use This Skill\n\nUse GRPO training when you need to:\n- **Enforce specific output formats** (e.g., XML tags, JSON, structured reasoning)\n- **Teach verifiable tasks** with objective correctness metrics (math, coding, fact-checking)\n- **Improve reasoning capabilities** by rewarding chain-of-thought patterns\n- **Align models to domain-specific behaviors** without labeled preference data\n- **Optimize for multiple objectives** simultaneously (format + correctness + style)\n\n**Do NOT use GRPO for:**\n- Simple supervised fine-tuning tasks (use SFT instead)\n- Tasks without clear reward signals\n- When you already have high-quality preference pairs (use DPO\u002FPPO instead)\n\n---\n\n## Core Concepts\n\n### 1. GRPO Algorithm Fundamentals\n\n**Key Mechanism:**\n- Generates **multiple completions** for each prompt (group size: 4-16)\n- Compares completions within each group using reward functions\n- Updates policy to favor higher-rewarded responses relative to the group\n\n**Critical Difference from PPO:**\n- No separate reward model needed\n- More sample-efficient (learns from within-group comparisons)\n- Simpler to implement and debug\n\n**Mathematical Intuition:**\n```\nFor each prompt p:\n  1. Generate N completions: {c₁, c₂, ..., cₙ}\n  2. Compute rewards: {r₁, r₂, ..., rₙ}\n  3. Learn to increase probability of high-reward completions\n     relative to low-reward ones in the same group\n```\n\n### 2. Reward Function Design Philosophy\n\n**Golden Rules:**\n1. **Compose multiple reward functions** - Each handles one aspect (format, correctness, style)\n2. **Scale rewards appropriately** - Higher weight = stronger signal\n3. **Use incremental rewards** - Partial credit for partial compliance\n4. **Test rewards independently** - Debug each reward function in isolation\n\n**Reward Function Types:**\n\n| Type | Use Case | Example Weight |\n|------|----------|----------------|\n| **Correctness** | Verifiable tasks (math, code) | 2.0 (highest) |\n| **Format** | Strict structure enforcement | 0.5-1.0 |\n| **Length** | Encourage verbosity\u002Fconciseness | 0.1-0.5 |\n| **Style** | Penalize unwanted patterns | -0.5 to 0.5 |\n\n---\n\n## Implementation Workflow\n\n### Step 1: Dataset Preparation\n\n**Critical Requirements:**\n- Prompts in chat format (list of dicts with 'role' and 'content')\n- Include system prompts to set expectations\n- For verifiable tasks, include ground truth answers as additional columns\n\n**Example Structure:**\n```python\nfrom datasets import load_dataset, Dataset\n\nSYSTEM_PROMPT = \"\"\"\nRespond in the following format:\n\u003Creasoning>\n[Your step-by-step thinking]\n\u003C\u002Freasoning>\n\u003Canswer>\n[Final answer]\n\u003C\u002Fanswer>\n\"\"\"\n\ndef prepare_dataset(raw_data):\n    \"\"\"\n    Transform raw data into GRPO-compatible format.\n\n    Returns: Dataset with columns:\n    - 'prompt': List[Dict] with role\u002Fcontent (system + user messages)\n    - 'answer': str (ground truth, optional but recommended)\n    \"\"\"\n    return raw_data.map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': extract_answer(x['raw_answer'])\n    })\n```\n\n**Pro Tips:**\n- Use one-shot or few-shot examples in system prompt for complex formats\n- Keep prompts concise (max_prompt_length: 256-512 tokens)\n- Validate data quality before training (garbage in = garbage out)\n\n### Step 2: Reward Function Implementation\n\n**Template Structure:**\n```python\ndef reward_function_name(\n    prompts,        # List[List[Dict]]: Original prompts\n    completions,    # List[List[Dict]]: Model generations\n    answer=None,    # Optional: Ground truth from dataset\n    **kwargs        # Additional dataset columns\n) -> list[float]:\n    \"\"\"\n    Evaluate completions and return rewards.\n\n    Returns: List of floats (one per completion)\n    \"\"\"\n    # Extract completion text\n    responses = [comp[0]['content'] for comp in completions]\n\n    # Compute rewards\n    rewards = []\n    for response in responses:\n        score = compute_score(response)\n        rewards.append(score)\n\n    return rewards\n```\n\n**Example 1: Correctness Reward (Math\u002FCoding)**\n```python\ndef correctness_reward(prompts, completions, answer, **kwargs):\n    \"\"\"Reward correct answers with high score.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    extracted = [extract_final_answer(r) for r in responses]\n    return [2.0 if ans == gt else 0.0\n            for ans, gt in zip(extracted, answer)]\n```\n\n**Example 2: Format Reward (Structured Output)**\n```python\nimport re\n\ndef format_reward(completions, **kwargs):\n    \"\"\"Reward XML-like structured format.\"\"\"\n    pattern = r'\u003Creasoning>.*?\u003C\u002Freasoning>\\s*\u003Canswer>.*?\u003C\u002Fanswer>'\n    responses = [comp[0]['content'] for comp in completions]\n    return [1.0 if re.search(pattern, r, re.DOTALL) else 0.0\n            for r in responses]\n```\n\n**Example 3: Incremental Format Reward (Partial Credit)**\n```python\ndef incremental_format_reward(completions, **kwargs):\n    \"\"\"Award partial credit for format compliance.\"\"\"\n    responses = [comp[0]['content'] for comp in completions]\n    rewards = []\n\n    for r in responses:\n        score = 0.0\n        if '\u003Creasoning>' in r:\n            score += 0.25\n        if '\u003C\u002Freasoning>' in r:\n            score += 0.25\n        if '\u003Canswer>' in r:\n            score += 0.25\n        if '\u003C\u002Fanswer>' in r:\n            score += 0.25\n        # Penalize extra text after closing tag\n        if r.count('\u003C\u002Fanswer>') == 1:\n            extra_text = r.split('\u003C\u002Fanswer>')[-1].strip()\n            score -= len(extra_text) * 0.001\n        rewards.append(score)\n\n    return rewards\n```\n\n**Critical Insight:**\nCombine 3-5 reward functions for robust training. Order matters less than diversity of signals.\n\n### Step 3: Training Configuration\n\n**Memory-Optimized Config (Small GPU)**\n```python\nfrom trl import GRPOConfig\n\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n\n    # Learning rate\n    learning_rate=5e-6,          # Lower = more stable\n    adam_beta1=0.9,\n    adam_beta2=0.99,\n    weight_decay=0.1,\n    warmup_ratio=0.1,\n    lr_scheduler_type='cosine',\n\n    # Batch settings\n    per_device_train_batch_size=1,\n    gradient_accumulation_steps=4,  # Effective batch = 4\n\n    # GRPO-specific\n    num_generations=8,            # Group size: 8-16 recommended\n    max_prompt_length=256,\n    max_completion_length=512,\n\n    # Training duration\n    num_train_epochs=1,\n    max_steps=None,               # Or set fixed steps (e.g., 500)\n\n    # Optimization\n    bf16=True,                    # Faster on A100\u002FH100\n    optim=\"adamw_8bit\",          # Memory-efficient optimizer\n    max_grad_norm=0.1,\n\n    # Logging\n    logging_steps=1,\n    save_steps=100,\n    report_to=\"wandb\",            # Or \"none\" for no logging\n)\n```\n\n**High-Performance Config (Large GPU)**\n```python\ntraining_args = GRPOConfig(\n    output_dir=\"outputs\u002Fgrpo-model\",\n    learning_rate=1e-5,\n    per_device_train_batch_size=4,\n    gradient_accumulation_steps=2,\n    num_generations=16,           # Larger groups = better signal\n    max_prompt_length=512,\n    max_completion_length=1024,\n    num_train_epochs=1,\n    bf16=True,\n    use_vllm=True,                # Fast generation with vLLM\n    logging_steps=10,\n)\n```\n\n**Critical Hyperparameters:**\n\n| Parameter | Impact | Tuning Advice |\n|-----------|--------|---------------|\n| `num_generations` | Group size for comparison | Start with 8, increase to 16 if GPU allows |\n| `learning_rate` | Convergence speed\u002Fstability | 5e-6 (safe), 1e-5 (faster, riskier) |\n| `max_completion_length` | Output verbosity | Match your task (512 for reasoning, 256 for short answers) |\n| `gradient_accumulation_steps` | Effective batch size | Increase if GPU memory limited |\n\n### Step 4: Model Setup and Training\n\n**Standard Setup (Transformers)**\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\nfrom trl import GRPOTrainer\n\n# Load model\nmodel_name = \"Qwen\u002FQwen2.5-1.5B-Instruct\"\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",  # 2-3x faster\n    device_map=\"auto\"\n)\n\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# Optional: LoRA for parameter-efficient training\npeft_config = LoraConfig(\n    r=16,                         # Rank (higher = more capacity)\n    lora_alpha=32,               # Scaling factor (typically 2*r)\n    target_modules=[\n        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n        \"gate_proj\", \"up_proj\", \"down_proj\"\n    ],\n    task_type=\"CAUSAL_LM\",\n    lora_dropout=0.05,\n)\n\n# Initialize trainer\ntrainer = GRPOTrainer(\n    model=model,\n    processing_class=tokenizer,\n    reward_funcs=[\n        incremental_format_reward,\n        format_reward,\n        correctness_reward,\n    ],\n    args=training_args,\n    train_dataset=dataset,\n    peft_config=peft_config,      # Remove for full fine-tuning\n)\n\n# Train\ntrainer.train()\n\n# Save\ntrainer.save_model(\"final_model\")\n```\n\n**Unsloth Setup (2-3x Faster)**\n```python\nfrom unsloth import FastLanguageModel\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"google\u002Fgemma-3-1b-it\",\n    max_seq_length=1024,\n    load_in_4bit=True,\n    fast_inference=True,\n    max_lora_rank=32,\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=32,\n    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n                    \"gate_proj\", \"up_proj\", \"down_proj\"],\n    lora_alpha=32,\n    use_gradient_checkpointing=\"unsloth\",\n)\n\n# Rest is identical to standard setup\ntrainer = GRPOTrainer(model=model, ...)\ntrainer.train()\n```\n\n---\n\n## Critical Training Insights\n\n### 1. Loss Behavior (EXPECTED PATTERN)\n- **Loss starts near 0 and INCREASES during training**\n- This is CORRECT - loss measures KL divergence from initial policy\n- Model is learning (diverging from original behavior to optimize rewards)\n- Monitor reward metrics instead of loss for progress\n\n### 2. Reward Tracking\nKey metrics to watch:\n- `reward`: Average across all completions\n- `reward_std`: Diversity within groups (should remain > 0)\n- `kl`: KL divergence from reference (should grow moderately)\n\n**Healthy Training Pattern:**\n```\nStep   Reward    Reward_Std   KL\n100    0.5       0.3          0.02\n200    0.8       0.25         0.05\n300    1.2       0.2          0.08  ← Good progression\n400    1.5       0.15         0.12\n```\n\n**Warning Signs:**\n- Reward std → 0 (model collapsing to single response)\n- KL exploding (> 0.5) (diverging too much, reduce LR)\n- Reward stuck (reward functions too harsh or model capacity issue)\n\n### 3. Common Pitfalls and Solutions\n\n| Problem | Symptom | Solution |\n|---------|---------|----------|\n| **Mode collapse** | All completions identical | Increase `num_generations`, add diversity penalty |\n| **No learning** | Flat rewards | Check reward function logic, increase LR |\n| **OOM errors** | GPU memory exceeded | Reduce `num_generations`, enable gradient checkpointing |\n| **Slow training** | \u003C 1 it\u002Fs | Enable `use_vllm=True`, use Unsloth, reduce seq length |\n| **Format ignored** | Model doesn't follow structure | Increase format reward weight, add incremental rewards |\n\n---\n\n## Advanced Patterns\n\n### 1. Multi-Stage Training\nFor complex tasks, train in stages:\n\n```python\n# Stage 1: Format compliance (epochs=1)\ntrainer_stage1 = GRPOTrainer(\n    model=model,\n    reward_funcs=[incremental_format_reward, format_reward],\n    ...\n)\ntrainer_stage1.train()\n\n# Stage 2: Correctness (epochs=1)\ntrainer_stage2 = GRPOTrainer(\n    model=model,\n    reward_funcs=[format_reward, correctness_reward],\n    ...\n)\ntrainer_stage2.train()\n```\n\n### 2. Adaptive Reward Scaling\n```python\nclass AdaptiveReward:\n    def __init__(self, base_reward_func, initial_weight=1.0):\n        self.func = base_reward_func\n        self.weight = initial_weight\n\n    def __call__(self, *args, **kwargs):\n        rewards = self.func(*args, **kwargs)\n        return [r * self.weight for r in rewards]\n\n    def adjust_weight(self, success_rate):\n        \"\"\"Increase weight if model struggling, decrease if succeeding.\"\"\"\n        if success_rate \u003C 0.3:\n            self.weight *= 1.2\n        elif success_rate > 0.8:\n            self.weight *= 0.9\n```\n\n### 3. Custom Dataset Integration\n```python\ndef load_custom_knowledge_base(csv_path):\n    \"\"\"Example: School communication platform docs.\"\"\"\n    import pandas as pd\n    df = pd.read_csv(csv_path)\n\n    dataset = Dataset.from_pandas(df).map(lambda x: {\n        'prompt': [\n            {'role': 'system', 'content': CUSTOM_SYSTEM_PROMPT},\n            {'role': 'user', 'content': x['question']}\n        ],\n        'answer': x['expert_answer']\n    })\n    return dataset\n```\n\n---\n\n## Deployment and Inference\n\n### Save and Merge LoRA\n```python\n# Merge LoRA adapters into base model\nif hasattr(trainer.model, 'merge_and_unload'):\n    merged_model = trainer.model.merge_and_unload()\n    merged_model.save_pretrained(\"production_model\")\n    tokenizer.save_pretrained(\"production_model\")\n```\n\n### Inference Example\n```python\nfrom transformers import pipeline\n\ngenerator = pipeline(\n    \"text-generation\",\n    model=\"production_model\",\n    tokenizer=tokenizer\n)\n\nresult = generator(\n    [\n        {'role': 'system', 'content': SYSTEM_PROMPT},\n        {'role': 'user', 'content': \"What is 15 + 27?\"}\n    ],\n    max_new_tokens=256,\n    do_sample=True,\n    temperature=0.7,\n    top_p=0.9\n)\nprint(result[0]['generated_text'])\n```\n\n---\n\n## Best Practices Checklist\n\n**Before Training:**\n- [ ] Validate dataset format (prompts as List[Dict])\n- [ ] Test reward functions on sample data\n- [ ] Calculate expected max_prompt_length from data\n- [ ] Choose appropriate num_generations based on GPU memory\n- [ ] Set up logging (wandb recommended)\n\n**During Training:**\n- [ ] Monitor reward progression (should increase)\n- [ ] Check reward_std (should stay > 0.1)\n- [ ] Watch for OOM errors (reduce batch size if needed)\n- [ ] Sample generations every 50-100 steps\n- [ ] Validate format compliance on holdout set\n\n**After Training:**\n- [ ] Merge LoRA weights if using PEFT\n- [ ] Test on diverse prompts\n- [ ] Compare to baseline model\n- [ ] Document reward weights and hyperparameters\n- [ ] Save reproducibility config\n\n---\n\n## Troubleshooting Guide\n\n### Debugging Workflow\n1. **Isolate reward functions** - Test each independently\n2. **Check data distribution** - Ensure diversity in prompts\n3. **Reduce complexity** - Start with single reward, add gradually\n4. **Monitor generations** - Print samples every N steps\n5. **Validate extraction logic** - Ensure answer parsing works\n\n### Quick Fixes\n```python\n# Debug reward function\ndef debug_reward(completions, **kwargs):\n    responses = [comp[0]['content'] for comp in completions]\n    for i, r in enumerate(responses[:2]):  # Print first 2\n        print(f\"Response {i}: {r[:200]}...\")\n    return [1.0] * len(responses)  # Dummy rewards\n\n# Test without training\ntrainer = GRPOTrainer(..., reward_funcs=[debug_reward])\ntrainer.generate_completions(dataset[:1])  # Generate without updating\n```\n\n---\n\n## References and Resources\n\n**Official Documentation:**\n- TRL GRPO Trainer: https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftrl\u002Fgrpo_trainer\n- DeepSeek R1 Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.12948\n- Unsloth Docs: https:\u002F\u002Fdocs.unsloth.ai\u002F\n\n**Example Repositories:**\n- Open R1 Implementation: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fopen-r1\n- TRL Examples: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftrl\u002Ftree\u002Fmain\u002Fexamples\n\n**Recommended Reading:**\n- Progressive Disclosure Pattern for agent instructions\n- Reward shaping in RL (Ng et al.)\n- LoRA paper (Hu et al., 2021)\n\n---\n\n## Usage Instructions for Agents\n\nWhen this skill is loaded:\n\n1. **Read this entire file** before implementing GRPO training\n2. **Start with the simplest reward function** (e.g., length-based) to validate setup\n3. **Use the templates** in `templates\u002F` directory as starting points\n4. **Reference examples** in `examples\u002F` for task-specific implementations\n5. **Follow the workflow** sequentially (don't skip steps)\n6. **Debug incrementally** - add one reward function at a time\n\n**Critical Reminders:**\n- Always use multiple reward functions (3-5 is optimal)\n- Monitor reward metrics, not loss\n- Test reward functions before training\n- Start small (num_generations=4), scale up gradually\n- Save checkpoints frequently (every 100 steps)\n\nThis skill is designed for **expert-level implementation**. Beginners should start with supervised fine-tuning before attempting GRPO.\n\n\n\n",{"name":5,"license":15,"metadata":54,"description":7},{"tags":55,"author":59,"version":60,"dependencies":61},[44,56,25,34,43,33,57,45,42,58],"Reinforcement Learning","Reasoning","Structured Output","Orchestra Research","1.0.0",[62,63,64,65,66],"transformers>=4.47.0","trl>=0.14.0","datasets>=3.2.0","peft>=0.14.0","torch","main","2b749c82b5d11fac39d9a6e021ce3f5b8a541bd1","cli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training",3931,"https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates","https:\u002F\u002Faitmpl.com","https:\u002F\u002Fgithub.com\u002Fdavila7","https:\u002F\u002Fdanielavila.me","Building AI dev tools with LLMs","2026-11-14 06:14:46",{"source":8,"skillName":5,"commands":78,"prompt":88,"zipAvailable":89},[79,82,85],{"tool":80,"command":81},"npx","npx skills add davila7\u002Fclaude-code-templates --skill post-training-grpo-rl-training",{"tool":83,"command":84},"bunx","bunx skills add davila7\u002Fclaude-code-templates --skill post-training-grpo-rl-training",{"tool":86,"command":87},"pnpm","pnpm dlx skills add davila7\u002Fclaude-code-templates --skill post-training-grpo-rl-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training\u002FSKILL.md",true,[91,97,103,107,113,117,121,125,129,133,137,141,145,149,153,157,161,165,169,173,177],{"path":92,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":95,"sourceHtmlUrl":96},"examples\u002Freward_functions_library.py","SCRIPT","脚本",11568,"https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training\u002Fexamples\u002Freward_functions_library.py",{"path":98,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":101,"sourceHtmlUrl":102},"README.md","REFERENCE","参考文档",3514,"https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training\u002FREADME.md",{"path":104,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":105,"sourceHtmlUrl":106},"sections\u002Fadversarial.md",14331,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fadversarial.md",{"path":108,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":111,"sourceHtmlUrl":112},"sections\u002Fadversarial.md.tmpl","OTHER","其他",1217,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fadversarial.md.tmpl",{"path":114,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":115,"sourceHtmlUrl":116},"sections\u002Fchangelog.md",1869,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fchangelog.md",{"path":118,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":119,"sourceHtmlUrl":120},"sections\u002Fchangelog.md.tmpl",28,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fchangelog.md.tmpl",{"path":122,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":123,"sourceHtmlUrl":124},"sections\u002Fgreptile.md",3443,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fgreptile.md",{"path":126,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":127,"sourceHtmlUrl":128},"sections\u002Fgreptile.md.tmpl",3328,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fgreptile.md.tmpl",{"path":130,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":131,"sourceHtmlUrl":132},"sections\u002Fmanifest.json",2077,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fmanifest.json",{"path":134,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":135,"sourceHtmlUrl":136},"sections\u002Fplan-completion.md",20692,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fplan-completion.md",{"path":138,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":139,"sourceHtmlUrl":140},"sections\u002Fplan-completion.md.tmpl",2074,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fplan-completion.md.tmpl",{"path":142,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":143,"sourceHtmlUrl":144},"sections\u002Fpr-body.md",11307,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fpr-body.md",{"path":146,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":147,"sourceHtmlUrl":148},"sections\u002Fpr-body.md.tmpl",11193,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Fpr-body.md.tmpl",{"path":150,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":151,"sourceHtmlUrl":152},"sections\u002Freview-army.md",21564,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Freview-army.md",{"path":154,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":155,"sourceHtmlUrl":156},"sections\u002Freview-army.md.tmpl",3301,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Freview-army.md.tmpl",{"path":158,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":159,"sourceHtmlUrl":160},"sections\u002Ftest-coverage.md",14083,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Ftest-coverage.md",{"path":162,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":163,"sourceHtmlUrl":164},"sections\u002Ftest-coverage.md.tmpl",1393,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Ftest-coverage.md.tmpl",{"path":166,"resourceType":99,"resourceTypeDesc":100,"sizeBytes":167,"sourceHtmlUrl":168},"sections\u002Ftests.md",16529,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Ftests.md",{"path":170,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":171,"sourceHtmlUrl":172},"sections\u002Ftests.md.tmpl",4706,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002Fsections\u002Ftests.md.tmpl",{"path":174,"resourceType":109,"resourceTypeDesc":110,"sizeBytes":175,"sourceHtmlUrl":176},"SKILL.md.tmpl",25331,"https:\u002F\u002Fgithub.com\u002Fshengdabai\u002FTony-Claude-Code-Skills\u002Fblob\u002Fmain\u002Fskills\u002Fgstack\u002Fship\u002FSKILL.md.tmpl",{"path":178,"resourceType":93,"resourceTypeDesc":94,"sizeBytes":179,"sourceHtmlUrl":180},"templates\u002Fbasic_grpo_training.py",6122,"https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fpost-training-grpo-rl-training\u002Ftemplates\u002Fbasic_grpo_training.py",[182,209,236,269,304,328],{"skillNo":183,"name":184,"nameZh":6,"description":185,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":186,"categories":187,"keywords":189,"publishedAt":49},"01M70NHBTQ8BV0A2SCAZXEPHCX","wealthville-pool-scoring","Score DeFi liquidity pools before providing liquidity. Returns a 0-100 pool score and an Enter\u002FHold\u002FExit\u002FReduce\u002FAvoid verdict with calibrated confidence for ~68,800 Solana pools and 575 EVM pools, plus a public miss-inclusive track record. Use when asked whether a pool is worth LPing into, which pools look best right now, or how accurate the scores have actually been.","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fweb-data\u002Fwealthville-pool-scoring\u002FSKILL.md",[188],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[190,191,192,193,194,30,195,196,197,198,199,200,201,202,203,204,205,206,207,208],"scoring","pool","wealthville pool scoring","liquidity","which","exit","now","use","when","reduce","with","record","track","verdict","evm","solana","how","public","look",{"skillNo":210,"name":211,"nameZh":6,"description":212,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":213,"categories":214,"keywords":216,"publishedAt":49},"01M70MB6VR8ERJ4EM8D5KHBMEV","technical-debt-auditor","Scans a codebase and produces a quantified technical debt report with a prioritized remediation plan. Analyzes dead code, TODO\u002FFIXME\u002FHACK markers, dependency staleness, complexity hotspots, code duplication signals, test coverage gaps, unused exports, and outdated patterns. Use this skill when the user asks to audit their codebase for tech debt, wants a health check of their code quality, says things like \"where should I focus refactoring\", \"find code smells\", \"audit my codebase\", \"what technical debt do we have\", \"prioritize cleanup work\", or asks about code quality metrics. Also trigger when the user mentions \"code rot\", \"legacy cleanup\", \"refactoring priorities\", or wants to understand the overall health of a project before a major effort. Trigger even if the user doesn't say \"technical debt\" explicitly — any request to systematically assess code quality and identify improvement priorities belongs here.","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fdevelopment\u002Ftechnical-debt-auditor\u002FSKILL.md",[215],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[217,218,219,220,221,222,223,224,225,226,227,198,228,229,230,231,232,233,234,235],"technical debt","debt","technical","technical debt auditor","auditor","coverage","complexity","markers","project","cleanup","unused","code","todo","work","plan","focus","signals","audit","dependency",{"skillNo":237,"name":238,"nameZh":6,"description":239,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":240,"categories":241,"keywords":251,"publishedAt":49},"01M70KPYH9QNHCNME5GJQGE22K","readonly-sql-guard","Make an agent's database access provably read-only. Use when an agent runs caller-supplied SQL against a production\u002FERP\u002Fanalytics database (a \"query\" MCP tool, text-to-SQL, an admin console), or when reviewing DB-access code for write\u002FDDL\u002Fside-effect\u002Ftransaction-escape risks. Checks what a statement CALLS, not just its shape.","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fdatabase\u002Freadonly-sql-guard\u002FSKILL.md",[242,245,246],{"categoryNo":243,"categoryName":244,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":247,"categoryName":248,"taxonomyType":249,"taxonomyTypeDesc":250},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[252,253,254,255,256,257,258,30,259,260,261,262,263,264,265,197,266,267,198,268],"readonly","guard","sql guard","sql","readonly sql guard","erp","effect","read-only","db","just","tool","calls","only","MCP","access","database","query",{"skillNo":270,"name":271,"nameZh":6,"description":272,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":273,"licenseFileUrl":274,"sourceHtmlUrl":275,"categories":276,"keywords":280,"publishedAt":49},"01M6AFY6F7CC2BBZP1QD6VTNVC","web-artifacts-builder","Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn\u002Fui). Use for complex artifacts requiring state management, routing, or shadcn\u002Fui components - not for simple single-file HTML\u002FJSX artifacts.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Futilities\u002Fweb-artifacts-builder\u002FSKILL.md",[277],{"categoryNo":278,"categoryName":279,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000009","前端开发",[281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,30,303],"artifacts builder","web","web artifacts builder","artifacts","builder","state","routing","state management","component","css","frontend","ui","Tools","file","components","shadcn","react","Tailwind","suite","jsx","html","single-file","ui components",{"skillNo":305,"name":306,"nameZh":6,"description":307,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":273,"licenseFileUrl":308,"sourceHtmlUrl":309,"categories":310,"keywords":315,"publishedAt":49},"01M6AFWF33SFV9Z3SF4PEEE4BF","webapp-testing","Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fsecurity\u002Fwebapp-testing\u002FSKILL.md",[311,314],{"categoryNo":312,"categoryName":313,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":278,"categoryName":279,"taxonomyType":21,"taxonomyTypeDesc":22},[316,317,318,319,320,282,321,291,292,322,323,324,325,30,326,200,327],"webapp testing","testing","webapp","browser","playwright","applications","debugging","toolkit","logs","screenshots","local","behavior",{"skillNo":329,"name":330,"nameZh":6,"description":331,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":273,"licenseFileUrl":332,"sourceHtmlUrl":333,"categories":334,"keywords":342,"publishedAt":49},"01M6AF67Q67JJJ5V51JR2P52KA","discernment-nudge","After you give a substantive answer or draft that the user may act on — advice or recommendations, drafted artifacts such as goals, plans, pitches, proposals, or emails, estimates or projections, analysis or interpretation of data, factual claims they may rely on, or a multi-step argument — invoke this skill BEFORE finalizing your reply and then, if it applies, append 2-3 short follow-up questions, each tied to something specific in what you just produced, that help the user check key facts, probe the reasoning or assumptions, and notice missing context. Do this at most once per conversation. Skip it when the user asked a trivial how-to or simple lookup, wants a purely educational explanation, asked you only to format, convert, or assemble a file from content they provided, is writing code they will run, is doing creative writing or casual chat, or already asked you to double-check, cite, or review — the skill file explains these boundaries and the exact output format.","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fdiscernment-nudge\u002FLICENSE.txt","https:\u002F\u002Fgithub.com\u002Fdavila7\u002Fclaude-code-templates\u002Fblob\u002Fmain\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fproductivity\u002Fdiscernment-nudge\u002FSKILL.md",[335,338,339],{"categoryNo":336,"categoryName":337,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":340,"categoryName":341,"taxonomyType":249,"taxonomyTypeDesc":250},"01JBSEEDCAT000000000000036","内容创作者",[343,344,345,346,347,48,348,349,284,350,294,228,351,352,353,354,355,264,356,198,357,358,359,360,361,206,362],"discernment nudge","nudge","interpretation","help","run","data","writing","lookup","content","review","conversation","short","up","invoke","you","analysis","claims","argument","assemble","recommendations"]