[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MB626F0YZMW9GYXAWBY1WA":3,"skill-same:01M6MB626F0YZMW9GYXAWBY1WA":84},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":50,"commitSha":51,"skillDirectory":52,"forksCount":53,"repositoryHtmlUrl":54,"repositoryHomepageUrl":55,"ownerHtmlUrl":56,"ownerBlogUrl":57,"ownerDescription":58,"updatedAt":59,"install":60,"resources":73},"01M6MB626F0YZMW9GYXAWBY1WA","slime-rl-training",null,"Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fslime-rl-training\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"training","slime","slime rl training","rl","GRPO","Post-Training","Ray","learning","use","router","integration","models","reinforcement learning","reinforcement","workflows","post","megatron","implementing","for","custom","2026-11-10 10:12:30",false,"---\nname: slime-rl-training\ndescription: Provides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: ml-training\n  maintainer: Kalaris Labs\n  tags: Reinforcement Learning, Megatron-LM, SGLang, GRPO, Post-Training, GLM\n  dependencies: sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0\n---\n\n# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `verl-rl-training`: Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL).\n- `miles-rl-training`: Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime.\n- `nanogpt`: Provides nanoGPT, Karpathy's minimal PyTorch GPT implementation (model.py and train.py), with workflows for training character-level Shakes…\n","# slime: LLM Post-Training Framework for RL Scaling\n\nslime is an LLM post-training framework from Tsinghua's THUDM team, powering GLM-4.5, GLM-4.6, and GLM-4.7. It connects Megatron-LM for training with SGLang for high-throughput rollout generation.\n\n## When to Use slime\n\n**Choose slime when you need:**\n- Megatron-LM native training with SGLang inference\n- Custom data generation workflows with flexible data buffers\n- Training GLM, Qwen3, DeepSeek V3, or Llama 3 models\n- Research-grade framework with production backing (Z.ai)\n\n**Consider alternatives when:**\n- You need enterprise-grade stability features → use **miles**\n- You want flexible backend swapping → use **verl**\n- You need PyTorch-native abstractions → use **torchforge**\n\n## Key Features\n\n- **Training**: Megatron-LM with full parallelism support (TP, PP, DP, SP)\n- **Rollout**: SGLang-based high-throughput generation with router\n- **Data Buffer**: Flexible prompt management and sample storage\n- **Models**: GLM-4.x, Qwen3, DeepSeek V3\u002FR1, Llama 3\n\n## Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Data Buffer                          │\n│ - Prompt initialization and management                  │\n│ - Custom data generation and filtering                  │\n│ - Rollout sample storage                                │\n└─────────────┬───────────────────────────┬───────────────┘\n              │                           │\n┌─────────────▼───────────┐ ┌─────────────▼───────────────┐\n│ Training (Megatron-LM)  │ │ Rollout (SGLang + Router)   │\n│ - Actor model training  │ │ - Response generation       │\n│ - Critic (optional)     │ │ - Reward\u002Fverifier output    │\n│ - Weight sync to rollout│ │ - Multi-turn support        │\n└─────────────────────────┘ └─────────────────────────────┘\n```\n\n## Installation\n\n```bash\n# Recommended: Docker\ndocker pull slimerl\u002Fslime:latest\ndocker run --rm --gpus all --ipc=host --shm-size=16g \\\n  -it slimerl\u002Fslime:latest \u002Fbin\u002Fbash\n\n# Inside container\ncd \u002Froot\u002Fslime && pip install -e . --no-deps\n```\n\n### From Source\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime.git\ncd slime\npip install -r requirements.txt\npip install -e .\n```\n\n## Quick Start: GRPO Training\n\n```bash\n# Source model configuration\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n\n# Launch training\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 4 \\\n    --rollout-num-gpus 4 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss --kl-loss-coef 0.001 \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl \\\n    ${MODEL_ARGS[@]} ${CKPT_ARGS[@]}\n```\n\n---\n\n## Workflow 1: Standard GRPO Training\n\nUse this workflow for training reasoning models with group-relative advantages.\n\n### Prerequisites Checklist\n- [ ] Docker environment or Megatron-LM + SGLang installed\n- [ ] Model checkpoint (HuggingFace or Megatron format)\n- [ ] Training data in JSONL format\n\n### Step 1: Prepare Data\n\n```python\n# data.jsonl format\n{\"prompt\": \"What is 2 + 2?\", \"label\": \"4\"}\n{\"prompt\": \"Solve: 3x = 12\", \"label\": \"x = 4\"}\n```\n\nOr with chat format:\n```python\n{\n    \"prompt\": [\n        {\"role\": \"system\", \"content\": \"You are a math tutor.\"},\n        {\"role\": \"user\", \"content\": \"What is 15 + 27?\"}\n    ],\n    \"label\": \"42\"\n}\n```\n\n### Step 2: Configure Model\n\nChoose a pre-configured model script:\n\n```bash\n# List available models\nls scripts\u002Fmodels\u002F\n# glm4-9B.sh, qwen3-4B.sh, qwen3-30B-A3B.sh, deepseek-v3.sh, llama3-8B.sh, ...\n\n# Source your model\nsource scripts\u002Fmodels\u002Fqwen3-4B.sh\n```\n\n### Step 3: Launch Training\n\n```bash\npython train.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --use-kl-loss \\\n    --kl-loss-coef 0.001 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    --input-key prompt \\\n    --label-key label \\\n    --apply-chat-template \\\n    --rollout-batch-size 32 \\\n    --n-samples-per-prompt 8 \\\n    --global-batch-size 256 \\\n    --num-rollout 3000 \\\n    --save-interval 100 \\\n    --eval-interval 50 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Step 4: Monitor Training\n- [ ] Check TensorBoard: `tensorboard --logdir outputs\u002F`\n- [ ] Verify reward curves are increasing\n- [ ] Monitor GPU utilization across nodes\n\n---\n\n## Workflow 2: Asynchronous Training\n\nUse async mode for higher throughput by overlapping rollout and training.\n\n### When to Use Async\n- Large models with long generation times\n- High GPU idle time in synchronous mode\n- Sufficient memory for buffering\n\n### Launch Async Training\n\n```bash\npython train_async.py \\\n    --actor-num-nodes 1 \\\n    --actor-num-gpus-per-node 8 \\\n    --rollout-num-gpus 8 \\\n    --advantage-estimator grpo \\\n    --async-buffer-size 4 \\\n    --prompt-data \u002Fpath\u002Fto\u002Ftrain.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\n### Async-Specific Parameters\n\n```bash\n--async-buffer-size 4        # Number of rollouts to buffer\n--update-weights-interval 2  # Sync weights every N rollouts\n```\n\n---\n\n## Workflow 3: Multi-Turn Agentic Training\n\nUse this workflow for training agents with tool use or multi-step reasoning.\n\n### Prerequisites\n- [ ] Custom generate function for multi-turn logic\n- [ ] Tool\u002Fenvironment interface\n\n### Step 1: Define Custom Generate Function\n\n```python\n# custom_generate.py\nasync def custom_generate(args, samples, evaluation=False):\n    \"\"\"Multi-turn generation with tool calling.\"\"\"\n    for sample in samples:\n        conversation = sample.prompt\n\n        for turn in range(args.max_turns):\n            # Generate response\n            response = await generate_single(conversation)\n\n            # Check for tool call\n            tool_call = extract_tool_call(response)\n            if tool_call:\n                tool_result = execute_tool(tool_call)\n                conversation.append({\"role\": \"assistant\", \"content\": response})\n                conversation.append({\"role\": \"tool\", \"content\": tool_result})\n            else:\n                break\n\n        sample.response = response\n        sample.reward = compute_reward(sample)\n\n    return samples\n```\n\n### Step 2: Launch with Custom Function\n\n```bash\npython train.py \\\n    --custom-generate-function-path custom_generate.py \\\n    --max-turns 5 \\\n    --prompt-data \u002Fpath\u002Fto\u002Fagent_data.jsonl \\\n    ${MODEL_ARGS[@]}\n```\n\nSee `examples\u002Fsearch-r1\u002F` for a complete multi-turn search example.\n\n---\n\n## Configuration Reference\n\n### Three Argument Categories\n\nslime uses three types of arguments:\n\n**1. Megatron Arguments** (passed directly):\n```bash\n--tensor-model-parallel-size 2\n--pipeline-model-parallel-size 1\n--num-layers 32\n--hidden-size 4096\n```\n\n**2. SGLang Arguments** (prefixed with `--sglang-`):\n```bash\n--sglang-mem-fraction-static 0.8\n--sglang-context-length 8192\n--sglang-log-level INFO\n```\n\n**3. slime Arguments**:\n```bash\n# Resource allocation\n--actor-num-nodes 1\n--actor-num-gpus-per-node 8\n--rollout-num-gpus 8\n--colocate  # Share GPUs between training\u002Finference\n\n# Data\n--prompt-data \u002Fpath\u002Fto\u002Fdata.jsonl\n--input-key prompt\n--label-key label\n\n# Training loop\n--num-rollout 3000\n--rollout-batch-size 32\n--n-samples-per-prompt 8\n--global-batch-size 256\n\n# Algorithm\n--advantage-estimator grpo  # or: gspo, ppo, reinforce_plus_plus\n--use-kl-loss\n--kl-loss-coef 0.001\n```\n\n### Key Constraints\n\n```\nrollout_batch_size × n_samples_per_prompt = global_batch_size × num_steps_per_rollout\n```\n\nExample: 32 × 8 = 256 × 1\n\n---\n\n## Data Buffer System\n\nslime's data buffer enables flexible data management:\n\n### Basic Data Source\n\n```python\nclass RolloutDataSource:\n    def get_samples(self, num_samples):\n        \"\"\"Fetch prompts from dataset.\"\"\"\n        return self.dataset.sample(num_samples)\n\n    def add_samples(self, samples):\n        \"\"\"Called after generation (no-op by default).\"\"\"\n        pass\n```\n\n### Buffered Data Source (Off-Policy)\n\n```python\nclass RolloutDataSourceWithBuffer(RolloutDataSource):\n    def __init__(self):\n        self.buffer = []\n\n    def add_samples(self, samples):\n        \"\"\"Store generated samples for reuse.\"\"\"\n        self.buffer.extend(samples)\n\n    def buffer_filter(self, args, buffer, num_samples):\n        \"\"\"Custom selection logic (prioritized, stratified, etc.).\"\"\"\n        return select_best(buffer, num_samples)\n```\n\n---\n\n## Common Issues and Solutions\n\n### Issue: SGLang Engine Crash\n\n**Symptoms**: Inference engine dies mid-training\n\n**Solutions**:\n```bash\n# Enable fault tolerance\n--use-fault-tolerance\n\n# Increase memory allocation\n--sglang-mem-fraction-static 0.85\n\n# Reduce batch size\n--rollout-batch-size 16\n```\n\n### Issue: Weight Sync Timeout\n\n**Symptoms**: Training hangs after rollout\n\n**Solutions**:\n```bash\n# Increase sync interval\n--update-weights-interval 5\n\n# Use colocated mode (no network transfer)\n--colocate\n```\n\n### Issue: OOM During Training\n\n**Symptoms**: CUDA OOM in backward pass\n\n**Solutions**:\n```bash\n# Enable gradient checkpointing\n--recompute-activations\n\n# Reduce micro-batch size\n--micro-batch-size 1\n\n# Enable sequence parallelism\n--sequence-parallel\n```\n\n### Issue: Slow Data Loading\n\n**Symptoms**: GPU idle during data fetch\n\n**Solutions**:\n```bash\n# Increase data workers\n--num-data-workers 4\n\n# Use streaming dataset\n--streaming-data\n```\n\n---\n\n## Supported Models\n\n| Model Family | Configurations |\n|--------------|----------------|\n| GLM | GLM-4.5, GLM-4.6, GLM-4.7, GLM-Z1-9B |\n| Qwen | Qwen3 (4B, 8B, 30B-A3B), Qwen3-MoE, Qwen2.5 |\n| DeepSeek | V3, V3.1, R1 |\n| Llama | Llama 3 (8B, 70B) |\n| Others | Kimi K2, Moonlight-16B |\n\nEach model has pre-configured scripts in `scripts\u002Fmodels\u002F`.\n\n---\n\n## Advanced Topics\n\n### Co-location Mode\n\nShare GPUs between training and inference to reduce memory:\n\n```bash\npython train.py \\\n    --colocate \\\n    --actor-num-gpus-per-node 8 \\\n    --sglang-mem-fraction-static 0.4 \\\n    ${MODEL_ARGS[@]}\n```\n\n### Custom Reward Model\n\n```python\n# custom_rm.py\nclass CustomRewardModel:\n    def __init__(self, model_path):\n        self.model = load_model(model_path)\n\n    def compute_reward(self, prompts, responses):\n        inputs = self.tokenize(prompts, responses)\n        scores = self.model(inputs)\n        return scores.tolist()\n```\n\n```bash\n--custom-rm-path custom_rm.py\n```\n\n### Evaluation Multi-Task\n\n```bash\n--eval-prompt-data aime \u002Fpath\u002Fto\u002Faime.jsonl \\\n--eval-prompt-data gsm8k \u002Fpath\u002Fto\u002Fgsm8k.jsonl \\\n--n-samples-per-eval-prompt 16\n```\n\n---\n\n## Resources\n\n- **Documentation**: https:\u002F\u002Fthudm.github.io\u002Fslime\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002FTHUDM\u002Fslime\n- **Blog**: https:\u002F\u002Flmsys.org\u002Fblog\u002F2025-07-09-slime\u002F\n- **Examples**: See `examples\u002F` directory for 14+ worked examples\n\n## Agent operating procedure\n\n1. **Check the environment.** Check GPU type, memory and driver\u002FCUDA versions (`nvidia-smi`), framework versions, and dataset location and size.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Do a smoke run: tiny model or subset, few steps, and confirm loss decreases and checkpoints save.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Track metrics on held-out data, compare against a baseline, and record seeds, configs and hardware.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| CUDA out-of-memory | Reduce batch size, enable gradient accumulation\u002Fcheckpointing or mixed precision, or shard the model. |\n| Loss is NaN or diverges | Lower the learning rate, check data for invalid values, and enable gradient clipping. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Never claim training results without logs; estimate compute cost before launching large jobs.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `verl-rl-training`: Provides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL).\n- `miles-rl-training`: Provides guidance for enterprise-grade RL training using miles, a production-ready fork of slime.\n- `nanogpt`: Provides nanoGPT, Karpathy's minimal PyTorch GPT implementation (model.py and train.py), with workflows for training character-level Shakes…\n",{"name":5,"license":15,"metadata":44,"description":7},{"tags":45,"version":46,"category":47,"maintainer":48,"dependencies":49},"Reinforcement Learning, Megatron-LM, SGLang, GRPO, Post-Training, GLM","1.0.0","ml-training","Kalaris Labs","sglang-router>=0.2.3, ray, torch>=2.0.0, transformers>=4.40.0","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fslime-rl-training",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill slime-rl-training",{"tool":66,"command":67},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill slime-rl-training",{"tool":69,"command":70},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill slime-rl-training","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fslime-rl-training\u002FSKILL.md",true,[74,80],{"path":75,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fapi-reference.md","REFERENCE","参考文档",11907,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fslime-rl-training\u002Freferences\u002Fapi-reference.md",{"path":81,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Ftroubleshooting.md",7221,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fslime-rl-training\u002Freferences\u002Ftroubleshooting.md",[85,94,102,110,118,126],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":89,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":93},"01M7DV332YQMGD8WTRJ9DZQXTB","worktable-review-with-annotations","Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":95,"name":96,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":98,"sourceHtmlUrl":99,"categories":100,"keywords":101,"publishedAt":93},"01M7DV2YGHCZVBEB1AHYB6CGHE","worktable-find-and-synthesize-context","Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FSKILL.md",[],[],{"skillNo":103,"name":104,"nameZh":6,"description":105,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":106,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":93},"01M7DV2TQN1H11ME2SAF9MXN7V","worktable-create-or-update-docs","Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":114,"sourceHtmlUrl":115,"categories":116,"keywords":117,"publishedAt":93},"01M7DV2NE452NEA8TXATKSNAJH","worktable-create-or-manage-records","Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FSKILL.md",[],[],{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":122,"sourceHtmlUrl":123,"categories":124,"keywords":125,"publishedAt":93},"01M7DV2HENRBR2KSCVM064WH0Y","worktable-create-or-manage-interactive-html","Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FSKILL.md",[],[],{"skillNo":127,"name":128,"nameZh":6,"description":129,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":130,"sourceHtmlUrl":131,"categories":132,"keywords":133,"publishedAt":93},"01M7DV2CADR5R7RQFFYPNAGR9Y","worktable-collaborate-threads","Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FSKILL.md",[],[]]