[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XGE429CC3S8D66FJ110A5S":3,"skill-same:01M6XGE429CC3S8D66FJ110A5S":104},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":52,"commitSha":53,"skillDirectory":54,"forksCount":14,"repositoryHtmlUrl":55,"repositoryHomepageUrl":6,"ownerHtmlUrl":56,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":57,"install":58,"resources":71},"01M6XGE429CC3S8D66FJ110A5S","pufferlib",null,"High-performance reinforcement learning framework optimized for speed and scale. Use when you need fast parallel training, vectorized environments, multi-agent systems, or integration with game environments (Atari, Procgen, NetHack). Achieves 2-10x speedups over standard implementations. For quick prototyping or standard algorithm implementations with extensive documentation, use stable-baselines3 instead.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[5,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"parallel","when","for","environments","you","quick","prototyping","high-performance","synthetic","speed","performance","multi-agent","10x","game","learning","vectorized","framework","training","documentation","2026-11-10 10:12:30",false,"---\nname: pufferlib\ndescription: High-performance reinforcement learning framework optimized for speed and scale. Use when you need fast parallel training, vectorized environments, multi-agent systems, or integration with game environments (Atari, Procgen, NetHack). Achieves 2-10x speedups over standard implementations. For quick prototyping or standard algorithm implementations with extensive documentation, use stable-baselines3 instead.\ncategory: ml-training\nlicense: MIT license\nmetadata:\n    skill-author: Synthetic Sciences\n---\n\n# PufferLib - High-Performance Reinforcement Learning\n\n## Overview\n\nPufferLib is a high-performance reinforcement learning library designed for fast parallel environment simulation and training. It achieves training at millions of steps per second through optimized vectorization, native multi-agent support, and efficient PPO implementation (PuffeRL). The library provides the Ocean suite of 20+ environments and seamless integration with Gymnasium, PettingZoo, and specialized RL frameworks.\n\n## When to Use This Skill\n\nUse this skill when:\n- **Training RL agents** with PPO on any environment (single or multi-agent)\n- **Creating custom environments** using the PufferEnv API\n- **Optimizing performance** for parallel environment simulation (vectorization)\n- **Integrating existing environments** from Gymnasium, PettingZoo, Atari, Procgen, etc.\n- **Developing policies** with CNN, LSTM, or custom architectures\n- **Scaling RL** to millions of steps per second for faster experimentation\n- **Multi-agent RL** with native multi-agent environment support\n\n## Core Capabilities\n\n### 1. High-Performance Training (PuffeRL)\n\nPuffeRL is PufferLib's optimized PPO+LSTM training algorithm achieving 1M-4M steps\u002Fsecond.\n\n**Quick start training:**\n```bash\n# CLI training\npuffer train procgen-coinrun --train.device cuda --train.learning-rate 3e-4\n\n# Distributed training\ntorchrun --nproc_per_node=4 train.py\n```\n\n**Python training loop:**\n```python\nimport pufferlib\nfrom pufferlib import PuffeRL\n\n# Create vectorized environment\nenv = pufferlib.make('procgen-coinrun', num_envs=256)\n\n# Create trainer\ntrainer = PuffeRL(\n    env=env,\n    policy=my_policy,\n    device='cuda',\n    learning_rate=3e-4,\n    batch_size=32768\n)\n\n# Training loop\nfor iteration in range(num_iterations):\n    trainer.evaluate()  # Collect rollouts\n    trainer.train()     # Train on batch\n    trainer.mean_and_log()  # Log results\n```\n\n**For comprehensive training guidance**, read `references\u002Ftraining.md` for:\n- Complete training workflow and CLI options\n- Hyperparameter tuning with Protein\n- Distributed multi-GPU\u002Fmulti-node training\n- Logger integration (Weights & Biases, Neptune)\n- Checkpointing and resume training\n- Performance optimization tips\n- Curriculum learning patterns\n\n### 2. Environment Development (PufferEnv)\n\nCreate custom high-performance environments with the PufferEnv API.\n\n**Basic environment structure:**\n```python\nimport numpy as np\nfrom pufferlib import PufferEnv\n\nclass MyEnvironment(PufferEnv):\n    def __init__(self, buf=None):\n        super().__init__(buf)\n\n        # Define spaces\n        self.observation_space = self.make_space((4,))\n        self.action_space = self.make_discrete(4)\n\n        self.reset()\n\n    def reset(self):\n        # Reset state and return initial observation\n        return np.zeros(4, dtype=np.float32)\n\n    def step(self, action):\n        # Execute action, compute reward, check done\n        obs = self._get_observation()\n        reward = self._compute_reward()\n        done = self._is_done()\n        info = {}\n\n        return obs, reward, done, info\n```\n\n**Use the template script:** `scripts\u002Fenv_template.py` provides complete single-agent and multi-agent environment templates with examples of:\n- Different observation space types (vector, image, dict)\n- Action space variations (discrete, continuous, multi-discrete)\n- Multi-agent environment structure\n- Testing utilities\n\n**For complete environment development**, read `references\u002Fenvironments.md` for:\n- PufferEnv API details and in-place operation patterns\n- Observation and action space definitions\n- Multi-agent environment creation\n- Ocean suite (20+ pre-built environments)\n- Performance optimization (Python to C workflow)\n- Environment wrappers and best practices\n- Debugging and validation techniques\n\n### 3. Vectorization and Performance\n\nAchieve maximum throughput with optimized parallel simulation.\n\n**Vectorization setup:**\n```python\nimport pufferlib\n\n# Automatic vectorization\nenv = pufferlib.make('environment_name', num_envs=256, num_workers=8)\n\n# Performance benchmarks:\n# - Pure Python envs: 100k-500k SPS\n# - C-based envs: 100M+ SPS\n# - With training: 400k-4M total SPS\n```\n\n**Key optimizations:**\n- Shared memory buffers for zero-copy observation passing\n- Busy-wait flags instead of pipes\u002Fqueues\n- Surplus environments for async returns\n- Multiple environments per worker\n\n**For vectorization optimization**, read `references\u002Fvectorization.md` for:\n- Architecture and performance characteristics\n- Worker and batch size configuration\n- Serial vs multiprocessing vs async modes\n- Shared memory and zero-copy patterns\n- Hierarchical vectorization for large scale\n- Multi-agent vectorization strategies\n- Performance profiling and troubleshooting\n\n### 4. Policy Development\n\nBuild policies as standard PyTorch modules with optional utilities.\n\n**Basic policy structure:**\n```python\nimport torch.nn as nn\nfrom pufferlib.pytorch import layer_init\n\nclass Policy(nn.Module):\n    def __init__(self, observation_space, action_space):\n        super().__init__()\n\n        # Encoder\n        self.encoder = nn.Sequential(\n            layer_init(nn.Linear(obs_dim, 256)),\n            nn.ReLU(),\n            layer_init(nn.Linear(256, 256)),\n            nn.ReLU()\n        )\n\n        # Actor and critic heads\n        self.actor = layer_init(nn.Linear(256, num_actions), std=0.01)\n        self.critic = layer_init(nn.Linear(256, 1), std=1.0)\n\n    def forward(self, observations):\n        features = self.encoder(observations)\n        return self.actor(features), self.critic(features)\n```\n\n**For complete policy development**, read `references\u002Fpolicies.md` for:\n- CNN policies for image observations\n- Recurrent policies with optimized LSTM (3x faster inference)\n- Multi-input policies for complex observations\n- Continuous action policies\n- Multi-agent policies (shared vs independent parameters)\n- Advanced architectures (attention, residual)\n- Observation normalization and gradient clipping\n- Policy debugging and testing\n\n### 5. Environment Integration\n\nSeamlessly integrate environments from popular RL frameworks.\n\n**Gymnasium integration:**\n```python\nimport gymnasium as gym\nimport pufferlib\n\n# Wrap Gymnasium environment\ngym_env = gym.make('CartPole-v1')\nenv = pufferlib.emulate(gym_env, num_envs=256)\n\n# Or use make directly\nenv = pufferlib.make('gym-CartPole-v1', num_envs=256)\n```\n\n**PettingZoo multi-agent:**\n```python\n# Multi-agent environment\nenv = pufferlib.make('pettingzoo-knights-archers-zombies', num_envs=128)\n```\n\n**Supported frameworks:**\n- Gymnasium \u002F OpenAI Gym\n- PettingZoo (parallel and AEC)\n- Atari (ALE)\n- Procgen\n- NetHack \u002F MiniHack\n- Minigrid\n- Neural MMO\n- Crafter\n- GPUDrive\n- MicroRTS\n- Griddly\n- And more...\n\n**For integration details**, read `references\u002Fintegration.md` for:\n- Complete integration examples for each framework\n- Custom wrappers (observation, reward, frame stacking, action repeat)\n- Space flattening and unflattening\n- Environment registration\n- Compatibility patterns\n- Performance considerations\n- Integration debugging\n\n## Quick Start Workflow\n\n### For Training Existing Environments\n\n1. Choose environment from Ocean suite or compatible framework\n2. Use `scripts\u002Ftrain_template.py` as starting point\n3. Configure hyperparameters for your task\n4. Run training with CLI or Python script\n5. Monitor with Weights & Biases or Neptune\n6. Refer to `references\u002Ftraining.md` for optimization\n\n### For Creating Custom Environments\n\n1. Start with `scripts\u002Fenv_template.py`\n2. Define observation and action spaces\n3. Implement `reset()` and `step()` methods\n4. Test environment locally\n5. Vectorize with `pufferlib.emulate()` or `make()`\n6. Refer to `references\u002Fenvironments.md` for advanced patterns\n7. Optimize with `references\u002Fvectorization.md` if needed\n\n### For Policy Development\n\n1. Choose architecture based on observations:\n   - Vector observations → MLP policy\n   - Image observations → CNN policy\n   - Sequential tasks → LSTM policy\n   - Complex observations → Multi-input policy\n2. Use `layer_init` for proper weight initialization\n3. Follow patterns in `references\u002Fpolicies.md`\n4. Test with environment before full training\n\n### For Performance Optimization\n\n1. Profile current throughput (steps per second)\n2. Check vectorization configuration (num_envs, num_workers)\n3. Optimize environment code (in-place ops, numpy vectorization)\n4. Consider C implementation for critical paths\n5. Use `references\u002Fvectorization.md` for systematic optimization\n\n## Resources\n\n### scripts\u002F\n\n**train_template.py** - Complete training script template with:\n- Environment creation and configuration\n- Policy initialization\n- Logger integration (WandB, Neptune)\n- Training loop with checkpointing\n- Command-line argument parsing\n- Multi-GPU distributed training setup\n\n**env_template.py** - Environment implementation templates:\n- Single-agent PufferEnv example (grid world)\n- Multi-agent PufferEnv example (cooperative navigation)\n- Multiple observation\u002Faction space patterns\n- Testing utilities\n\n### references\u002F\n\n**training.md** - Comprehensive training guide:\n- Training workflow and CLI options\n- Hyperparameter configuration\n- Distributed training (multi-GPU, multi-node)\n- Monitoring and logging\n- Checkpointing\n- Protein hyperparameter tuning\n- Performance optimization\n- Common training patterns\n- Troubleshooting\n\n**environments.md** - Environment development guide:\n- PufferEnv API and characteristics\n- Observation and action spaces\n- Multi-agent environments\n- Ocean suite environments\n- Custom environment development workflow\n- Python to C optimization path\n- Third-party environment integration\n- Wrappers and best practices\n- Debugging\n\n**vectorization.md** - Vectorization optimization:\n- Architecture and key optimizations\n- Vectorization modes (serial, multiprocessing, async)\n- Worker and batch configuration\n- Shared memory and zero-copy patterns\n- Advanced vectorization (hierarchical, custom)\n- Multi-agent vectorization\n- Performance monitoring and profiling\n- Troubleshooting and best practices\n\n**policies.md** - Policy architecture guide:\n- Basic policy structure\n- CNN policies for images\n- LSTM policies with optimization\n- Multi-input policies\n- Continuous action policies\n- Multi-agent policies\n- Advanced architectures (attention, residual)\n- Observation processing and unflattening\n- Initialization and normalization\n- Debugging and testing\n\n**integration.md** - Framework integration guide:\n- Gymnasium integration\n- PettingZoo integration (parallel and AEC)\n- Third-party environments (Procgen, NetHack, Minigrid, etc.)\n- Custom wrappers (observation, reward, frame stacking, etc.)\n- Space conversion and unflattening\n- Environment registration\n- Compatibility patterns\n- Performance considerations\n- Debugging integration\n\n## Tips for Success\n\n1. **Start simple**: Begin with Ocean environments or Gymnasium integration before creating custom environments\n\n2. **Profile early**: Measure steps per second from the start to identify bottlenecks\n\n3. **Use templates**: `scripts\u002Ftrain_template.py` and `scripts\u002Fenv_template.py` provide solid starting points\n\n4. **Read references as needed**: Each reference file is self-contained and focused on a specific capability\n\n5. **Optimize progressively**: Start with Python, profile, then optimize critical paths with C if needed\n\n6. **Leverage vectorization**: PufferLib's vectorization is key to achieving high throughput\n\n7. **Monitor training**: Use WandB or Neptune to track experiments and identify issues early\n\n8. **Test environments**: Validate environment logic before scaling up training\n\n9. **Check existing environments**: Ocean suite provides 20+ pre-built environments\n\n10. **Use proper initialization**: Always use `layer_init` from `pufferlib.pytorch` for policies\n\n## Common Use Cases\n\n### Training on Standard Benchmarks\n```python\n# Atari\nenv = pufferlib.make('atari-pong', num_envs=256)\n\n# Procgen\nenv = pufferlib.make('procgen-coinrun', num_envs=256)\n\n# Minigrid\nenv = pufferlib.make('minigrid-empty-8x8', num_envs=256)\n```\n\n### Multi-Agent Learning\n```python\n# PettingZoo\nenv = pufferlib.make('pettingzoo-pistonball', num_envs=128)\n\n# Shared policy for all agents\npolicy = create_policy(env.observation_space, env.action_space)\ntrainer = PuffeRL(env=env, policy=policy)\n```\n\n### Custom Task Development\n```python\n# Create custom environment\nclass MyTask(PufferEnv):\n    # ... implement environment ...\n\n# Vectorize and train\nenv = pufferlib.emulate(MyTask, num_envs=256)\ntrainer = PuffeRL(env=env, policy=my_policy)\n```\n\n### High-Performance Optimization\n```python\n# Maximize throughput\nenv = pufferlib.make(\n    'my-env',\n    num_envs=1024,      # Large batch\n    num_workers=16,     # Many workers\n    envs_per_worker=64  # Optimize per worker\n)\n```\n\n## Installation\n\n```bash\nuv pip install pufferlib\n```\n\n## Documentation\n\n- Official docs: https:\u002F\u002Fpuffer.ai\u002Fdocs.html\n- GitHub: https:\u002F\u002Fgithub.com\u002FPufferAI\u002FPufferLib\n- Discord: Community support available\n\n","# PufferLib - High-Performance Reinforcement Learning\n\n## Overview\n\nPufferLib is a high-performance reinforcement learning library designed for fast parallel environment simulation and training. It achieves training at millions of steps per second through optimized vectorization, native multi-agent support, and efficient PPO implementation (PuffeRL). The library provides the Ocean suite of 20+ environments and seamless integration with Gymnasium, PettingZoo, and specialized RL frameworks.\n\n## When to Use This Skill\n\nUse this skill when:\n- **Training RL agents** with PPO on any environment (single or multi-agent)\n- **Creating custom environments** using the PufferEnv API\n- **Optimizing performance** for parallel environment simulation (vectorization)\n- **Integrating existing environments** from Gymnasium, PettingZoo, Atari, Procgen, etc.\n- **Developing policies** with CNN, LSTM, or custom architectures\n- **Scaling RL** to millions of steps per second for faster experimentation\n- **Multi-agent RL** with native multi-agent environment support\n\n## Core Capabilities\n\n### 1. High-Performance Training (PuffeRL)\n\nPuffeRL is PufferLib's optimized PPO+LSTM training algorithm achieving 1M-4M steps\u002Fsecond.\n\n**Quick start training:**\n```bash\n# CLI training\npuffer train procgen-coinrun --train.device cuda --train.learning-rate 3e-4\n\n# Distributed training\ntorchrun --nproc_per_node=4 train.py\n```\n\n**Python training loop:**\n```python\nimport pufferlib\nfrom pufferlib import PuffeRL\n\n# Create vectorized environment\nenv = pufferlib.make('procgen-coinrun', num_envs=256)\n\n# Create trainer\ntrainer = PuffeRL(\n    env=env,\n    policy=my_policy,\n    device='cuda',\n    learning_rate=3e-4,\n    batch_size=32768\n)\n\n# Training loop\nfor iteration in range(num_iterations):\n    trainer.evaluate()  # Collect rollouts\n    trainer.train()     # Train on batch\n    trainer.mean_and_log()  # Log results\n```\n\n**For comprehensive training guidance**, read `references\u002Ftraining.md` for:\n- Complete training workflow and CLI options\n- Hyperparameter tuning with Protein\n- Distributed multi-GPU\u002Fmulti-node training\n- Logger integration (Weights & Biases, Neptune)\n- Checkpointing and resume training\n- Performance optimization tips\n- Curriculum learning patterns\n\n### 2. Environment Development (PufferEnv)\n\nCreate custom high-performance environments with the PufferEnv API.\n\n**Basic environment structure:**\n```python\nimport numpy as np\nfrom pufferlib import PufferEnv\n\nclass MyEnvironment(PufferEnv):\n    def __init__(self, buf=None):\n        super().__init__(buf)\n\n        # Define spaces\n        self.observation_space = self.make_space((4,))\n        self.action_space = self.make_discrete(4)\n\n        self.reset()\n\n    def reset(self):\n        # Reset state and return initial observation\n        return np.zeros(4, dtype=np.float32)\n\n    def step(self, action):\n        # Execute action, compute reward, check done\n        obs = self._get_observation()\n        reward = self._compute_reward()\n        done = self._is_done()\n        info = {}\n\n        return obs, reward, done, info\n```\n\n**Use the template script:** `scripts\u002Fenv_template.py` provides complete single-agent and multi-agent environment templates with examples of:\n- Different observation space types (vector, image, dict)\n- Action space variations (discrete, continuous, multi-discrete)\n- Multi-agent environment structure\n- Testing utilities\n\n**For complete environment development**, read `references\u002Fenvironments.md` for:\n- PufferEnv API details and in-place operation patterns\n- Observation and action space definitions\n- Multi-agent environment creation\n- Ocean suite (20+ pre-built environments)\n- Performance optimization (Python to C workflow)\n- Environment wrappers and best practices\n- Debugging and validation techniques\n\n### 3. Vectorization and Performance\n\nAchieve maximum throughput with optimized parallel simulation.\n\n**Vectorization setup:**\n```python\nimport pufferlib\n\n# Automatic vectorization\nenv = pufferlib.make('environment_name', num_envs=256, num_workers=8)\n\n# Performance benchmarks:\n# - Pure Python envs: 100k-500k SPS\n# - C-based envs: 100M+ SPS\n# - With training: 400k-4M total SPS\n```\n\n**Key optimizations:**\n- Shared memory buffers for zero-copy observation passing\n- Busy-wait flags instead of pipes\u002Fqueues\n- Surplus environments for async returns\n- Multiple environments per worker\n\n**For vectorization optimization**, read `references\u002Fvectorization.md` for:\n- Architecture and performance characteristics\n- Worker and batch size configuration\n- Serial vs multiprocessing vs async modes\n- Shared memory and zero-copy patterns\n- Hierarchical vectorization for large scale\n- Multi-agent vectorization strategies\n- Performance profiling and troubleshooting\n\n### 4. Policy Development\n\nBuild policies as standard PyTorch modules with optional utilities.\n\n**Basic policy structure:**\n```python\nimport torch.nn as nn\nfrom pufferlib.pytorch import layer_init\n\nclass Policy(nn.Module):\n    def __init__(self, observation_space, action_space):\n        super().__init__()\n\n        # Encoder\n        self.encoder = nn.Sequential(\n            layer_init(nn.Linear(obs_dim, 256)),\n            nn.ReLU(),\n            layer_init(nn.Linear(256, 256)),\n            nn.ReLU()\n        )\n\n        # Actor and critic heads\n        self.actor = layer_init(nn.Linear(256, num_actions), std=0.01)\n        self.critic = layer_init(nn.Linear(256, 1), std=1.0)\n\n    def forward(self, observations):\n        features = self.encoder(observations)\n        return self.actor(features), self.critic(features)\n```\n\n**For complete policy development**, read `references\u002Fpolicies.md` for:\n- CNN policies for image observations\n- Recurrent policies with optimized LSTM (3x faster inference)\n- Multi-input policies for complex observations\n- Continuous action policies\n- Multi-agent policies (shared vs independent parameters)\n- Advanced architectures (attention, residual)\n- Observation normalization and gradient clipping\n- Policy debugging and testing\n\n### 5. Environment Integration\n\nSeamlessly integrate environments from popular RL frameworks.\n\n**Gymnasium integration:**\n```python\nimport gymnasium as gym\nimport pufferlib\n\n# Wrap Gymnasium environment\ngym_env = gym.make('CartPole-v1')\nenv = pufferlib.emulate(gym_env, num_envs=256)\n\n# Or use make directly\nenv = pufferlib.make('gym-CartPole-v1', num_envs=256)\n```\n\n**PettingZoo multi-agent:**\n```python\n# Multi-agent environment\nenv = pufferlib.make('pettingzoo-knights-archers-zombies', num_envs=128)\n```\n\n**Supported frameworks:**\n- Gymnasium \u002F OpenAI Gym\n- PettingZoo (parallel and AEC)\n- Atari (ALE)\n- Procgen\n- NetHack \u002F MiniHack\n- Minigrid\n- Neural MMO\n- Crafter\n- GPUDrive\n- MicroRTS\n- Griddly\n- And more...\n\n**For integration details**, read `references\u002Fintegration.md` for:\n- Complete integration examples for each framework\n- Custom wrappers (observation, reward, frame stacking, action repeat)\n- Space flattening and unflattening\n- Environment registration\n- Compatibility patterns\n- Performance considerations\n- Integration debugging\n\n## Quick Start Workflow\n\n### For Training Existing Environments\n\n1. Choose environment from Ocean suite or compatible framework\n2. Use `scripts\u002Ftrain_template.py` as starting point\n3. Configure hyperparameters for your task\n4. Run training with CLI or Python script\n5. Monitor with Weights & Biases or Neptune\n6. Refer to `references\u002Ftraining.md` for optimization\n\n### For Creating Custom Environments\n\n1. Start with `scripts\u002Fenv_template.py`\n2. Define observation and action spaces\n3. Implement `reset()` and `step()` methods\n4. Test environment locally\n5. Vectorize with `pufferlib.emulate()` or `make()`\n6. Refer to `references\u002Fenvironments.md` for advanced patterns\n7. Optimize with `references\u002Fvectorization.md` if needed\n\n### For Policy Development\n\n1. Choose architecture based on observations:\n   - Vector observations → MLP policy\n   - Image observations → CNN policy\n   - Sequential tasks → LSTM policy\n   - Complex observations → Multi-input policy\n2. Use `layer_init` for proper weight initialization\n3. Follow patterns in `references\u002Fpolicies.md`\n4. Test with environment before full training\n\n### For Performance Optimization\n\n1. Profile current throughput (steps per second)\n2. Check vectorization configuration (num_envs, num_workers)\n3. Optimize environment code (in-place ops, numpy vectorization)\n4. Consider C implementation for critical paths\n5. Use `references\u002Fvectorization.md` for systematic optimization\n\n## Resources\n\n### scripts\u002F\n\n**train_template.py** - Complete training script template with:\n- Environment creation and configuration\n- Policy initialization\n- Logger integration (WandB, Neptune)\n- Training loop with checkpointing\n- Command-line argument parsing\n- Multi-GPU distributed training setup\n\n**env_template.py** - Environment implementation templates:\n- Single-agent PufferEnv example (grid world)\n- Multi-agent PufferEnv example (cooperative navigation)\n- Multiple observation\u002Faction space patterns\n- Testing utilities\n\n### references\u002F\n\n**training.md** - Comprehensive training guide:\n- Training workflow and CLI options\n- Hyperparameter configuration\n- Distributed training (multi-GPU, multi-node)\n- Monitoring and logging\n- Checkpointing\n- Protein hyperparameter tuning\n- Performance optimization\n- Common training patterns\n- Troubleshooting\n\n**environments.md** - Environment development guide:\n- PufferEnv API and characteristics\n- Observation and action spaces\n- Multi-agent environments\n- Ocean suite environments\n- Custom environment development workflow\n- Python to C optimization path\n- Third-party environment integration\n- Wrappers and best practices\n- Debugging\n\n**vectorization.md** - Vectorization optimization:\n- Architecture and key optimizations\n- Vectorization modes (serial, multiprocessing, async)\n- Worker and batch configuration\n- Shared memory and zero-copy patterns\n- Advanced vectorization (hierarchical, custom)\n- Multi-agent vectorization\n- Performance monitoring and profiling\n- Troubleshooting and best practices\n\n**policies.md** - Policy architecture guide:\n- Basic policy structure\n- CNN policies for images\n- LSTM policies with optimization\n- Multi-input policies\n- Continuous action policies\n- Multi-agent policies\n- Advanced architectures (attention, residual)\n- Observation processing and unflattening\n- Initialization and normalization\n- Debugging and testing\n\n**integration.md** - Framework integration guide:\n- Gymnasium integration\n- PettingZoo integration (parallel and AEC)\n- Third-party environments (Procgen, NetHack, Minigrid, etc.)\n- Custom wrappers (observation, reward, frame stacking, etc.)\n- Space conversion and unflattening\n- Environment registration\n- Compatibility patterns\n- Performance considerations\n- Debugging integration\n\n## Tips for Success\n\n1. **Start simple**: Begin with Ocean environments or Gymnasium integration before creating custom environments\n\n2. **Profile early**: Measure steps per second from the start to identify bottlenecks\n\n3. **Use templates**: `scripts\u002Ftrain_template.py` and `scripts\u002Fenv_template.py` provide solid starting points\n\n4. **Read references as needed**: Each reference file is self-contained and focused on a specific capability\n\n5. **Optimize progressively**: Start with Python, profile, then optimize critical paths with C if needed\n\n6. **Leverage vectorization**: PufferLib's vectorization is key to achieving high throughput\n\n7. **Monitor training**: Use WandB or Neptune to track experiments and identify issues early\n\n8. **Test environments**: Validate environment logic before scaling up training\n\n9. **Check existing environments**: Ocean suite provides 20+ pre-built environments\n\n10. **Use proper initialization**: Always use `layer_init` from `pufferlib.pytorch` for policies\n\n## Common Use Cases\n\n### Training on Standard Benchmarks\n```python\n# Atari\nenv = pufferlib.make('atari-pong', num_envs=256)\n\n# Procgen\nenv = pufferlib.make('procgen-coinrun', num_envs=256)\n\n# Minigrid\nenv = pufferlib.make('minigrid-empty-8x8', num_envs=256)\n```\n\n### Multi-Agent Learning\n```python\n# PettingZoo\nenv = pufferlib.make('pettingzoo-pistonball', num_envs=128)\n\n# Shared policy for all agents\npolicy = create_policy(env.observation_space, env.action_space)\ntrainer = PuffeRL(env=env, policy=policy)\n```\n\n### Custom Task Development\n```python\n# Create custom environment\nclass MyTask(PufferEnv):\n    # ... implement environment ...\n\n# Vectorize and train\nenv = pufferlib.emulate(MyTask, num_envs=256)\ntrainer = PuffeRL(env=env, policy=my_policy)\n```\n\n### High-Performance Optimization\n```python\n# Maximize throughput\nenv = pufferlib.make(\n    'my-env',\n    num_envs=1024,      # Large batch\n    num_workers=16,     # Many workers\n    envs_per_worker=64  # Optimize per worker\n)\n```\n\n## Installation\n\n```bash\nuv pip install pufferlib\n```\n\n## Documentation\n\n- Official docs: https:\u002F\u002Fpuffer.ai\u002Fdocs.html\n- GitHub: https:\u002F\u002Fgithub.com\u002FPufferAI\u002FPufferLib\n- Discord: Community support available\n\n",{"name":5,"license":48,"category":49,"metadata":50,"description":7},"MIT license","ml-training",{"skill-author":51},"Synthetic Sciences","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":59,"prompt":69,"zipAvailable":70},[60,63,66],{"tool":61,"command":62},"npx","npx skills add fadrienne\u002FMyGitHub --skill pufferlib",{"tool":64,"command":65},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill pufferlib",{"tool":67,"command":68},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill pufferlib","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002FSKILL.md",true,[72,78,82,86,90,94,100],{"path":73,"resourceType":74,"resourceTypeDesc":75,"sizeBytes":76,"sourceHtmlUrl":77},"references\u002Fenvironments.md","REFERENCE","参考文档",12489,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Freferences\u002Fenvironments.md",{"path":79,"resourceType":74,"resourceTypeDesc":75,"sizeBytes":80,"sourceHtmlUrl":81},"references\u002Fintegration.md",15165,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Freferences\u002Fintegration.md",{"path":83,"resourceType":74,"resourceTypeDesc":75,"sizeBytes":84,"sourceHtmlUrl":85},"references\u002Fpolicies.md",19183,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Freferences\u002Fpolicies.md",{"path":87,"resourceType":74,"resourceTypeDesc":75,"sizeBytes":88,"sourceHtmlUrl":89},"references\u002Ftraining.md",8127,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Freferences\u002Ftraining.md",{"path":91,"resourceType":74,"resourceTypeDesc":75,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fvectorization.md",12962,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Freferences\u002Fvectorization.md",{"path":95,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":98,"sourceHtmlUrl":99},"scripts\u002Fenv_template.py","SCRIPT","脚本",10157,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Fscripts\u002Fenv_template.py",{"path":101,"resourceType":96,"resourceTypeDesc":97,"sizeBytes":102,"sourceHtmlUrl":103},"scripts\u002Ftrain_template.py",8027,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fml-training\u002Fpufferlib\u002Fscripts\u002Ftrain_template.py",[105,132,157,191,217,244],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":112,"publishedAt":43},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[111],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[113,114,115,116,117,118,119,120,25,121,122,123,124,125,126,127,128,129,130,131],"power","statistical power","effect","scipy","regression","choose","design","uv","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":133,"name":134,"nameZh":6,"description":135,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":141,"publishedAt":43},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[138],{"categoryNo":139,"categoryName":140,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[123,142,113,143,144,145,25,28,146,147,26,148,149,150,151,152,153,154,155,156],"statistical analysis","academic research","implementing","selection","reporting","data","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":158,"name":159,"nameZh":6,"description":160,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":161,"categories":162,"keywords":172,"publishedAt":43},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[163,166,167],{"categoryNo":164,"categoryName":165,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":168,"categoryName":169,"taxonomyType":170,"taxonomyTypeDesc":171},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[173,174,175,176,177,178,179,180,181,182,121,183,184,185,186,187,188,189,146,190],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":192,"name":193,"nameZh":6,"description":194,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":195,"categories":196,"keywords":201,"publishedAt":43},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[197,200],{"categoryNo":198,"categoryName":199,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[202,174,203,204,205,29,206,207,208,181,25,209,210,211,212,213,214,215,26,216],"visualization","scientific visualization","error","ready","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":218,"name":219,"nameZh":6,"description":220,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":221,"categories":222,"keywords":229,"publishedAt":43},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[223,226],{"categoryNo":224,"categoryName":225,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":227,"categoryName":228,"taxonomyType":170,"taxonomyTypeDesc":171},"01JBSEEDCAT000000000000030","开发工程师",[230,231,152,232,233,234,235,120,236,25,121,123,237,238,239,240,241,242,189,243],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":245,"name":246,"nameZh":6,"description":247,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":248,"licenseFileUrl":6,"sourceHtmlUrl":249,"categories":250,"keywords":257,"publishedAt":43},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[251,254,255,256],{"categoryNo":252,"categoryName":253,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":164,"categoryName":165,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":168,"categoryName":169,"taxonomyType":170,"taxonomyTypeDesc":171},[173,258,259,260,261,262,202,263,264,25,121,123,265,266,267,42,268,269,174,189],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","references","diagram"]