AGENT SKILL

pufferlib

Version-aware guidance for PufferLib reinforcement-learning environments, vectorization, policies, PuffeRL training, evaluation, and safe checkpoint review. Use when adapting Gymnasium/PettingZoo environments to published PufferLib 3.0.0 or working with the redesigned native 4.0 source line.

License 许可
MIT
最近更新
2026-11-14
Stars
★ 1.6k
Forks
491

01 前言 Front Matter

name
pufferlib
license
MIT
metadata
{ "tags": "reinforcement learning, RL environments, training, machine learning", "version": "1.2", "category": "data-science-and-ml", "maintainer": "Kalaris Labs", "last-reviewed": "2026-07-23" }
description
Version-aware guidance for PufferLib reinforcement-learning environments, vectorization, policies, PuffeRL training, evaluation, and safe checkpoint review. Use when adapting Gymnasium/PettingZoo environments to published PufferLib 3.0.0 or working with the redesigned native 4.0 source line.
allowed-tools
Read Bash Grep Python
compatibility
Bundled CLIs require Python 3.10+ and use only the standard library. Published pufferlib 3.0.0 supports Python >=3.9 but ships as a native-code source archive; current 4.0 source requires Python >=3.10, Torch >=2.9, and an audited CPU/CUDA toolchain. Network, GPU, native builds, environment plug-ins, assets, checkpoints, and external logging are never required by the bundled CLIs.

02 资源文件 14

03 SKILL.md 正文

PufferLib

Use PufferLib with an explicit version profile. Upstream currently has two incompatible surfaces:

Profile Status on 2026-07-23 Main use
pufferlib==3.0.0 Latest stable PyPI release, published 2025-06-23 Python/Gymnasium/PettingZoo emulation, pufferlib.vector, Torch PuffeRL
source 4.0 Upstream default branch; not the latest stable PyPI artifact Native C Ocean environments, native CUDA trainer, optional Torch fallback

Do not combine 3.0 imports with 4.0 config/CLI examples. The 4.0 redesign removed the 3.0 emulation, vector, and pytorch modules from the current package tree.

Safe defaults

  1. Start with bundled synthetic, CPU-only, network-free tools.
  2. Do not import an arbitrary environment by dotted path. Bundled tools accept only allowlisted built-ins and slug identifiers.
  3. Do not install or execute an unreviewed environment package, native extension, ROM, map, checkpoint, or pickle file.
  4. Verify official source, immutable revision, licenses, checksums or attestations, and build hooks. Sandbox native builds and first execution.
  5. Cap steps, environments, agents, workers, threads, buffers, memory, disk, render size, and wall time.
  6. Keep training and evaluation environments/seeds separate.
  7. Default logging to local/none. External logging requires explicit opt-in, disclosure acknowledgment, and separate artifact-upload approval.
  8. Never pass W&B or Neptune credentials via CLI, INI, JSON, tags, run names, or logger configuration. Never print them.
  9. Never dump all environment variables or recursively search for .env.
  10. Hash checkpoint bytes before trusted, sandboxed loading; metadata inspection is not proof of safety.

First local checks

All bundled CLIs are dependency-free and emit strict JSON:

python3 scripts/env_template.py --help
python3 scripts/env_contract_validator.py
python3 scripts/benchmark_vectorization.py --backend serial
python3 scripts/train_template.py
python3 scripts/validate_plan.py
python3 scripts/repro_plan.py

Defaults are synthetic, deterministic, bounded, local, CPU-only, no-network, and dry-run where training would otherwise occur.

Installation and provenance

Published 3.0.0

PyPI supplies only pufferlib-3.0.0.tar.gz:

sha256: 7df3a3e3f5f894d78d2a1f5374097890aec01473183e748abefe4f3faa10eaa9
Requires-Python: >=3.9

After source/build review, create a pinned uv project:

uv venv --python 3.11
uv add --exact --no-sync "pufferlib==3.0.0"
uv lock
uv sync --frozen

Commit pyproject.toml and uv.lock; verify the archive digest and every resolved dependency. The source build can compile native code and fetch build assets, so resolve/build in a sandbox without credentials or sensitive mounts. The uploaded metadata does not pin Torch or CUDA; do not claim a supported CUDA matrix that PyPI does not declare.

Current 4.0 source

The reviewed branch head on 2026-07-23 was:

25647630e1b15330bb3153a5a0d3ff8d234c3acf

Pin the commit, not branch 4.0:

uv add --no-sync \
  "pufferlib @ git+https://github.com/PufferAI/PufferLib.git@25647630e1b15330bb3153a5a0d3ff8d234c3acf"
uv lock

The current package declares Python >=3.10 and Torch >=2.9. Upstream PufferTank currently uses Ubuntu 24.04, Python 3.12, and an NVIDIA CUDA 13.0.2/cuDNN development image with the cu130 Torch index, but does not pin the exact Torch wheel or all system packages. Treat it as a reference, not a complete lock. Never execute a remote installer directly from a pipe.

Read references/training.md before any installation or build.

Environment workflow

1. Validate the contract

Gymnasium reset returns (observation, info). Step returns:

(observation, reward, terminated, truncated, info)

Validate spaces, shapes, dtypes, finite rewards, booleans, reset-before-step, reset-after-end, seeding, and cleanup. terminated is an MDP terminal; truncated is an external cutoff such as a time limit. Preserve the distinction for bootstrapping and metrics.

python3 scripts/env_contract_validator.py \
  --steps 64 --episodes 8 --seed 42

2. Adapt only after review

Published 3.0 uses explicit wrappers:

import pufferlib.emulation

wrapped = pufferlib.emulation.GymnasiumPufferEnv(reviewed_gymnasium_instance)

For a reviewed PettingZoo Parallel environment:

wrapped = pufferlib.emulation.PettingZooPufferEnv(reviewed_parallel_instance)

There is no supported 3.0 pufferlib.emulate(...) shortcut matching the old skill. Read references/environments.md and references/integration.md.

3. Native environments

Published 3.0 PufferEnv requires single_observation_space, single_action_space, and num_agents before super().__init__(buf). It uses in-place vector buffers and returns separate terminal/truncation arrays plus a list of info dictionaries.

Current 4.0 uses C bindings. Start from upstream ocean/squared (single-agent) or ocean/target (multi-agent), build one environment in local/sanitized mode, and verify every buffer size/type/index before optimization.

Vectorization workflow

Published 3.0:

import pufferlib.vector

vecenv = pufferlib.vector.make(
    reviewed_creator,
    backend=pufferlib.vector.Serial,
    num_envs=4,
    seed=42,
)

Move to Multiprocessing only after serial traces pass. Record num_envs, num_workers, batch_size, zero-copy mode, start method, agent count, masks, and actual returned shapes. For multi-agent environments, batch length is based on agent slots, not necessarily num_envs.

Current 4.0 config instead uses:

[vec]
total_agents = 4096
num_buffers = 2
num_threads = 16

Read references/vectorization.md. Benchmark fixed work with warmup and at least three repeats; report simulation and end-to-end training SPS separately. The bundled benchmark measures only its synthetic harness.

Policy workflow

Published 3.0 policies are Torch modules sized from single_observation_space/single_action_space. Stable recurrent composition uses encode_observations and decode_actions; structured emulation uses pufferlib.pytorch.nativize_dtype and nativize_tensor.

Current 4.0 Torch fallback composes:

pufferlib.models.Policy(encoder=encoder, decoder=decoder, network=network)

It provides MLP, MinGRU, LSTM, and GRU network choices; --slowly selects this fallback instead of the native backend. Check output/state shapes, masks, finite values, gradients, and eager-versus-compiled behavior. See references/policies.md.

Training and evaluation

Published 3.0 trainer import:

from pufferlib import pufferl

trainer = pufferl.PuffeRL(train_config, vecenv, policy)

Current 4.0 CLI:

puffer train ENV_NAME
puffer eval ENV_NAME --load-model-path EXACT_TRUSTED_PATH
puffer sweep ENV_NAME

Generate a plan instead of launching by default:

python3 scripts/train_template.py \
  --profile pypi-3.0.0 \
  --environment synthetic \
  --device cpu \
  --total-timesteps 10000

Validate a custom strict-JSON plan:

python3 scripts/validate_plan.py --root . --config plan.json

The schema rejects secret-bearing keys, unbounded resources, dotted environment paths, invalid vector divisibility, mixed-version options, and coupled train/eval seeds. See references/training.md.

Logging

PufferLib 3.0 exposes W&B and Neptune; current 4.0 CLI exposes W&B. Both are optional external services. They may transmit configuration, metrics, source metadata, hardware telemetry, output, and approved artifacts, with privacy, retention, access-control, and cost implications.

  • W&B credential: named environment variable WANDB_API_KEY.
  • Neptune credential: named environment variable NEPTUNE_API_TOKEN.
  • Never put values in arguments/config/logs.
  • Sanitize config keys before logging.
  • Keep source/model upload off unless explicitly approved.

The planner requires both:

python3 scripts/train_template.py \
  --logger wandb \
  --enable-external-logging \
  --acknowledge-external-disclosure

It reports only the required variable name and never reads its value.

Checkpoint workflow

PufferLib 3.0 and the 4.0 Torch fallback use Torch serialization; current native 4.0 writes opaque .bin weights. PyTorch warns that untrusted models are programs and that torch.load uses unpickling.

python3 scripts/inspect_checkpoint.py checkpoint.pt \
  --root . \
  --expected-sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

The inspector hashes and classifies only. It does not call torch.load, import pickle/Torch, inspect archive members, or extract files. Verify source, license, architecture, environment revision, sidecar metadata, and checksum before any sandboxed load. Never use latest in a reproducible evaluation.

Bundled files

Scripts

  • scripts/env_template.py — deterministic synthetic Gymnasium-style template.
  • scripts/env_contract_validator.py — bounded contract and seed checks.
  • scripts/benchmark_vectorization.py — capped serial/spawn synthetic benchmark.
  • scripts/train_template.py — non-executing 3.0/4.0 training-plan generator.
  • scripts/validate_plan.py — strict config/resource/security validator.
  • scripts/inspect_checkpoint.py — metadata/hash inspection without deserialization.
  • scripts/repro_plan.py — separate-seed evaluation and benchmark plan.

References

  • references/environments.md — Gymnasium, stable PufferEnv, emulation, native C.
  • references/vectorization.md — backends, shapes, start methods, benchmarks.
  • references/policies.md — stable/current policy contracts and state safety.
  • references/training.md — installs, config, CLI, PuffeRL, eval, logs, checkpoints.
  • references/integration.md — migration matrix, third-party and credential safety.

Dated upstream sources

Related skills

  • stable-baselines3: Production-ready reinforcement learning algorithms (PPO, SAC, DQN, TD3, DDPG, A2C) with scikit-learn-like API.
  • evaluating-cosmos-policy: Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments.
  • fine-tuning-serving-openpi: Fine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot policy inference acr…

04 同仓库更多 Skills

hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-review-with-annotations

Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-find-and-synthesize-context

Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-update-docs

Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-records

Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-create-or-manage-interactive-html

Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.

★ 1.6k
hashgraph-online
hashgraph-online/awesome-codex-plugins

worktable-collaborate-threads

Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.

★ 1.6k