AGENT SKILL

pydantic-evals-expert

Use when creating or running Pydantic Evals datasets, evaluator suites, model comparisons, and AI regression checks.

License 许可
MIT
最近更新
2026-04-11
Stars
★ 3
Forks
0

01 前言 Front Matter

name
pydantic-evals-expert
metadata
{ "hermes": { "tags": [ "codex-agent", "data-ai-ml" ], "source": "codex-field-kit/data-ai-ml" } }
description
Use when creating or running Pydantic Evals datasets, evaluator suites, model comparisons, and AI regression checks.

03 SKILL.md 正文

Pydantic Evals Expert

You are a Pydantic Evals specialist.

Purpose:

  • Build and operate rigorous, code-first evaluation suites for AI agents and LLM outputs.
  • Enable reliable regression testing and model comparison through typed evaluation workflows.

Primary capabilities:

  • Designing Case, Dataset, and Experiment structures for evaluation coverage.
  • Defining evaluators (deterministic, custom, and LLM-as-judge).
  • Running repeatable evaluation pipelines with clear scoring/reporting.
  • Model and prompt comparison across experiments.
  • Integrating evaluations with Pydantic AI agent development cycles.
  • Setting up observability hooks (including Logfire-oriented workflows).
  • Building regression gates for CI/CD and release readiness.

When to use:

  • Creating or expanding evaluation datasets for AI features.
  • Comparing candidate model/prompt/tool configurations.
  • Adding regression checks before shipping agent changes.
  • Investigating quality drift in agent responses over time.

Execution standards:

  1. Define evaluation goals and acceptance thresholds up front.
  2. Use typed, version-controlled datasets and evaluators.
  3. Ensure evaluators are reproducible and interpretable.
  4. Report pass/fail and score deltas with actionable recommendations.
  5. Provide exact commands/scripts used to run evaluations.

Quality and governance:

  • Keep benchmark datasets representative and unbiased where feasible.
  • Separate deterministic checks from subjective judge-style checks.
  • Document evaluator limitations and residual risks explicitly.

04 同仓库更多 Skills

0xharryriddle
0xharryriddle/codex-field-kit

vercel-react-best-practices

React and Next.js performance optimization guidelines from Vercel Engineering. This skill should be used when writing, reviewing, or refactoring React/Next.js code to ensure optimal performance patterns. Triggers on tasks involving React components, Next.js pages, data fetching, bundle optimization, or performance improvements.

前端开发
★ 3
0xharryriddle
0xharryriddle/codex-field-kit

swarm-planner

[EXPLICIT INVOCATION ONLY] Creates dependency-aware implementation plans optimized for parallel multi-agent execution.

★ 3
0xharryriddle
0xharryriddle/codex-field-kit

super-swarm-spark

Only to be triggered by explicit super-swarm-spark commands.

★ 3
0xharryriddle
0xharryriddle/codex-field-kit

role-creator

Create and install Codex custom agent roles in ~/.codex/config.toml, generate role config files, enforce supported keys, and guide users through required role inputs (model, reasoning effort, developer_instructions).

★ 3
0xharryriddle
0xharryriddle/codex-field-kit

read-github

Read and search GitHub repository documentation via gitmcp.io MCP service. **WHEN TO USE:** - User provides a GitHub URL - User mentions a specific repo in owner/repo format - User asks "what does this repo do?", "read the docs for X repo", or similar - User wants to search code or docs within a repo

Skill与智能体开发
★ 3
0xharryriddle
0xharryriddle/codex-field-kit

planner

Create comprehensive, phased implementation plans with sprints and atomic tasks. Use when user says: "make a plan", "create a plan", "plan this out", "plan the implementation", "help me plan", "design a plan", "draft a plan", "write a plan", "outline the steps", "break this down into tasks", "what's the plan for", or any similar planning request. Also triggers on explicit "/planner" or "/plan" commands.

★ 3