AGENT SKILL

create-experiment

Run a scoped, local-only experiment and save the report to docs/experiments/. Triggers on: "let's try", "compare A vs B", "prototype", "explore".

License 许可
MIT
最近更新
2026-10-26
Stars
★ 0
Forks
0

01 前言 Front Matter

id
create-experiment
name
create-experiment
task
[ "review", "documentation" ]
type
skill
model
sonnet
owner
@kaiohenricunha
tools
Read, Grep, Glob, Bash, Write
domain
[ "devex" ]
created
1777420800000
updated
1777507200000
version
1.0.0
dotbabel
{ "compute": { "mode": "dynamic", "binding": "consumer", "rationale": "Scoped engineering follows an existing implementation or review workflow with checkable results.", "requirement": "routine" } }
maturity
validated
platform
[ "none" ]
description
Run a scoped, local-only experiment and save the report to docs/experiments/. Triggers on: "let's try", "compare A vs B", "prototype", "explore".
allowed-tools
Read Grep Glob Bash Write
argument-hint
[topic or hypothesis]

03 SKILL.md 正文

Run a scoped, local-only experiment and produce a structured report saved to the project's docs/experiments/ directory.

Trigger: when the user asks "let's try X", "I want to compare A vs B", "can we prototype Y", "explore Z before we commit", or invokes /create-experiment directly. Use this skill before /spec (which is for design docs, not exploratory probes) and after plain shell tinkering becomes too messy to track.

Arguments: $ARGUMENTS — a topic or hypothesis (e.g. "ripgrep vs grep on this repo", "switch from pnpm to bun", "does Postgres LISTEN/NOTIFY scale to N clients"). Required — if empty, ask the user what they want to explore.

Purpose

/create-experiment is not a spec (no formal design), not an audit (it doesn't enumerate issues), not a fix (it doesn't ship anything). It is a decision-grade probe: refine a hypothesis, run it locally, capture results — including negative ones — and recommend a next move.

The output is a dated markdown file under docs/experiments/ plus a sandbox directory containing the runnable artifacts. Both are left untracked for the user to review.

Steps

Step 0 — Refine the goal (interactive, blocking)

If $ARGUMENTS already covers hypothesis + observable success signal, echo the refined goal back as a paragraph + bulleted success criteria and wait for user sign-off before proceeding. Skip questions already answered in $ARGUMENTS or recent context.

Otherwise, ask up to 4 short questions:

  1. What are you trying to learn? — one-sentence hypothesis.
  2. What does "it worked" look like? — concrete, observable signal (a benchmark number, a working prototype, a config that boots, a passing test).
  3. What's out of scope? — what should NOT be touched.
  4. Time-box — sketch (≤30 min), half-day (≤4 hr), or full-day (≤8 hr).

Echo the refined goal back as a paragraph + bulleted success criteria. Wait for explicit user sign-off before continuing. Do not proceed to Step 1 until the user confirms.

Step 1 — Plan the experiment

  • Propose 1–3 approaches to try. Each is a distinct path to the same hypothesis.
  • For each approach, sketch: what gets installed, what runs, what's measured.
  • Pick the sandbox target:
    • Default: a fresh git worktree at .claude/worktrees/experiment-<slug>/ branched from the latest origin/main. Run git fetch origin main first.
    • Fallback: ~/experiments/<slug>/ if there is no enclosing git repo.
  • Show this plan to the user and get a final go-ahead before touching the system.

Step 2 — Set up the environment

Execute setup commands in the sandbox. Capture every command and its output as the reproducibility ledger — redact any tokens or credentials before including output in the report. Examples:

  • git worktree add .claude/worktrees/experiment-<slug> -b experiment/<slug> origin/main
  • npm i <pkg> / pnpm add <pkg> / bun add <pkg>
  • python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt
  • docker compose up -d
  • cargo new <name> && cd <name>

If a setup step fails, document the failure, attempt one recovery (e.g. install a missing system dep), and proceed only if it succeeds. Do not silently swallow errors.

Step 3 — Execute the approaches

Approaches are independent by default — execute concurrently when possible (parallel tool calls or background jobs with per-approach log files). Fall back to serial execution only when one approach depends on another's artifact.

For each approach:

  • Save code/config under experiments/<slug>/<approach>/ inside the sandbox.
  • Run the approach. For measurement commands and any failing command, capture full output — at minimum the last 10 lines.
  • Evaluate against the success criteria from Step 0. Record the result as PASS, PARTIAL, or FAIL.
  • If an approach surfaces a blocker that invalidates the hypothesis itself, stop and surface that to the user before continuing — they may want to refine the goal.

Step 4 — Compare & recommend

Build a comparison table that maps each approach against the agreed success criteria. Pick a recommendation, or explicitly say "none of these — here's why and what to try next." A clean negative result is a complete experiment.

Step 5 — Write the report

Generate a filename: <topic-slug>-<YYYY-MM-DD>.md in lowercase kebab-case (e.g. ripgrep-vs-grep-2026-04-29.md). Create docs/experiments/ if it doesn't exist. Use this structure:

# Experiment: <Topic> — <YYYY-MM-DD>

<One-sentence hypothesis.>

## Goal

<2–3 sentences. What we're learning and why this experiment was run now.>

## Success Criteria

- <observable signal 1>
- <observable signal 2>

## Environment Setup

Sandbox: `<absolute path chosen in Step 1>`

```bash
<exact commands run, in order>
```

<output snippets for any measurement or failing command — last 10 lines minimum>

## Approaches Tried

### Approach 1: <Name>

- **Idea:** <one line>
- **Code/config:** `<path-in-sandbox>`
- **Commands:**

  ```bash
  <commands>
  ```

- **Result:** PASS | PARTIAL | FAIL — <observed signal vs criterion>
- **Notes:** <gotchas, surprises, dead-ends>

<!-- Add sections for each approach tried; omit if only one approach was attempted -->

## Comparison

| Approach | Result | <Criterion 1> | <Criterion 2> | Effort |
| -------- | ------ | ------------- | ------------- | ------ |
| ...      | ...    | ...           | ...           | ...    |

## Recommendation

**<Adopt Approach N | Iterate further | Abandon>**

<2–3 sentences. Why this conclusion best matches the success criteria. Call out
assumptions or unresolved questions.>

## Next Step

- Promote to `/spec <topic>` to formalize the chosen approach, OR
- Run `/fix-with-evidence <topic>` to implement directly, OR
- Run `/reproduce-bug <symptom>` to capture a surprising failure as a failing regression test, OR
- Re-run `/create-experiment` with a refined hypothesis, OR
- Drop it — this document is the negative-result record.

## Sandbox Cleanup

```bash
git worktree remove .claude/worktrees/experiment-<slug>   # or
rm -rf ~/experiments/<slug>
```

Step 6 — Report back

Reply with: doc path, one-sentence recommendation, and sandbox path. Do not paste the full document into chat.

Rules

  • All work runs locally. No production endpoints, no cloud writes, no real auth tokens, no shared infrastructure. If the experiment needs an external service, mock it or use a disposable test account.
  • Environment setup is part of the experiment — every install/config/boot command must appear in the report. Reproducibility is the bar.
  • Capture failed approaches too. Negative results are valuable signal and must be in the report.
  • Do not commit the experiment doc or the sandbox. Leave both untracked for the user.
  • Do not install global tools (apt, brew, npm i -g) without explicit user approval during Step 1.
  • Tables and code blocks over prose. No filler.
  • An experiment is done when success criteria are evaluated — not when "everything works." A clean failure is a finished experiment.

04 同仓库更多 Skills

kaiohenricunha
kaiohenricunha/dotbabel

yolo

Start temporary autonomy for the current task or an explicit scope with a required --for expiry. Codex permits scoped pushes, merges, deploys, and releases after strict testing and validation; Full access is a separate runtime setting. Claude Code uses a session permission hook with a protected-action denylist.

开发工程师
★ 0
kaiohenricunha
kaiohenricunha/dotbabel

veracity-audit

Audit a data pipeline for Veracity and Value. Dispatches data-scientist, compliance-auditor, and data-engineer agents with project context injected at dispatch time. All source paths are supplied via flags — no defaults, no project assumptions. Subcommands: audit (full pipeline walk), score-check (scoring math only), gate-check (gate coverage only), source-trace <label> (single source end-to-end). Invoke when: "audit pipeline", "veracity check", "scoring math correct?", "check gates", "trace source", "verify quality gates", "formula correct?".

律师与法务Skill与智能体开发
★ 0
kaiohenricunha
kaiohenricunha/dotbabel

validate-spec

Audit an already-implemented spec against the codebase. Walks each constraint (ARCH-N, PERF-N, KD-N, etc.) and acceptance criterion, grounds findings in file:line evidence, runs the spec.json acceptance_commands, and writes a single audit doc to docs/audits/. Use whenever the user asks to "validate a spec", "audit a spec", "check if spec is implemented", "verify the spec is done", "is this spec really done", or otherwise wants closure on spec-driven work. Read-only against the spec — produces an audit, never modifies the spec itself.

★ 0
kaiohenricunha
kaiohenricunha/dotbabel

terragrunt-specialist

Deep-dive Terragrunt hierarchy review, DRY pattern audits, and run-all orchestration analysis. Use for structured investigations of multi-environment Terragrunt layouts, dependency graphs, remote state config, and hook correctness. Triggers on: "Terragrunt audit", "run-all review", "dependency block", "DRY pattern review", "env hierarchy audit", "mock_outputs", "terragrunt hooks".

运维与部署
★ 0
kaiohenricunha
kaiohenricunha/dotbabel

terraform-specialist

Deep-dive Terraform architecture review, module design, state management, and migration. Use for structured investigations of Terraform workspaces, provider configuration, module coupling, import workflows, and test coverage. Triggers on: "Terraform audit", "module review", "state management", "Terraform import", "workspace design", "provider config review", "Terraform testing".

测试与质量运维与部署
★ 0
kaiohenricunha
kaiohenricunha/dotbabel

spec

Create structured engineering specs through interactive pairing. Use when the user wants to create a spec, design doc, technical specification, architecture document, RFC, or engineering plan. Triggers on "let's spec this out", "create a spec", "design doc", "write a technical plan", "plan the architecture". Works for greenfield and brownfield projects. Outputs to docs/specs/.

设计师视觉设计
★ 0