mutation
暂无简介
- License 许可
- MIT
- 最近更新
- 2026-05-26
- Stars
- ★ 0
- Forks
- 0
03 SKILL.md 正文
karpathy-mutation
Purpose
Generate controlled single-dimension mutation candidates from a baseline SKILL.md, producing one variant per mutation type so that each can be independently evaluated.
Use when
- You have a baseline EvalResult and want to explore whether targeted changes improve the score
- The baseline scores below 0.85 on at least one dimension and you have identified the lowest-scoring dimensions
- You are running a full optimization cycle and need the mutation stage to produce candidates
- You want to test whether decomposing compound steps or tightening wording improves correctness or simplicity scores
Do NOT use when
- The baseline final_score is already above 0.90 — the improvement margin is unlikely to justify the cycle cost — use a more specific skill instead.
- You want to change multiple dimensions at once — that violates the one-dimension constraint and makes it impossible to isolate the cause of any score change — use a more specific skill instead.
- You have not yet run
karpathy-evaluationon the baseline — you need a baseline score before mutating — use a more specific skill instead. - The skill has failing binary checks — fix structural issues before running mutations — use a more specific skill instead.
Inputs required
- Full baseline SKILL.md content
- A stable
skill_idandparent_id(use"baseline"for unmodified content) - The desired number of candidates (1–6, default 5)
- The baseline EvalResult (to know which dimensions to prioritise)
Constraints
- Each candidate must change exactly ONE mutation dimension
- All 13 required SKILL.md sections must be present in every candidate
- The Purpose and Output contract sections must not be emptied or materially degraded
- Mutation must not introduce forbidden strings (TODO, TBD, FIXME, {{}})
- Each candidate_id must be stable:
<parent_id>-<mutation_type>
Step-by-step workflow
- Confirm the baseline content has all 13 required sections and no placeholder text.
- For each selected mutation type (up to n), apply exactly the one-dimension transformation described in
docs/mutation-system.md. - After each mutation, verify that the section count in the candidate equals the section count in the baseline.
- Count tokens for each candidate:
ceil(word_count × 1.3). - Assemble each MutationCandidate dict with: candidate_id, parent_id, mutation_type, mutation_description, content, token_count.
- Return the list of candidates for downstream regression and evaluation.
Output contract
- A list of MutationCandidate dicts, each matching
evals/schemas/run_report.schema.json#/definitions/MutationCandidate - Each dict includes: candidate_id, parent_id, mutation_type, mutation_description, content, token_count
- At most 6 candidates per run (one per mutation type)
- The list is appended to
memory/candidate_archive.json
Validation checklist
- Confirm each candidate has exactly 12
##section headers (matching the baseline count) - Confirm each candidate_id follows the pattern
<parent_id>-<mutation_type> - Confirm no candidate content contains TODO, TBD, FIXME, or
{{ - Run
python -m runners.mutation_runner --skill <path> --n 3and confirm valid JSON returned - Confirm the
mutation_descriptionaccurately describes what changed in one sentence
Related skills
karpathy-evaluation— must be run on baseline before calling this skillkarpathy-regression— must be run on every candidate output from this skillkarpathy-optimizer— selects the best candidate from this skill's output
References
../../docs/mutation-system.md../../agents/mutation_agent.py../../evals/testcases/flutter_deeplink/
Real example
Historical static snapshot: given the flutter deep link baseline, the decomposition_steps mutation finds the compound step "Configure the route and register it in the router" and splits it into "Configure the route in app_router.dart" and "Keep the route definition in app_router.dart in sync with the router configuration." The simplicity dimension improves because each step is now independently executable.
Real file output sample
memory/candidate_archive.json
04 同仓库更多 Skills
toolkit-i18n-operator
Use the toolkit-i18n CLI for repeated Flutter localization workflows with compact outputs, CSV validation, key coverage reporting, generated JSON exports, and safe non-mutating behavior.
toolkit-design-operator
Use the toolkit-design CLI for repeated repository workflows.
toolkit-ci-operator
Use the toolkit-ci CLI for repeated repository workflows.
toolkit-arch-operator
Use the toolkit-arch CLI for repeated repository workflows, including Flutter i18n usage scans, layer checks, and localization coverage.
web-testing
暂无简介
web-state
暂无简介