[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6R62MM4Z9ZAZPWEGACC4KGC":3,"skill-same:01M6R62MM4Z9ZAZPWEGACC4KGC":96},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":35,"linkOnly":36,"markdownContent":37,"markdownBody":38,"frontMatter":39,"refName":40,"commitSha":41,"skillDirectory":42,"forksCount":13,"repositoryHtmlUrl":43,"repositoryHomepageUrl":6,"ownerHtmlUrl":44,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":45,"install":46,"resources":59},"01M6R62MM4Z9ZAZPWEGACC4KGC","eval-a-skill",null,"Scaffold a structurally valid eval triplet for a target skill under .ai\u002Fevals\u002F. CI checks structure only; the LM-judge runs out-of-band, never in CI.","r3dlex\u002Fskills","r3dlex","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F291373160?v=4","01M6PEFNGGBVB7SHAPWXFFX933","01M6R5SDWFHT94734KJFMYE1SX",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34],"eval","eval a skill","valid","scaffold","ci","out-of-band","never","only","evals","structure","judge","for","2026-11-10 10:12:30",false,"---\nname: eval-a-skill\ndescription: 'Scaffold a structurally valid eval triplet for a target skill under .ai\u002Fevals\u002F. CI checks structure only; the LM-judge runs out-of-band, never in CI.'\n---\n\n# Eval a Skill\n\n## Quick Start\n\nGiven a TARGET skill, scaffold the eval triplet under `.ai\u002Fevals\u002F\u003Cskill>\u002F`,\nmatching the P0 eval shape (see `ai-catapult-init\u002Fmodules\u002Fevals.md` and the golden\nfixture `reference\u002Ffixtures\u002Fv3\u002Fstandalone\u002F.ai\u002Fevals\u002Fexample-output-eval\u002F`).\n\nRun the generator:\n\n```bash\npython3 eval-a-skill\u002Fscaffold-eval.py --skill \u003Ctarget> --root .\n```\n\nThis writes three artifacts and makes no model or network call. Re-running is\nidempotent — it rewrites the triplet in place, never duplicates.\n\n## The eval triplet (P0 shape)\n\n| Artifact | Purpose |\n| --- | --- |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Fevalset.json` | Labelled cases: `schema_version`, stable `set_id`, `kind` (`output` or `trajectory`), and a non-empty `cases` array. |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Frubric.md` | Required scoring rubric. Criteria, dimensions, weights summing to `1.0`, and a passing threshold. |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Fjudge-config.json` | LM-judge harness stub: judge `tier`, `mode`, `harness`, `evaluates`, threshold, and `execution: out-of-band`. |\n\nAfter scaffolding, fill the rubric and cases with behavior specific to the\ntarget skill. A missing or empty rubric fails the eval-coverage gate.\n\n## Structural validation — CI only\n\nIn CI the bar is **structural only** and offline: the triplet must exist and be\nwell-formed (both JSON files parse and carry their required keys; the rubric is\nnon-empty). CI performs no model or network call. This mirrors the eval-coverage\ngate in `ai-catapult-init\u002Fmodules\u002Fci-policy.md`.\n\nValidate the scaffolded triplet structurally:\n\n```bash\npython3 -m json.tool .ai\u002Fevals\u002F\u003Cskill>\u002Fevalset.json     >\u002Fdev\u002Fnull\npython3 -m json.tool .ai\u002Fevals\u002F\u003Cskill>\u002Fjudge-config.json >\u002Fdev\u002Fnull\ntest -s .ai\u002Fevals\u002F\u003Cskill>\u002Frubric.md\n```\n\nCI proves the eval is **declared**, not that it passes. Eval quality is a\nseparate, out-of-band step.\n\n## Judge execution — out-of-band only\n\nThe LM-judge that scores the target skill against the rubric runs\n**out-of-band**, outside CI, and is **never invoked in CI**. `judge-config.json`\ndeclares `execution: out-of-band` and ships a non-executing `stub` harness: it\ndocuments the shape an out-of-band runner consumes, nothing more.\n\nThe opt-in out-of-band runner (Option B in the plan) is the only path that\nactually invokes the judge. Run it manually, away from CI, and record the\njudgment as evidence — see the worked example\n`reference\u002Ffixtures\u002Fv3\u002Fstandalone\u002F.ai\u002Fevals\u002Fexample-output-eval\u002Fjudgment-demo.json`\nand its discoverable offline checker `tests\u002Flm_judge_demo_test.sh`. The recorded\njudgment carries a per-criterion score, the rubric-weighted aggregate, and a\nverdict against the passing threshold.\n\n## Safety rules\n\n- Do not weaken the structural validator to make a missing triplet pass; author\n  the triplet or record an audited exception in\n  `.ai\u002Fevals\u002Fcoverage-exceptions.json`.\n- Do not add a model or network dependency to the CI eval path.\n- Do not claim a passing eval in CI; CI proves declaration, the judge proves\n  quality out-of-band.\n\n## References\n\n- [ai-catapult-init\u002Fmodules\u002Fevals.md](..\u002Fai-catapult-init\u002Fmodules\u002Fevals.md) — eval scaffold spec and the coverage gate.\n- [ADR 0002](..\u002Fdocs\u002Farchitecture\u002Fadr\u002F0002-evals-as-verification-gate.md) — evals as a verification gate.\n","# Eval a Skill\n\n## Quick Start\n\nGiven a TARGET skill, scaffold the eval triplet under `.ai\u002Fevals\u002F\u003Cskill>\u002F`,\nmatching the P0 eval shape (see `ai-catapult-init\u002Fmodules\u002Fevals.md` and the golden\nfixture `reference\u002Ffixtures\u002Fv3\u002Fstandalone\u002F.ai\u002Fevals\u002Fexample-output-eval\u002F`).\n\nRun the generator:\n\n```bash\npython3 eval-a-skill\u002Fscaffold-eval.py --skill \u003Ctarget> --root .\n```\n\nThis writes three artifacts and makes no model or network call. Re-running is\nidempotent — it rewrites the triplet in place, never duplicates.\n\n## The eval triplet (P0 shape)\n\n| Artifact | Purpose |\n| --- | --- |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Fevalset.json` | Labelled cases: `schema_version`, stable `set_id`, `kind` (`output` or `trajectory`), and a non-empty `cases` array. |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Frubric.md` | Required scoring rubric. Criteria, dimensions, weights summing to `1.0`, and a passing threshold. |\n| `.ai\u002Fevals\u002F\u003Cskill>\u002Fjudge-config.json` | LM-judge harness stub: judge `tier`, `mode`, `harness`, `evaluates`, threshold, and `execution: out-of-band`. |\n\nAfter scaffolding, fill the rubric and cases with behavior specific to the\ntarget skill. A missing or empty rubric fails the eval-coverage gate.\n\n## Structural validation — CI only\n\nIn CI the bar is **structural only** and offline: the triplet must exist and be\nwell-formed (both JSON files parse and carry their required keys; the rubric is\nnon-empty). CI performs no model or network call. This mirrors the eval-coverage\ngate in `ai-catapult-init\u002Fmodules\u002Fci-policy.md`.\n\nValidate the scaffolded triplet structurally:\n\n```bash\npython3 -m json.tool .ai\u002Fevals\u002F\u003Cskill>\u002Fevalset.json     >\u002Fdev\u002Fnull\npython3 -m json.tool .ai\u002Fevals\u002F\u003Cskill>\u002Fjudge-config.json >\u002Fdev\u002Fnull\ntest -s .ai\u002Fevals\u002F\u003Cskill>\u002Frubric.md\n```\n\nCI proves the eval is **declared**, not that it passes. Eval quality is a\nseparate, out-of-band step.\n\n## Judge execution — out-of-band only\n\nThe LM-judge that scores the target skill against the rubric runs\n**out-of-band**, outside CI, and is **never invoked in CI**. `judge-config.json`\ndeclares `execution: out-of-band` and ships a non-executing `stub` harness: it\ndocuments the shape an out-of-band runner consumes, nothing more.\n\nThe opt-in out-of-band runner (Option B in the plan) is the only path that\nactually invokes the judge. Run it manually, away from CI, and record the\njudgment as evidence — see the worked example\n`reference\u002Ffixtures\u002Fv3\u002Fstandalone\u002F.ai\u002Fevals\u002Fexample-output-eval\u002Fjudgment-demo.json`\nand its discoverable offline checker `tests\u002Flm_judge_demo_test.sh`. The recorded\njudgment carries a per-criterion score, the rubric-weighted aggregate, and a\nverdict against the passing threshold.\n\n## Safety rules\n\n- Do not weaken the structural validator to make a missing triplet pass; author\n  the triplet or record an audited exception in\n  `.ai\u002Fevals\u002Fcoverage-exceptions.json`.\n- Do not add a model or network dependency to the CI eval path.\n- Do not claim a passing eval in CI; CI proves declaration, the judge proves\n  quality out-of-band.\n\n## References\n\n- [ai-catapult-init\u002Fmodules\u002Fevals.md](..\u002Fai-catapult-init\u002Fmodules\u002Fevals.md) — eval scaffold spec and the coverage gate.\n- [ADR 0002](..\u002Fdocs\u002Farchitecture\u002Fadr\u002F0002-evals-as-verification-gate.md) — evals as a verification gate.\n",{"name":5,"description":7},"main","037fe34299eb9c4a30b51f1a6bd101eae1c469b1","04-validate-handoff\u002Feval-a-skill","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills","https:\u002F\u002Fgithub.com\u002Fr3dlex","2026-10-29 16:22:54",{"source":8,"skillName":5,"commands":47,"prompt":57,"zipAvailable":58},[48,51,54],{"tool":49,"command":50},"npx","npx skills add r3dlex\u002Fskills --skill eval-a-skill",{"tool":52,"command":53},"bunx","bunx skills add r3dlex\u002Fskills --skill eval-a-skill",{"tool":55,"command":56},"pnpm","pnpm dlx skills add r3dlex\u002Fskills --skill eval-a-skill","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002FSKILL.md",true,[60,66,72,76,80,84,88,92],{"path":61,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":64,"sourceHtmlUrl":65},"lib\u002Finvoked.d.mts","OTHER","其他",67,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Finvoked.d.mts",{"path":67,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":70,"sourceHtmlUrl":71},"lib\u002Finvoked.mjs","SCRIPT","脚本",327,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Finvoked.mjs",{"path":73,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":74,"sourceHtmlUrl":75},"lib\u002Fnode-version.d.mts",514,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Fnode-version.d.mts",{"path":77,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":78,"sourceHtmlUrl":79},"lib\u002Fnode-version.mjs",860,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Fnode-version.mjs",{"path":81,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":82,"sourceHtmlUrl":83},"lib\u002Fscaffold-eval.d.mts",148,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Fscaffold-eval.d.mts",{"path":85,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":86,"sourceHtmlUrl":87},"lib\u002Fscaffold-eval.mjs",10781,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Fscaffold-eval.mjs",{"path":89,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":90,"sourceHtmlUrl":91},"lib\u002Fscaffold-eval.mjs.map",3030,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Flib\u002Fscaffold-eval.mjs.map",{"path":93,"resourceType":68,"resourceTypeDesc":69,"sizeBytes":94,"sourceHtmlUrl":95},"scaffold-eval.py",1194,"https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002F04-validate-handoff\u002Feval-a-skill\u002Fscaffold-eval.py",[97,105,114,123,133,158],{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":35},"01M6R652E42RZT7M2V3RWW991Z","skill","Fixture.","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Ftests\u002Ffixtures\u002Fcatalog-malformed\u002Fskill\u002FSKILL.md",[],[104],"fixture",{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":112,"publishedAt":35},"01M6R64MTPF0FGWHFS4DWP40GR","stable-skill","Fixture stable-skill.","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Ftests\u002Ffixtures\u002Fcatalog-lifecycle\u002Fstable-skill\u002FSKILL.md",[111],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},[113,104],"stable skill",{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":118,"categories":119,"keywords":121,"publishedAt":35},"01M6R64HD4QVAYM2R0JSYNVDJW","experimental-skill","Fixture experimental-skill.","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Ftests\u002Ffixtures\u002Fcatalog-lifecycle\u002Fexperimental-skill\u002FSKILL.md",[120],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},[122,104],"experimental skill",{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":130,"publishedAt":35},"01M6R64FZH05TEJQW75QX1E5XD","deprecated-skill","Fixture deprecated-skill.","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Ftests\u002Ffixtures\u002Fcatalog-lifecycle\u002Fdeprecated-skill\u002FSKILL.md",[129],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},[131,132,104],"deprecated skill","deprecated",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":139,"publishedAt":35},"01M6R64D13X5DC980GREWFSSFQ","karpathy-guidelines","Apply think-before-coding, simplicity, surgical-change, and goal-driven engineering rules. Use when planning, implementing, or reviewing code.","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Freference\u002Fgolden-skills\u002F.agents\u002Fskills\u002Fkarpathy-guidelines\u002FSKILL.md",[],[140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157],"karpathy guidelines","karpathy","guidelines","Rules","use","simplicity","goal","think","driven","reviewing code","implementing","apply","when","coding","change","code","planning","engineering",{"skillNo":159,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":160,"categories":161,"keywords":162,"publishedAt":35},"01M6R640ZS60BRV9P49FT46ZJX","https:\u002F\u002Fgithub.com\u002Fr3dlex\u002Fskills\u002Fblob\u002Fmain\u002Freference\u002Fgolden-root\u002F.agents\u002Fskills\u002Fkarpathy-guidelines\u002FSKILL.md",[],[142,140,141,154,148,155,156,157,153,143,144,145,146,147,149,150,151,152]]