[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CYGB7GKNFHT1HP1TC5J2DX":3,"skill-same:01M6CYGB7GKNFHT1HP1TC5J2DX":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":47,"repositoryHtmlUrl":48,"repositoryHomepageUrl":6,"ownerHtmlUrl":49,"ownerBlogUrl":50,"ownerDescription":6,"updatedAt":51,"install":52,"resources":65},"01M6CYGB7GKNFHT1HP1TC5J2DX","validate-skill-tests",null,"Use after creating or modifying a plugin skill to verify promptfoo test coverage. Checks dataset existence, fixture reference validity, new behavior coverage, and assertion quality.","rnagrodzki\u002Fsdlc-marketplace","rnagrodzki","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F303481?v=4","01M6CYFWQK3A8WR9W324Z5Q5DR","Rafal Nagrodzki","01M6CYFWVKS23ECZCP9R7DTQVN",0,"MIT","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fvalidate-skill-tests\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"validate","validate skill tests","test","assertion quality","plugin","promptfoo","verify","quality","behavior","coverage","use","test coverage","plugin skill","reference","fixture","2026-11-10 10:12:30",false,"---\nname: validate-skill-tests\ndescription: \"Use after creating or modifying a plugin skill to verify promptfoo test coverage. Checks dataset existence, fixture reference validity, new behavior coverage, and assertion quality.\"\n---\n\n# Validate Skill Tests\n\nAfter creating a new plugin skill or modifying a skill's workflow or flags, run through\nthis checklist to verify promptfoo test coverage is adequate.\n\n## When to Use\n\nInvoke this skill when you have:\n\n- Created a new skill under `plugins\u002Fsdlc-utilities\u002Fskills\u002F`\n- Modified a skill's workflow steps or flags (argument-hint changed)\n- Changed a skill's behavioral output (different format, new branches)\n\n## Step 1 — Identify Modified Skills\n\nCheck which skills were modified in the current changeset:\n\n```bash\ngit diff --name-only HEAD | grep 'plugins\u002Fsdlc-utilities\u002Fskills\u002F.*\u002FSKILL.md'\n```\n\nExtract skill names from the paths.\n\n## Step 2 — Dataset Existence\n\nFor each modified skill, check whether a test dataset exists:\n\n```\ntests\u002Fpromptfoo\u002Fdatasets\u002F\u003Cskill-name>.yaml\n```\n\nIf the dataset does not exist, create one using `tests\u002Fpromptfoo\u002Fdatasets\u002Freview-sdlc.yaml`\nas the canonical reference for structure and assertion patterns.\n\nEach test case must include:\n- `description` — identifies skill and specific behavior (e.g., `\"commit-sdlc: --auto flag skips confirmation\"`)\n- `vars.skill_path` — path to the SKILL.md being tested\n- `vars.project_context` — path to a fixture file in `tests\u002Fpromptfoo\u002Ffixtures\u002F`\n- `assert` — array of assertions\n\n## Step 3 — Fixture Reference Validity\n\nFor each test case in the dataset, verify:\n\n1. Every `file:\u002F\u002Ffixtures\u002F\u003Cname>.md` reference points to an existing file under `tests\u002Fpromptfoo\u002Ffixtures\u002F`\n2. Every `file:\u002F\u002Ffixtures-fs\u002F\u003Cname>\u002F` reference points to an existing directory under `tests\u002Fpromptfoo\u002Ffixtures-fs\u002F`\n\nReport any broken references.\n\n## Step 4 — New Behavior Coverage\n\nIf the skill gained a new flag (compare `argument-hint` in the SKILL.md before\u002Fafter) or\na new workflow step:\n\n1. Check whether any existing test case exercises the new behavior\n2. If not, suggest adding a test case with:\n   - A descriptive name referencing the new flag\u002Fstep\n   - A fixture that exercises the new code path\n   - Both structural and behavioral assertions\n\n## Step 5 — Assertion Quality\n\nReview each test case's assertions for completeness:\n\n| Required | Type | Purpose |\n|----------|------|---------|\n| At least 1 | `icontains` or `regex` | Structural — verifies specific output strings |\n| At least 1 | `llm-rubric` | Behavioral — verifies semantic correctness |\n\nFlag test cases that have only structural OR only behavioral assertions.\nWell-written `llm-rubric` assertions are specific enough to distinguish pass from fail\n(e.g., \"The commit message follows conventional commits format and references the changed files\"\nnot \"The output looks correct\").\n\n## Step 6 — Report Findings\n\nPresent findings in this format:\n\n| Skill | Check | Status | Detail |\n|-------|-------|--------|--------|\n| `\u003Cname>` | Dataset exists | ✓\u002F✗ | path or \"missing\" |\n| `\u003Cname>` | Fixture refs valid | ✓\u002F✗ | broken refs listed |\n| `\u003Cname>` | New behavior covered | ✓\u002F✗\u002FN\u002FA | uncovered flags\u002Fsteps |\n| `\u003Cname>` | Assertion quality | ✓\u002F✗ | test cases with gaps |\n\n## DO NOT\n\n- Run `promptfoo eval` automatically — evaluation runs must always be initiated by the user\n- Create test cases that test implementation details rather than observable behavior\n- Write assertions that match on volatile output (timestamps, random IDs, file paths with user dirs)\n","# Validate Skill Tests\n\nAfter creating a new plugin skill or modifying a skill's workflow or flags, run through\nthis checklist to verify promptfoo test coverage is adequate.\n\n## When to Use\n\nInvoke this skill when you have:\n\n- Created a new skill under `plugins\u002Fsdlc-utilities\u002Fskills\u002F`\n- Modified a skill's workflow steps or flags (argument-hint changed)\n- Changed a skill's behavioral output (different format, new branches)\n\n## Step 1 — Identify Modified Skills\n\nCheck which skills were modified in the current changeset:\n\n```bash\ngit diff --name-only HEAD | grep 'plugins\u002Fsdlc-utilities\u002Fskills\u002F.*\u002FSKILL.md'\n```\n\nExtract skill names from the paths.\n\n## Step 2 — Dataset Existence\n\nFor each modified skill, check whether a test dataset exists:\n\n```\ntests\u002Fpromptfoo\u002Fdatasets\u002F\u003Cskill-name>.yaml\n```\n\nIf the dataset does not exist, create one using `tests\u002Fpromptfoo\u002Fdatasets\u002Freview-sdlc.yaml`\nas the canonical reference for structure and assertion patterns.\n\nEach test case must include:\n- `description` — identifies skill and specific behavior (e.g., `\"commit-sdlc: --auto flag skips confirmation\"`)\n- `vars.skill_path` — path to the SKILL.md being tested\n- `vars.project_context` — path to a fixture file in `tests\u002Fpromptfoo\u002Ffixtures\u002F`\n- `assert` — array of assertions\n\n## Step 3 — Fixture Reference Validity\n\nFor each test case in the dataset, verify:\n\n1. Every `file:\u002F\u002Ffixtures\u002F\u003Cname>.md` reference points to an existing file under `tests\u002Fpromptfoo\u002Ffixtures\u002F`\n2. Every `file:\u002F\u002Ffixtures-fs\u002F\u003Cname>\u002F` reference points to an existing directory under `tests\u002Fpromptfoo\u002Ffixtures-fs\u002F`\n\nReport any broken references.\n\n## Step 4 — New Behavior Coverage\n\nIf the skill gained a new flag (compare `argument-hint` in the SKILL.md before\u002Fafter) or\na new workflow step:\n\n1. Check whether any existing test case exercises the new behavior\n2. If not, suggest adding a test case with:\n   - A descriptive name referencing the new flag\u002Fstep\n   - A fixture that exercises the new code path\n   - Both structural and behavioral assertions\n\n## Step 5 — Assertion Quality\n\nReview each test case's assertions for completeness:\n\n| Required | Type | Purpose |\n|----------|------|---------|\n| At least 1 | `icontains` or `regex` | Structural — verifies specific output strings |\n| At least 1 | `llm-rubric` | Behavioral — verifies semantic correctness |\n\nFlag test cases that have only structural OR only behavioral assertions.\nWell-written `llm-rubric` assertions are specific enough to distinguish pass from fail\n(e.g., \"The commit message follows conventional commits format and references the changed files\"\nnot \"The output looks correct\").\n\n## Step 6 — Report Findings\n\nPresent findings in this format:\n\n| Skill | Check | Status | Detail |\n|-------|-------|--------|--------|\n| `\u003Cname>` | Dataset exists | ✓\u002F✗ | path or \"missing\" |\n| `\u003Cname>` | Fixture refs valid | ✓\u002F✗ | broken refs listed |\n| `\u003Cname>` | New behavior covered | ✓\u002F✗\u002FN\u002FA | uncovered flags\u002Fsteps |\n| `\u003Cname>` | Assertion quality | ✓\u002F✗ | test cases with gaps |\n\n## DO NOT\n\n- Run `promptfoo eval` automatically — evaluation runs must always be initiated by the user\n- Create test cases that test implementation details rather than observable behavior\n- Write assertions that match on volatile output (timestamps, random IDs, file paths with user dirs)\n",{"name":5,"description":7},"main","3480d74fa3b8cb64fff5c365fbeb6e00d2708e53",".claude\u002Fskills\u002Fvalidate-skill-tests",6,"https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace","https:\u002F\u002Fgithub.com\u002Frnagrodzki","http:\u002F\u002Fnagrodzki.net","2026-09-05 21:47:31",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add rnagrodzki\u002Fsdlc-marketplace --skill validate-skill-tests",{"tool":58,"command":59},"bunx","bunx skills add rnagrodzki\u002Fsdlc-marketplace --skill validate-skill-tests",{"tool":61,"command":62},"pnpm","pnpm dlx skills add rnagrodzki\u002Fsdlc-marketplace --skill validate-skill-tests","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Fvalidate-skill-tests\u002FSKILL.md",true,[],[67,79,86,96,105,120],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":74,"publishedAt":39},"01M6CYJDSW8MAAJAM62G7ZPRVB","example-skill","Test fixture — passing case for script-resolution-version rule.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fresolution-validator-cases\u002Fgood-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[73],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[75,76,38,26,77,78],"example skill","example","version","resolution",{"skillNo":80,"name":69,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":85,"publishedAt":39},"01M6CYJCBDNB7S40VZ1B0TEFE1","Test fixture — failing case for script-resolution-version rule.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fresolution-validator-cases\u002Fbad-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[84],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[76,75,26,77,38,78],{"skillNo":87,"name":69,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":92,"publishedAt":39},"01M6CYJARTK87CH0T6EJ7ZQCZV","Test fixture — passing case for the orchestrator-await-barrier rule (#487). Marks a fan-out dispatch as await-barrier-required and states the guard.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Forchestrator-await-barrier-cases\u002Fgood-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[91],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[75,76,93,94,95,38,26],"guard","orchestrator","dispatch",{"skillNo":97,"name":69,"nameZh":6,"description":98,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":99,"categories":100,"keywords":102,"publishedAt":39},"01M6CYJ8T4G8CQDWGZKGW1THNZ","Test fixture — failing case for the orchestrator-await-barrier rule (#487). Marks a fan-out dispatch as await-barrier-required but never states the guard.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Forchestrator-await-barrier-cases\u002Fbad-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[101],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[76,75,26,93,94,103,95,104,38],"never","but",{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":39},"01M6CYJ78NG35A6BGVDG3DTDV8","pr-sdlc","Creates or updates GitHub pull requests with auto-generated 8-section descriptions.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fhealthy-marketplace\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fpr-sdlc\u002FSKILL.md",[],[112,113,114,115,116,117,118,119],"pr sdlc","sdlc","pr","auto","github","pull requests","pull","generated",{"skillNo":121,"name":69,"nameZh":6,"description":122,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":123,"categories":124,"keywords":126,"publishedAt":39},"01M6CYJ5W0AYJTBNDN0W4K6AJZ","Fixture skill for validate-cost-tiers drift tests.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fcost-tiers-drift\u002Fskills\u002Fexample-skill\u002FSKILL.md",[125],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[75,76,24,127,128,129,38],"cost","fixture skill","drift"]