[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CYG1SHF5FZ3CAVGPX274EA":3,"skill-same:01M6CYG1SHF5FZ3CAVGPX274EA":79},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":46,"linkOnly":47,"markdownContent":48,"markdownBody":49,"frontMatter":50,"refName":51,"commitSha":52,"skillDirectory":53,"forksCount":54,"repositoryHtmlUrl":55,"repositoryHomepageUrl":6,"ownerHtmlUrl":56,"ownerBlogUrl":57,"ownerDescription":6,"updatedAt":58,"install":59,"resources":72},"01M6CYG1SHF5FZ3CAVGPX274EA","test-report",null,"Use this skill when analyzing promptfoo test results and generating evidence reports. Consumes pre-computed data from test-report-prepare.js, generates categorized markdown report with failure diagnosis and fix plan. Triggers on: test report, analyze test results, promptfoo report, test evidence, test failures, generate evidence.","rnagrodzki\u002Fsdlc-marketplace","rnagrodzki","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F303481?v=4","01M6CYFWQK3A8WR9W324Z5Q5DR","Rafal Nagrodzki","01M6CYFWVKS23ECZCP9R7DTQVN",0,"MIT","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Ftest-report\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000034","行政与办公人员","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"test report","report","test","plan","reports","evidence","use","fix","results","analyze","js","markdown","promptfoo","data","analyzing","generate","prepare","2026-11-10 10:12:30",false,"---\nname: test-report\ndescription: \"Use this skill when analyzing promptfoo test results and generating evidence reports. Consumes pre-computed data from test-report-prepare.js, generates categorized markdown report with failure diagnosis and fix plan. Triggers on: test report, analyze test results, promptfoo report, test evidence, test failures, generate evidence.\"\n---\n\n# Analyzing Promptfoo Test Results\n\nConsume pre-computed test data from `test-report-prepare.js` and generate a\ndetailed evidence report in `.evidences\u002F` with failure categorization, root\ncause analysis, and a prioritized fix plan.\n\n## When to Use This Skill\n\n- After running `promptfoo eval` and wanting a structured evidence report\n- When asked to analyze test results, failures, or generate a test evidence file\n- When the `\u002Ftest-report` command delegates here after running `test-report-prepare.js`\n\n## Step 0 — Run the Prepare Script\n\n> **VERBATIM** — Run this bash block exactly as written. Do not modify, rephrase, or simplify the commands.\n\n```bash\n# Safe: test-report-prepare.js ships only under .claude\u002Fskills\u002F (repo-level), never under plugins\u002Fsdlc-utilities\u002Fscripts\u002F, so this find never matches a plugin-cache copy and cannot carry #485's contamination.\nSCRIPT=$(find ~\u002F.claude\u002Fplugins -name \"test-report-prepare.js\" 2>\u002Fdev\u002Fnull | sort -V | tail -1)\n[ -z \"$SCRIPT\" ] && [ -f \".claude\u002Fskills\u002Ftest-report\u002Ftest-report-prepare.js\" ] && SCRIPT=\".claude\u002Fskills\u002Ftest-report\u002Ftest-report-prepare.js\"\n[ -z \"$SCRIPT\" ] && { echo \"ERROR: Could not locate test-report-prepare.js. Is the sdlc plugin installed?\" >&2; exit 2; }\n\nDATA_FILE=$(node \"$SCRIPT\" --output-file $ARGUMENTS)\nEXIT_CODE=$?\n```\n\n- If exit code **1**: Read the JSON, surface `errors[]` to the user, stop.\n- If exit code **2**: Report the script crash, stop.\n- If exit code **0**: Read `DATA_FILE` into `REPORT_DATA_JSON`.\n\n**Error-to-GitHub issue proposal**:\n\nFor exit code 2 (script crash), locate the procedure: Glob for `**\u002Ferror-report-sdlc\u002FREFERENCE.md`\nunder `~\u002F.claude\u002Fplugins`, then retry with cwd. If found, follow the procedure with:\n\n- **Skill**: test-report\n- **Step**: Step 0 — test-report-prepare.js execution\n- **Operation**: Running test-report-prepare.js to pre-compute test data\n- **Error**: Exit code 2 — script crash (full error on stderr)\n- **Suggested investigation**: Check Node.js version; inspect stderr for stack trace; verify test-report-prepare.js is accessible via the plugin path\n\nIf not found, skip — the capability is not installed.\n\nClean up `DATA_FILE` after writing the report.\n\n## Step 1 — Consume Context\n\nRead `REPORT_DATA_JSON`. Display a brief orientation:\n\n```\nEval: {evalMeta.evalId} ({evalMeta.createdAt})\nType: {evalMeta.configType}\nPassed: {summary.passed}\u002F{summary.executed} ({summary.passRate}%)\nFailed: {summary.failed} | Missing: {summary.missing}\n```\n\nKey fields reference:\n\n| Field | Description |\n|-------|-------------|\n| `evalMeta.evalId` | Eval run identifier |\n| `evalMeta.configType` | `behavioral` or `exec` |\n| `summary.passRate` | Integer percentage |\n| `passingTests[]` | Tests with `success=1` |\n| `failingTests[]` | Tests with `success=0`; each has `assertions[]`, `responsePreview`, `gradingReason` |\n| `missingTests[]` | Tests defined in datasets but absent from results |\n| `comparison` | null or `{regressions[], improvements[], unchanged}` |\n\n## Step 2 (PLAN) — Categorize Failures and Draft Report\n\nAnalyze `failingTests[]` and classify each into exactly one category:\n\n**Category A — Infrastructure Failures**\n- `latencyMs \u003C 10000` AND (`responsePreview` is empty OR contains \"Command failed\")\n- Root cause: CLI crash, rate limit, concurrent session exhaustion\n\n**Category B — Assertion Failures**\n- LLM produced meaningful output (non-empty `responsePreview`, no \"Command failed\")\n- Specific assertions in `assertions[]` have `pass: false`\n- Sub-classify:\n  - **B-false-positive**: Failing assertion is `not-regex` or `not-icontains` AND response shows correct behavior (e.g., explicitly refuses or negates the thing being checked). Most other assertions passed.\n  - **B-true-failure**: LLM genuinely produced incorrect output — the assertion correctly caught a real behavioral problem.\n\n**Category C — Missing Tests**\n- Items from `missingTests[]`\n\nFor each Category B failure, draft:\n- Failed assertion(s) with type and value\n- Diagnosis (why the assertion fired, whether behavior is correct)\n- Response preview excerpt (from `responsePreview` — do not fabricate)\n- Specific fix suggestion (e.g., change assertion value, narrow regex, fix skill)\n\nDraft the full markdown report using this template:\n\n```markdown\n# Promptfoo {configType|title} Test Report\n\n**Generated:** {current timestamp YYYY-MM-DD HH:MM:SS}\n**Eval ID:** `{evalMeta.evalId}`\n**Config:** `tests\u002Fpromptfoo\u002Fpromptfooconfig{-exec if exec}.yaml`\n\n---\n\n## Summary\n\n| Metric | Value |\n|--------|-------|\n| Tests defined | {summary.totalTests} |\n| Tests executed | {summary.executed} |\n| Tests missing | {summary.missing} |\n| Passed | {summary.passed} ({summary.passRate}%) |\n| Failed — infrastructure | {count of Category A} |\n| Failed — assertion | {count of Category B} |\n| Avg latency | {summary.avgLatencyMs}ms |\n| Total cost | ${summary.totalCost} |\n\n**Overall health: {HEALTHY if passRate >= 90 | WARNING if 70-89 | CRITICAL if \u003C 70} — {summary.passRate}% pass rate**\n\n---\n\n## Passing Tests ({summary.passed}\u002F{summary.executed})\n\n| Test | Skill | Score |\n|------|-------|-------|\n{one row per passing test}\n\n---\n\n## Category A — Infrastructure Failures ({count} tests)\n\n{Description of the common failure pattern and root cause}\n\n| Test | Skill | Latency |\n|------|-------|---------|\n{one row per Cat-A failure}\n\n---\n\n## Category B — Assertion Failures ({count} tests)\n\n### B{n}: {test description}\n- **Score:** {score} ({passed_count}\u002F{total_count} assertions passed)\n- **Failed assertion:** `{type} \"{value}\"`\n- **Diagnosis:** {root cause analysis}\n- **LLM behavior:** Correct\u002FIncorrect — {explanation}\n- **Response preview:**\n  ```\n  {responsePreview, max 400 chars}\n  ```\n- **Fix:** {specific, actionable fix}\n\n---\n\n## Category C — Missing Tests ({count} tests)\n\n| Test | Source |\n|------|--------|\n{one row per missing test}\n\n---\n\n## Failure Attribution\n\n| Root Cause | Tests Affected | Fix |\n|------------|---------------|-----|\n{one row per distinct root cause}\n\n**True behavioral failures:** {count of B-true-failure} — {summary sentence}\n\n---\n\n{IF comparison is not null:}\n## Comparison with Previous Run\n\n**Previous eval:** `{comparison.previousEvalId}` ({comparison.previousCreatedAt})\n\n### Regressions ({count})\n| Test | Previous Score | Current Score |\n|------|---------------|--------------|\n\n### Improvements ({count})\n| Test | Previous Score | Current Score |\n|------|---------------|--------------|\n\n### Unchanged\n- Still passing: {comparison.unchanged.stillPassing}\n- Still failing: {comparison.unchanged.stillFailing}\n{END IF}\n\n---\n\n## Fix Plan\n\n### Priority 1 — Infrastructure ({count} tests affected)\n{numbered action items: config changes, concurrency settings}\n\n### Priority 2 — Assertion Corrections ({count} tests affected)\n{numbered action items with file paths and specific assertion changes}\n\n### Priority 3 — Behavioral Fixes ({count} tests affected)\n{numbered action items with skill paths and what needs to change}\n\n---\n\n## Conclusion\n\n{1-2 paragraph summary of overall health, key takeaways, and recommended next steps}\n```\n\n## Step 3 (CRITIQUE) — Self-Review the Report\n\nCheck these gates before proceeding:\n\n| Gate | Check |\n|------|-------|\n| Completeness | Every `failingTests[]` entry appears in exactly one category |\n| Count accuracy | Category A + B counts + missing = `summary.failed + summary.missing` |\n| Passing count | Passing tests table row count = `summary.passed` |\n| No fabrication | All response previews and assertion details come from `REPORT_DATA_JSON` |\n| Actionability | Every failure has a concrete fix (not \"investigate further\") |\n| Attribution | Every distinct root cause has a row in Failure Attribution table |\n| No DB queries | Skill never runs `sqlite3` — all data from script |\n\nList any issues found.\n\n## Step 4 (IMPROVE) — Revise the Report\n\nFix all issues from Step 3. Repeat critique\u002Fimprove up to 2 times until all gates pass.\n\n## Step 5 (DO) — Write the Report\n\n1. Create `.evidences\u002F` directory if it does not exist:\n   ```bash\n   mkdir -p .evidences\n   ```\n\n2. Generate filename using current timestamp:\n   ```bash\n   TIMESTAMP=$(date '+%Y-%m-%d_%H-%M-%S')\n   REPORT_FILE=\".evidences\u002Fpromptfoo-${CONFIG_TYPE}-report-${TIMESTAMP}.md\"\n   ```\n\n3. Write the final report to `REPORT_FILE`.\n\n4. Clean up the temp data file (`DATA_FILE`).\n\n## Step 6 (CRITIQUE) — Self-Review the Fix Plan\n\nOnly if `summary.failed > 0`. Check:\n\n| Gate | Check |\n|------|-------|\n| Ordering | Infrastructure fixes listed before assertion fixes before skill fixes |\n| Specificity | Each action references a specific file path and\u002For assertion value |\n| No eval runs | Does not suggest running `promptfoo eval` automatically |\n| Completeness | All root causes from Attribution table have a fix action item |\n\nList any issues.\n\n## Step 7 (IMPROVE) — Revise the Fix Plan\n\nFix all issues from Step 6. Up to 2 iterations.\n\n## Step 8 — Present Summary to User\n\nShow:\n```\nReport written to: {REPORT_FILE}\n\nHealth: {HEALTHY|WARNING|CRITICAL} — {passRate}%\n  Passed:   {summary.passed}\u002F{summary.executed}\n  Category A (infra):    {count}\n  Category B (assert):   {count}\n  Category C (missing):  {count}\n\nTop fix priorities:\n  1. {first priority action}\n  2. {second priority action}\n  3. {third priority action}\n\n{IF --compare: \"Regressions: {count} | Improvements: {count}\"}\n```\n\n## Error Recovery\n\n> **Flow**: detect → diagnose → auto-recover (retry once if transient) → invoke `error-report-sdlc` for persistent actionable failures.\n\n| Error | Recovery | Invoke error-report-sdlc? |\n|-------|----------|---------------------------|\n| `test-report-prepare.js` exit 1 | Show `errors[]`, stop | No — user input error |\n| `test-report-prepare.js` exit 2 (crash) | Show stderr, stop | Yes |\n| Zero test results (empty eval) | List available evals; ask user to verify | No — expected (empty run) |\n| `.evidences\u002F` directory not writable | Show error; ask user to check permissions | No — environment issue |\n\nWhen invoking `error-report-sdlc`, provide:\n- **Skill**: test-report\n- **Step**: Step 0 — test-report-prepare.js execution\n- **Operation**: Running test-report-prepare.js to pre-compute test report data\n- **Error**: Exit code 2 — script crash (full error on stderr)\n- **Suggested investigation**: Check Node.js version; inspect stderr for stack trace; verify promptfoo DB exists at `tests\u002Fpromptfoo\u002F.promptfoo-data\u002Fpromptfoo.db`\n\n---\n\n## DO NOT\n\n- Run `sqlite3` queries — all data comes from the prepare script JSON\n- Run `promptfoo eval` or suggest running it automatically\n- Fabricate response content not present in `REPORT_DATA_JSON`\n- Write to any file other than the `.evidences\u002F` report\n- Skip the plan-critique-improve cycle\n\n## Learning Capture\n\nIf analysis reveals patterns (e.g., \"Cat-A failures correlate with concurrency > N\",\n\"specific skill consistently produces false-positive assertion failures\"), append\nentries to `.sdlc\u002Flearnings\u002Flog.md`.\n","# Analyzing Promptfoo Test Results\n\nConsume pre-computed test data from `test-report-prepare.js` and generate a\ndetailed evidence report in `.evidences\u002F` with failure categorization, root\ncause analysis, and a prioritized fix plan.\n\n## When to Use This Skill\n\n- After running `promptfoo eval` and wanting a structured evidence report\n- When asked to analyze test results, failures, or generate a test evidence file\n- When the `\u002Ftest-report` command delegates here after running `test-report-prepare.js`\n\n## Step 0 — Run the Prepare Script\n\n> **VERBATIM** — Run this bash block exactly as written. Do not modify, rephrase, or simplify the commands.\n\n```bash\n# Safe: test-report-prepare.js ships only under .claude\u002Fskills\u002F (repo-level), never under plugins\u002Fsdlc-utilities\u002Fscripts\u002F, so this find never matches a plugin-cache copy and cannot carry #485's contamination.\nSCRIPT=$(find ~\u002F.claude\u002Fplugins -name \"test-report-prepare.js\" 2>\u002Fdev\u002Fnull | sort -V | tail -1)\n[ -z \"$SCRIPT\" ] && [ -f \".claude\u002Fskills\u002Ftest-report\u002Ftest-report-prepare.js\" ] && SCRIPT=\".claude\u002Fskills\u002Ftest-report\u002Ftest-report-prepare.js\"\n[ -z \"$SCRIPT\" ] && { echo \"ERROR: Could not locate test-report-prepare.js. Is the sdlc plugin installed?\" >&2; exit 2; }\n\nDATA_FILE=$(node \"$SCRIPT\" --output-file $ARGUMENTS)\nEXIT_CODE=$?\n```\n\n- If exit code **1**: Read the JSON, surface `errors[]` to the user, stop.\n- If exit code **2**: Report the script crash, stop.\n- If exit code **0**: Read `DATA_FILE` into `REPORT_DATA_JSON`.\n\n**Error-to-GitHub issue proposal**:\n\nFor exit code 2 (script crash), locate the procedure: Glob for `**\u002Ferror-report-sdlc\u002FREFERENCE.md`\nunder `~\u002F.claude\u002Fplugins`, then retry with cwd. If found, follow the procedure with:\n\n- **Skill**: test-report\n- **Step**: Step 0 — test-report-prepare.js execution\n- **Operation**: Running test-report-prepare.js to pre-compute test data\n- **Error**: Exit code 2 — script crash (full error on stderr)\n- **Suggested investigation**: Check Node.js version; inspect stderr for stack trace; verify test-report-prepare.js is accessible via the plugin path\n\nIf not found, skip — the capability is not installed.\n\nClean up `DATA_FILE` after writing the report.\n\n## Step 1 — Consume Context\n\nRead `REPORT_DATA_JSON`. Display a brief orientation:\n\n```\nEval: {evalMeta.evalId} ({evalMeta.createdAt})\nType: {evalMeta.configType}\nPassed: {summary.passed}\u002F{summary.executed} ({summary.passRate}%)\nFailed: {summary.failed} | Missing: {summary.missing}\n```\n\nKey fields reference:\n\n| Field | Description |\n|-------|-------------|\n| `evalMeta.evalId` | Eval run identifier |\n| `evalMeta.configType` | `behavioral` or `exec` |\n| `summary.passRate` | Integer percentage |\n| `passingTests[]` | Tests with `success=1` |\n| `failingTests[]` | Tests with `success=0`; each has `assertions[]`, `responsePreview`, `gradingReason` |\n| `missingTests[]` | Tests defined in datasets but absent from results |\n| `comparison` | null or `{regressions[], improvements[], unchanged}` |\n\n## Step 2 (PLAN) — Categorize Failures and Draft Report\n\nAnalyze `failingTests[]` and classify each into exactly one category:\n\n**Category A — Infrastructure Failures**\n- `latencyMs \u003C 10000` AND (`responsePreview` is empty OR contains \"Command failed\")\n- Root cause: CLI crash, rate limit, concurrent session exhaustion\n\n**Category B — Assertion Failures**\n- LLM produced meaningful output (non-empty `responsePreview`, no \"Command failed\")\n- Specific assertions in `assertions[]` have `pass: false`\n- Sub-classify:\n  - **B-false-positive**: Failing assertion is `not-regex` or `not-icontains` AND response shows correct behavior (e.g., explicitly refuses or negates the thing being checked). Most other assertions passed.\n  - **B-true-failure**: LLM genuinely produced incorrect output — the assertion correctly caught a real behavioral problem.\n\n**Category C — Missing Tests**\n- Items from `missingTests[]`\n\nFor each Category B failure, draft:\n- Failed assertion(s) with type and value\n- Diagnosis (why the assertion fired, whether behavior is correct)\n- Response preview excerpt (from `responsePreview` — do not fabricate)\n- Specific fix suggestion (e.g., change assertion value, narrow regex, fix skill)\n\nDraft the full markdown report using this template:\n\n```markdown\n# Promptfoo {configType|title} Test Report\n\n**Generated:** {current timestamp YYYY-MM-DD HH:MM:SS}\n**Eval ID:** `{evalMeta.evalId}`\n**Config:** `tests\u002Fpromptfoo\u002Fpromptfooconfig{-exec if exec}.yaml`\n\n---\n\n## Summary\n\n| Metric | Value |\n|--------|-------|\n| Tests defined | {summary.totalTests} |\n| Tests executed | {summary.executed} |\n| Tests missing | {summary.missing} |\n| Passed | {summary.passed} ({summary.passRate}%) |\n| Failed — infrastructure | {count of Category A} |\n| Failed — assertion | {count of Category B} |\n| Avg latency | {summary.avgLatencyMs}ms |\n| Total cost | ${summary.totalCost} |\n\n**Overall health: {HEALTHY if passRate >= 90 | WARNING if 70-89 | CRITICAL if \u003C 70} — {summary.passRate}% pass rate**\n\n---\n\n## Passing Tests ({summary.passed}\u002F{summary.executed})\n\n| Test | Skill | Score |\n|------|-------|-------|\n{one row per passing test}\n\n---\n\n## Category A — Infrastructure Failures ({count} tests)\n\n{Description of the common failure pattern and root cause}\n\n| Test | Skill | Latency |\n|------|-------|---------|\n{one row per Cat-A failure}\n\n---\n\n## Category B — Assertion Failures ({count} tests)\n\n### B{n}: {test description}\n- **Score:** {score} ({passed_count}\u002F{total_count} assertions passed)\n- **Failed assertion:** `{type} \"{value}\"`\n- **Diagnosis:** {root cause analysis}\n- **LLM behavior:** Correct\u002FIncorrect — {explanation}\n- **Response preview:**\n  ```\n  {responsePreview, max 400 chars}\n  ```\n- **Fix:** {specific, actionable fix}\n\n---\n\n## Category C — Missing Tests ({count} tests)\n\n| Test | Source |\n|------|--------|\n{one row per missing test}\n\n---\n\n## Failure Attribution\n\n| Root Cause | Tests Affected | Fix |\n|------------|---------------|-----|\n{one row per distinct root cause}\n\n**True behavioral failures:** {count of B-true-failure} — {summary sentence}\n\n---\n\n{IF comparison is not null:}\n## Comparison with Previous Run\n\n**Previous eval:** `{comparison.previousEvalId}` ({comparison.previousCreatedAt})\n\n### Regressions ({count})\n| Test | Previous Score | Current Score |\n|------|---------------|--------------|\n\n### Improvements ({count})\n| Test | Previous Score | Current Score |\n|------|---------------|--------------|\n\n### Unchanged\n- Still passing: {comparison.unchanged.stillPassing}\n- Still failing: {comparison.unchanged.stillFailing}\n{END IF}\n\n---\n\n## Fix Plan\n\n### Priority 1 — Infrastructure ({count} tests affected)\n{numbered action items: config changes, concurrency settings}\n\n### Priority 2 — Assertion Corrections ({count} tests affected)\n{numbered action items with file paths and specific assertion changes}\n\n### Priority 3 — Behavioral Fixes ({count} tests affected)\n{numbered action items with skill paths and what needs to change}\n\n---\n\n## Conclusion\n\n{1-2 paragraph summary of overall health, key takeaways, and recommended next steps}\n```\n\n## Step 3 (CRITIQUE) — Self-Review the Report\n\nCheck these gates before proceeding:\n\n| Gate | Check |\n|------|-------|\n| Completeness | Every `failingTests[]` entry appears in exactly one category |\n| Count accuracy | Category A + B counts + missing = `summary.failed + summary.missing` |\n| Passing count | Passing tests table row count = `summary.passed` |\n| No fabrication | All response previews and assertion details come from `REPORT_DATA_JSON` |\n| Actionability | Every failure has a concrete fix (not \"investigate further\") |\n| Attribution | Every distinct root cause has a row in Failure Attribution table |\n| No DB queries | Skill never runs `sqlite3` — all data from script |\n\nList any issues found.\n\n## Step 4 (IMPROVE) — Revise the Report\n\nFix all issues from Step 3. Repeat critique\u002Fimprove up to 2 times until all gates pass.\n\n## Step 5 (DO) — Write the Report\n\n1. Create `.evidences\u002F` directory if it does not exist:\n   ```bash\n   mkdir -p .evidences\n   ```\n\n2. Generate filename using current timestamp:\n   ```bash\n   TIMESTAMP=$(date '+%Y-%m-%d_%H-%M-%S')\n   REPORT_FILE=\".evidences\u002Fpromptfoo-${CONFIG_TYPE}-report-${TIMESTAMP}.md\"\n   ```\n\n3. Write the final report to `REPORT_FILE`.\n\n4. Clean up the temp data file (`DATA_FILE`).\n\n## Step 6 (CRITIQUE) — Self-Review the Fix Plan\n\nOnly if `summary.failed > 0`. Check:\n\n| Gate | Check |\n|------|-------|\n| Ordering | Infrastructure fixes listed before assertion fixes before skill fixes |\n| Specificity | Each action references a specific file path and\u002For assertion value |\n| No eval runs | Does not suggest running `promptfoo eval` automatically |\n| Completeness | All root causes from Attribution table have a fix action item |\n\nList any issues.\n\n## Step 7 (IMPROVE) — Revise the Fix Plan\n\nFix all issues from Step 6. Up to 2 iterations.\n\n## Step 8 — Present Summary to User\n\nShow:\n```\nReport written to: {REPORT_FILE}\n\nHealth: {HEALTHY|WARNING|CRITICAL} — {passRate}%\n  Passed:   {summary.passed}\u002F{summary.executed}\n  Category A (infra):    {count}\n  Category B (assert):   {count}\n  Category C (missing):  {count}\n\nTop fix priorities:\n  1. {first priority action}\n  2. {second priority action}\n  3. {third priority action}\n\n{IF --compare: \"Regressions: {count} | Improvements: {count}\"}\n```\n\n## Error Recovery\n\n> **Flow**: detect → diagnose → auto-recover (retry once if transient) → invoke `error-report-sdlc` for persistent actionable failures.\n\n| Error | Recovery | Invoke error-report-sdlc? |\n|-------|----------|---------------------------|\n| `test-report-prepare.js` exit 1 | Show `errors[]`, stop | No — user input error |\n| `test-report-prepare.js` exit 2 (crash) | Show stderr, stop | Yes |\n| Zero test results (empty eval) | List available evals; ask user to verify | No — expected (empty run) |\n| `.evidences\u002F` directory not writable | Show error; ask user to check permissions | No — environment issue |\n\nWhen invoking `error-report-sdlc`, provide:\n- **Skill**: test-report\n- **Step**: Step 0 — test-report-prepare.js execution\n- **Operation**: Running test-report-prepare.js to pre-compute test report data\n- **Error**: Exit code 2 — script crash (full error on stderr)\n- **Suggested investigation**: Check Node.js version; inspect stderr for stack trace; verify promptfoo DB exists at `tests\u002Fpromptfoo\u002F.promptfoo-data\u002Fpromptfoo.db`\n\n---\n\n## DO NOT\n\n- Run `sqlite3` queries — all data comes from the prepare script JSON\n- Run `promptfoo eval` or suggest running it automatically\n- Fabricate response content not present in `REPORT_DATA_JSON`\n- Write to any file other than the `.evidences\u002F` report\n- Skip the plan-critique-improve cycle\n\n## Learning Capture\n\nIf analysis reveals patterns (e.g., \"Cat-A failures correlate with concurrency > N\",\n\"specific skill consistently produces false-positive assertion failures\"), append\nentries to `.sdlc\u002Flearnings\u002Flog.md`.\n",{"name":5,"description":7},"main","3480d74fa3b8cb64fff5c365fbeb6e00d2708e53",".claude\u002Fskills\u002Ftest-report",6,"https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace","https:\u002F\u002Fgithub.com\u002Frnagrodzki","http:\u002F\u002Fnagrodzki.net","2026-09-05 21:47:31",{"source":8,"skillName":5,"commands":60,"prompt":70,"zipAvailable":71},[61,64,67],{"tool":62,"command":63},"npx","npx skills add rnagrodzki\u002Fsdlc-marketplace --skill test-report",{"tool":65,"command":66},"bunx","bunx skills add rnagrodzki\u002Fsdlc-marketplace --skill test-report",{"tool":68,"command":69},"pnpm","pnpm dlx skills add rnagrodzki\u002Fsdlc-marketplace --skill test-report","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Ftest-report\u002FSKILL.md",true,[73],{"path":74,"resourceType":75,"resourceTypeDesc":76,"sizeBytes":77,"sourceHtmlUrl":78},"test-report-prepare.js","SCRIPT","脚本",15577,"https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002F.claude\u002Fskills\u002Ftest-report\u002Ftest-report-prepare.js",[80,95,102,112,121,136],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":89,"publishedAt":46},"01M6CYJDSW8MAAJAM62G7ZPRVB","example-skill","Test fixture — passing case for script-resolution-version rule.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fresolution-validator-cases\u002Fgood-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[86],{"categoryNo":87,"categoryName":88,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[90,91,92,31,93,94],"example skill","example","fixture","version","resolution",{"skillNo":96,"name":82,"nameZh":6,"description":97,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":101,"publishedAt":46},"01M6CYJCBDNB7S40VZ1B0TEFE1","Test fixture — failing case for script-resolution-version rule.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fresolution-validator-cases\u002Fbad-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[100],{"categoryNo":87,"categoryName":88,"taxonomyType":21,"taxonomyTypeDesc":22},[91,90,31,93,92,94],{"skillNo":103,"name":82,"nameZh":6,"description":104,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":105,"categories":106,"keywords":108,"publishedAt":46},"01M6CYJARTK87CH0T6EJ7ZQCZV","Test fixture — passing case for the orchestrator-await-barrier rule (#487). Marks a fan-out dispatch as await-barrier-required and states the guard.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Forchestrator-await-barrier-cases\u002Fgood-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[107],{"categoryNo":87,"categoryName":88,"taxonomyType":21,"taxonomyTypeDesc":22},[90,91,109,110,111,92,31],"guard","orchestrator","dispatch",{"skillNo":113,"name":82,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":118,"publishedAt":46},"01M6CYJ8T4G8CQDWGZKGW1THNZ","Test fixture — failing case for the orchestrator-await-barrier rule (#487). Marks a fan-out dispatch as await-barrier-required but never states the guard.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Forchestrator-await-barrier-cases\u002Fbad-project\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fexample-skill\u002FSKILL.md",[117],{"categoryNo":87,"categoryName":88,"taxonomyType":21,"taxonomyTypeDesc":22},[91,90,31,109,110,119,111,120,92],"never","but",{"skillNo":122,"name":123,"nameZh":6,"description":124,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":125,"categories":126,"keywords":127,"publishedAt":46},"01M6CYJ78NG35A6BGVDG3DTDV8","pr-sdlc","Creates or updates GitHub pull requests with auto-generated 8-section descriptions.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fhealthy-marketplace\u002Fplugins\u002Fsdlc-utilities\u002Fskills\u002Fpr-sdlc\u002FSKILL.md",[],[128,129,130,131,132,133,134,135],"pr sdlc","sdlc","pr","auto","github","pull requests","pull","generated",{"skillNo":137,"name":82,"nameZh":6,"description":138,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":139,"categories":140,"keywords":142,"publishedAt":46},"01M6CYJ5W0AYJTBNDN0W4K6AJZ","Fixture skill for validate-cost-tiers drift tests.","https:\u002F\u002Fgithub.com\u002Frnagrodzki\u002Fsdlc-marketplace\u002Fblob\u002Fmain\u002Ftests\u002Fpromptfoo\u002Ffixtures-fs\u002Fcost-tiers-drift\u002Fskills\u002Fexample-skill\u002FSKILL.md",[141],{"categoryNo":87,"categoryName":88,"taxonomyType":21,"taxonomyTypeDesc":22},[90,91,143,144,145,146,92],"validate","cost","fixture skill","drift"]