[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6NTPT0FW1SJQPV58F0S6M9D":3,"skill-same:01M6NTPT0FW1SJQPV58F0S6M9D":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":14,"repositoryHtmlUrl":47,"repositoryHomepageUrl":6,"ownerHtmlUrl":48,"ownerBlogUrl":49,"ownerDescription":50,"updatedAt":51,"install":52,"resources":65},"01M6NTPT0FW1SJQPV58F0S6M9D","eval-harness",null,"Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles","rubicanjr\u002FFinCognis","rubicanjr","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F234912519?v=4","01M6NT156NND8J474G34SB3923","Rubi","01M6NT15B70RWXE1N3YV0BZY8A",0,"MIT","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Feval-harness\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"harness","eval harness","eval","for","sessions","implementing","principles","driven","edit","write","read","grep","Bash","claude code sessions","Evaluation","framework","code","development","evaluation framework","2026-11-10 10:12:30",false,"---\nname: eval-harness\ndescription: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n","# Eval Harness Skill\n\nA formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.\n\n## Philosophy\n\nEval-Driven Development treats evals as the \"unit tests of AI development\":\n- Define expected behavior BEFORE implementation\n- Run evals continuously during development\n- Track regressions with each change\n- Use pass@k metrics for reliability measurement\n\n## Eval Types\n\n### Capability Evals\nTest if Claude can do something it couldn't before:\n```markdown\n[CAPABILITY EVAL: feature-name]\nTask: Description of what Claude should accomplish\nSuccess Criteria:\n  - [ ] Criterion 1\n  - [ ] Criterion 2\n  - [ ] Criterion 3\nExpected Output: Description of expected result\n```\n\n### Regression Evals\nEnsure changes don't break existing functionality:\n```markdown\n[REGRESSION EVAL: feature-name]\nBaseline: SHA or checkpoint name\nTests:\n  - existing-test-1: PASS\u002FFAIL\n  - existing-test-2: PASS\u002FFAIL\n  - existing-test-3: PASS\u002FFAIL\nResult: X\u002FY passed (previously Y\u002FY)\n```\n\n## Grader Types\n\n### 1. Code-Based Grader\nDeterministic checks using code:\n```bash\n# Check if file contains expected pattern\ngrep -q \"export function handleAuth\" src\u002Fauth.ts && echo \"PASS\" || echo \"FAIL\"\n\n# Check if tests pass\nnpm test -- --testPathPattern=\"auth\" && echo \"PASS\" || echo \"FAIL\"\n\n# Check if build succeeds\nnpm run build && echo \"PASS\" || echo \"FAIL\"\n```\n\n### 2. Model-Based Grader\nUse Claude to evaluate open-ended outputs:\n```markdown\n[MODEL GRADER PROMPT]\nEvaluate the following code change:\n1. Does it solve the stated problem?\n2. Is it well-structured?\n3. Are edge cases handled?\n4. Is error handling appropriate?\n\nScore: 1-5 (1=poor, 5=excellent)\nReasoning: [explanation]\n```\n\n### 3. Human Grader\nFlag for manual review:\n```markdown\n[HUMAN REVIEW REQUIRED]\nChange: Description of what changed\nReason: Why human review is needed\nRisk Level: LOW\u002FMEDIUM\u002FHIGH\n```\n\n## Metrics\n\n### pass@k\n\"At least one success in k attempts\"\n- pass@1: First attempt success rate\n- pass@3: Success within 3 attempts\n- Typical target: pass@3 > 90%\n\n### pass^k\n\"All k trials succeed\"\n- Higher bar for reliability\n- pass^3: 3 consecutive successes\n- Use for critical paths\n\n## Eval Workflow\n\n### 1. Define (Before Coding)\n```markdown\n## EVAL DEFINITION: feature-xyz\n\n### Capability Evals\n1. Can create new user account\n2. Can validate email format\n3. Can hash password securely\n\n### Regression Evals\n1. Existing login still works\n2. Session management unchanged\n3. Logout flow intact\n\n### Success Metrics\n- pass@3 > 90% for capability evals\n- pass^3 = 100% for regression evals\n```\n\n### 2. Implement\nWrite code to pass the defined evals.\n\n### 3. Evaluate\n```bash\n# Run capability evals\n[Run each capability eval, record PASS\u002FFAIL]\n\n# Run regression evals\nnpm test -- --testPathPattern=\"existing\"\n\n# Generate report\n```\n\n### 4. Report\n```markdown\nEVAL REPORT: feature-xyz\n========================\n\nCapability Evals:\n  create-user:     PASS (pass@1)\n  validate-email:  PASS (pass@2)\n  hash-password:   PASS (pass@1)\n  Overall:         3\u002F3 passed\n\nRegression Evals:\n  login-flow:      PASS\n  session-mgmt:    PASS\n  logout-flow:     PASS\n  Overall:         3\u002F3 passed\n\nMetrics:\n  pass@1: 67% (2\u002F3)\n  pass@3: 100% (3\u002F3)\n\nStatus: READY FOR REVIEW\n```\n\n## Integration Patterns\n\n### Pre-Implementation\n```\n\u002Feval define feature-name\n```\nCreates eval definition file at `.claude\u002Fevals\u002Ffeature-name.md`\n\n### During Implementation\n```\n\u002Feval check feature-name\n```\nRuns current evals and reports status\n\n### Post-Implementation\n```\n\u002Feval report feature-name\n```\nGenerates full eval report\n\n## Eval Storage\n\nStore evals in project:\n```\n.claude\u002F\n  evals\u002F\n    feature-xyz.md      # Eval definition\n    feature-xyz.log     # Eval run history\n    baseline.json       # Regression baselines\n```\n\n## Best Practices\n\n1. **Define evals BEFORE coding** - Forces clear thinking about success criteria\n2. **Run evals frequently** - Catch regressions early\n3. **Track pass@k over time** - Monitor reliability trends\n4. **Use code graders when possible** - Deterministic > probabilistic\n5. **Human review for security** - Never fully automate security checks\n6. **Keep evals fast** - Slow evals don't get run\n7. **Version evals with code** - Evals are first-class artifacts\n\n## Example: Adding Authentication\n\n```markdown\n## EVAL: add-authentication\n\n### Phase 1: Define (10 min)\nCapability Evals:\n- [ ] User can register with email\u002Fpassword\n- [ ] User can login with valid credentials\n- [ ] Invalid credentials rejected with proper error\n- [ ] Sessions persist across page reloads\n- [ ] Logout clears session\n\nRegression Evals:\n- [ ] Public routes still accessible\n- [ ] API responses unchanged\n- [ ] Database schema compatible\n\n### Phase 2: Implement (varies)\n[Write code]\n\n### Phase 3: Evaluate\nRun: \u002Feval check add-authentication\n\n### Phase 4: Report\nEVAL REPORT: add-authentication\n==============================\nCapability: 5\u002F5 passed (pass@3: 100%)\nRegression: 3\u002F3 passed (pass^3: 100%)\nStatus: SHIP IT\n```\n",{"name":5,"tools":43,"description":7},"Read, Write, Edit, Bash, Grep, Glob","main","70e1c350500ee750987420bbae48837afa906b7e","skills\u002Feval-harness","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis","https:\u002F\u002Fgithub.com\u002Frubicanjr","linktr.ee\u002Ffinanskodu","Industrial Engineer & CIO @FinansKodu. Architecting AI-driven financial agents and autonomous workflows. 🏗️ Building systems, not just prompts.","2026-06-09 13:50:15",{"source":8,"skillName":5,"commands":53,"prompt":63,"zipAvailable":64},[54,57,60],{"tool":55,"command":56},"npx","npx skills add rubicanjr\u002FFinCognis --skill eval-harness",{"tool":58,"command":59},"bunx","bunx skills add rubicanjr\u002FFinCognis --skill eval-harness",{"tool":61,"command":62},"pnpm","pnpm dlx skills add rubicanjr\u002FFinCognis --skill eval-harness","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Feval-harness\u002FSKILL.md",true,[],[67,88,97,117,144,163],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":78,"publishedAt":38},"01M6NWS9DAXJTADT4T5MB49X8W","workflow-router","Goal-based workflow orchestration - routes tasks to specialist agents based on user goals","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fworkflow-router\u002FSKILL.md",[73],{"categoryNo":74,"categoryName":75,"taxonomyType":76,"taxonomyTypeDesc":77},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[79,80,81,82,83,84,85,86,87],"workflow router","workflow","router","orchestration","goals","goal","user","workflow orchestration","tasks",{"skillNo":89,"name":90,"nameZh":6,"description":91,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":38},"01M6NWS3RETSC0SHP16HP66E14","wiring","Wiring Verification","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fwiring\u002FSKILL.md",[],[90,95,96],"verification","false",{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":106,"publishedAt":38},"01M6NWS0XW9CVQZBDA1EGW0TWC","websocket-patterns","Connection management, room patterns, reconnection strategies, message buffering, and binary protocol design.","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fwebsocket-patterns\u002Fskill.md",[103],{"categoryNo":104,"categoryName":105,"taxonomyType":76,"taxonomyTypeDesc":77},"01JBSEEDCAT000000000000015","视觉设计",[107,108,109,110,111,112,113,114,115,116],"websocket","patterns","websocket patterns","design","protocol","management","message","strategies","binary","connection",{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":127,"publishedAt":38},"01M6NWRDQDX9ZNETAWRRMQWV52","visual-verdict","Screenshot comparison QA for frontend development. Takes a screenshot of the current implementation, scores it across multiple visual dimensions, and returns a structured PASS\u002FREVISE\u002FFAIL verdict with concrete fixes. Use when implementing UI from a design reference or verifying visual correctness.","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fvisual-verdict\u002FSKILL.md",[123,126],{"categoryNo":124,"categoryName":125,"taxonomyType":76,"taxonomyTypeDesc":77},"01JBSEEDCAT000000000000009","前端开发",{"categoryNo":74,"categoryName":75,"taxonomyType":76,"taxonomyTypeDesc":77},[128,129,130,131,132,133,134,135,24,22,136,137,138,110,36,139,140,141,142,143],"visual","verdict","visual verdict","ui","across","revise","comparison","correctness","fixes","dimensions","frontend","qa","screenshot","returns","use","frontend development",{"skillNo":145,"name":146,"nameZh":6,"description":147,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":148,"categories":149,"keywords":150,"publishedAt":38},"01M6NWR8HSNRS7MNDK2J02QYMN","verification-loop","Comprehensive verification system covering build, types, lint, tests, security, and diff review before a PR.","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fverification-loop\u002FSKILL.md",[],[151,152,95,153,154,155,156,157,158,159,160,161,162],"verification loop","loop","lint","review","security","pr","system","diff","build","tests","diff review","types",{"skillNo":164,"name":165,"nameZh":6,"description":166,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":167,"categories":168,"keywords":169,"publishedAt":38},"01M6NWR4HHSFR0VR5EXFRWA747","vector-db-patterns","Embedding strategies, ANN algorithms, hybrid search, RAG chunking strategies, and reranking for semantic search and retrieval.","https:\u002F\u002Fgithub.com\u002Frubicanjr\u002FFinCognis\u002Fblob\u002Fmain\u002Fskills\u002Fvector-db-patterns\u002Fskill.md",[],[170,171,172,173,108,174,175,176,177,114,178,179,180,181,182,22,183,184],"vector db patterns","vector db","db","vector","semantic search","search","embedding strategies","algorithms","chunking","semantic","retrieval","embedding","Hybrid Search","hybrid","rag"]