[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6Q8GC3DX1AR1Y0897KTRBGG":3,"skill-same:01M6Q8GC3DX1AR1Y0897KTRBGG":75},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":52,"commitSha":53,"skillDirectory":54,"forksCount":14,"repositoryHtmlUrl":55,"repositoryHomepageUrl":56,"ownerHtmlUrl":57,"ownerBlogUrl":58,"ownerDescription":59,"updatedAt":60,"install":61,"resources":74},"01M6Q8GC3DX1AR1Y0897KTRBGG","agent-eval",null,"Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics. Use when choosing between coding agents, or when a change to an agent setup needs measured pass rate, cost, and time rather than an impression.","Tashima-Tarsh\u002FDisha6.6","Tashima-Tarsh","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F269906498?v=4","01M6PBH98X03XERH0HFT4PFB1P","NITISH KUMAR","01M6PBH9DA0M6KM3Z12P848F0K",0,"MIT","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fagent-eval\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"eval","agent eval","use","rate","custom","consistency","with","comparison","aider","time","cost","change","edit","write","read","coding","Bash","metrics","code","tasks","2026-11-10 10:12:30",false,"---\nname: agent-eval\ndescription: Head-to-head comparison of coding agents (Claude Code, Aider, Codex, etc.) on custom tasks with pass rate, cost, time, and consistency metrics. Use when choosing between coding agents, or when a change to an agent setup needs measured pass rate, cost, and time rather than an impression.\nlicense: MIT\nmetadata:\n  origin: ECC\ntools: Read, Write, Edit, Bash, Grep, Glob\n---\n\n# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n","# Agent Eval Skill\n\nA lightweight CLI tool for comparing coding agents head-to-head on reproducible tasks. Every \"which coding agent is best?\" comparison runs on vibes — this tool systematizes it.\n\n## When to Activate\n\n- Comparing coding agents (Claude Code, Aider, Codex, etc.) on your own codebase\n- Measuring agent performance before adopting a new tool or model\n- Running regression checks when an agent updates its model or tooling\n- Producing data-backed agent selection decisions for a team\n\n## Installation\n\n> **Note:** Install agent-eval from its repository after reviewing the source.\n\n## Core Concepts\n\n### YAML Task Definitions\n\nDefine tasks declaratively. Each task specifies what to do, which files to touch, and how to judge success:\n\n```yaml\nname: add-retry-logic\ndescription: Add exponential backoff retry to the HTTP client\nrepo: .\u002Fmy-project\nfiles:\n  - src\u002Fhttp_client.py\nprompt: |\n  Add retry logic with exponential backoff to all HTTP requests.\n  Max 3 retries. Initial delay 1s, max delay 30s.\njudge:\n  - type: pytest\n    command: pytest tests\u002Ftest_http_client.py -v\n  - type: grep\n    pattern: \"exponential_backoff|retry\"\n    files: src\u002Fhttp_client.py\ncommit: \"abc1234\"  # pin to specific commit for reproducibility\n```\n\n### Git Worktree Isolation\n\nEach agent run gets its own git worktree — no Docker required. This provides reproducibility isolation so agents cannot interfere with each other or corrupt the base repo.\n\n### Metrics Collected\n\n| Metric | What It Measures |\n|--------|-----------------|\n| Pass rate | Did the agent produce code that passes the judge? |\n| Cost | API spend per task (when available) |\n| Time | Wall-clock seconds to completion |\n| Consistency | Pass rate across repeated runs (e.g., 3\u002F3 = 100%) |\n\n## Workflow\n\n### 1. Define Tasks\n\nCreate a `tasks\u002F` directory with YAML files, one per task:\n\n```bash\nmkdir tasks\n# Write task definitions (see template above)\n```\n\n### 2. Run Agents\n\nExecute agents against your tasks:\n\n```bash\nagent-eval run --task tasks\u002Fadd-retry-logic.yaml --agent claude-code --agent aider --runs 3\n```\n\nEach run:\n1. Creates a fresh git worktree from the specified commit\n2. Hands the prompt to the agent\n3. Runs the judge criteria\n4. Records pass\u002Ffail, cost, and time\n\n### 3. Compare Results\n\nGenerate a comparison report:\n\n```bash\nagent-eval report --format table\n```\n\n```\nTask: add-retry-logic (3 runs each)\n┌──────────────┬───────────┬────────┬────────┬─────────────┐\n│ Agent        │ Pass Rate │ Cost   │ Time   │ Consistency │\n├──────────────┼───────────┼────────┼────────┼─────────────┤\n│ claude-code  │ 3\u002F3       │ $0.12  │ 45s    │ 100%        │\n│ aider        │ 2\u002F3       │ $0.08  │ 38s    │  67%        │\n└──────────────┴───────────┴────────┴────────┴─────────────┘\n```\n\n## Judge Types\n\n### Code-Based (deterministic)\n\n```yaml\njudge:\n  - type: pytest\n    command: pytest tests\u002F -v\n  - type: command\n    command: npm run build\n```\n\n### Pattern-Based\n\n```yaml\njudge:\n  - type: grep\n    pattern: \"class.*Retry\"\n    files: src\u002F**\u002F*.py\n```\n\n### Model-Based (LLM-as-judge)\n\n```yaml\njudge:\n  - type: llm\n    prompt: |\n      Does this implementation correctly handle exponential backoff?\n      Check for: max retries, increasing delays, jitter.\n```\n\n## Best Practices\n\n- **Start with 3-5 tasks** that represent your real workload, not toy examples\n- **Run at least 3 trials** per agent to capture variance — agents are non-deterministic\n- **Pin the commit** in your task YAML so results are reproducible across days\u002Fweeks\n- **Include at least one deterministic judge** (tests, build) per task — LLM judges add noise\n- **Track cost alongside pass rate** — a 95% agent at 10x the cost may not be the right choice\n- **Version your task definitions** — they are test fixtures, treat them as code\n\n## Links\n\n- Repository: [github.com\u002Fjoaquinhuigomez\u002Fagent-eval](https:\u002F\u002Fgithub.com\u002Fjoaquinhuigomez\u002Fagent-eval)\n",{"name":5,"tools":49,"license":15,"metadata":50,"description":7},"Read, Write, Edit, Bash, Grep, Glob",{"origin":51},"ECC","main","0da4485785ee599388900be50db1635489d53056",".agents\u002Fskills\u002Fagent-eval","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6","https:\u002F\u002Fdisha6.6.thenitishkr.in\u002F","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh","www.thenitishkr","thenitishkr | AI • Cyber-security • Research\r\n","2026-11-08 03:33:25",{"source":8,"skillName":5,"commands":62,"prompt":72,"zipAvailable":73},[63,66,69],{"tool":64,"command":65},"npx","npx skills add Tashima-Tarsh\u002FDisha6.6 --skill agent-eval",{"tool":67,"command":68},"bunx","bunx skills add Tashima-Tarsh\u002FDisha6.6 --skill agent-eval",{"tool":70,"command":71},"pnpm","pnpm dlx skills add Tashima-Tarsh\u002FDisha6.6 --skill agent-eval","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fagent-eval\u002FSKILL.md",true,[],[76,106,129,153,178,201],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":85,"publishedAt":44},"01M6QRD07D4TGNDK6VJ4EHF8BV","youtube-full","Use when the user needs YouTube transcripts, video search, channel browsing, playlist extraction, or content monitoring. Trigger phrases: 'get the transcript for', 'search YouTube for', 'what are the latest videos on', 'list this playlist', 'monitor this channel', or any request involving a YouTube URL, video ID, or @handle. Do NOT use for downloading video or audio files, YouTube engagement data (likes, comments), or private\u002Fage-restricted videos.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fyoutube-full\u002FSKILL.md",[82],{"categoryNo":83,"categoryName":84,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000016","图像与多媒体",[86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105],"youtube full","youtube","full","transcript","search","monitoring","data","Video","Audio","list","files","do","get","age","trigger","URL","for","browsing","request","youtube transcripts",{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":44},"01M6QRCJ3112VYPA97YG7KGMC2","x-twitter-growth","X\u002FTwitter growth engine for building audience, crafting viral content, and analyzing engagement. Use when the user wants to grow on X\u002FTwitter, write tweets or threads, analyze their X profile, research competitors on X, plan a posting strategy, or optimize engagement. Complements social-content (generic multi-platform) with X-specific depth: algorithm mechanics, thread engineering, reply strategy, profile optimization, and competitive intelligence via web search.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fx-twitter-growth\u002FSKILL.md",[],[113,114,115,116,90,117,118,119,120,121,122,123,37,124,26,125,126,127,128,102],"twitter","x twitter growth","growth","research","content","engineering","plan","web search","competitors","analyze","engine","strategy","platform","grow","posting","competitive",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":136,"publishedAt":44},"01M6QRBN3EHQNJY42RERP7A149","write-a-skill","Create new agent skills with proper structure, progressive disclosure, and bundled resources. Use when user wants to create, write, build, or author a new skill.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrite-a-skill\u002FSKILL.md",[135],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[137,37,138,139,140,141,142,143,26,144,145,52,146,147,148,149,30,150,151,152],"write a skill","github","create","https","productivity","matt pocock","direct","example","build","new skill","structure","new agent","resources","tree","progressive disclosure","author",{"skillNo":154,"name":155,"nameZh":6,"description":156,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":157,"categories":158,"keywords":159,"publishedAt":44},"01M6QRB6G8P33PNGAF9Y69EJ9T","wrap-up","Close out a launched Claude Managed Agent — recap every primitive the founder now owns, regenerate the single-file overview page, and suggest the next 1-2 upgrades. Use when the user says \"wrap up\", \"close this out\", \"what do I own now\", \"give me the summary\", \"recap the agent\", or when the orchestrator routes phase=wrap-up. primitives_inventory.py tables everything owned (agent, environment, session, memory, outcome, deployment); overview_page.py regenerates a self-contained .\u002Fmy-agent\u002Fagent-overview.html; upgrade_suggester.py ranks the next moves from recorded deferrals plus standing hardening steps. Companion to run-without-you; the last stop before phase=done.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fwrap-up\u002FSKILL.md",[],[160,161,162,163,164,165,166,167,168,26,169,97,170,171,172,173,174,175,176,177],"wrap up","up","wrap","gemini","deployment","run","file","hardening","closeout","stop","upgrade","founder","close","companion","you","tables","directions","phase",{"skillNo":179,"name":180,"nameZh":6,"description":181,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":182,"categories":183,"keywords":185,"publishedAt":44},"01M6QRA5AYC1RBFZ39GTWGS3N8","workflow-builder","Design and write deterministic multi-agent workflow scripts (.js files in .claude\u002Fworkflows\u002F) for Claude Code's Workflow tool. Use when a user wants to build, create, author, scaffold, or run a custom Claude Code workflow, orchestrate sub-agents (fan-out, pipeline, loop, judge-panel), or automate a repeatable multi-step task across fresh-context agents.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fworkflow-builder\u002FSKILL.md",[184],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},[186,187,188,189,190,139,165,42,191,192,193,194,37,195,96,196,197,198,199,200],"workflow","workflow builder","builder","design","workflows","orchestrate","pipeline","loop","multi-agent","context","across","fresh","scripts","judge","claude code workflow",{"skillNo":202,"name":203,"nameZh":6,"description":204,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":205,"categories":206,"keywords":207,"publishedAt":44},"01M6QR7QMCZESKQA0ERW1E657M","weekly-review","Use when someone wants to run a weekly review, close open loops, audit stalled projects and commitments, get their system back to trusted, restart a lapsed review habit, or says \"\u002Fcs:weekly-review\". Walks David Allen's three-phase loop — GET CLEAR, GET CURRENT, GET CREATIVE — with deterministic scripts that inventory open loops, gate the checklist with named gaps, and score commitment health 0-100.","https:\u002F\u002Fgithub.com\u002FTashima-Tarsh\u002FDisha6.6\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fweekly-review\u002FSKILL.md",[],[208,209,210,211,212,213,214,215,216,193,217,218,219,220,221,222,98,223,224,177],"review","weekly review","weekly","deterministic scripts","reflect","feeds","inventory","open","team","projects","project-management","scanner","conversation","back","project","sprint","cadence"]