advanced-evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
- License 许可
- MIT
- 最近更新
- 2026-04-19
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- advanced-evaluation
- risk
- safe
- source
- community
- date_added
- 1773792000000
- description
- This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
02 资源文件 4
- 参考文档resources/bias-mitigation.md2.0 KB
- 参考文档resources/case-studies.md2.3 KB
- 参考文档resources/evaluation-frameworks.md1.6 KB
- 参考文档resources/rubric-design.md1.5 KB
03 SKILL.md 正文
Advanced Evaluation
Overview
Implement production-grade LLM-as-judge patterns to evaluate model outputs. This skill provides a taxonomy for choosing the right evaluation method and protocols for mitigating systematic biases.
? Resources
| Resource | Description |
|---|---|
| [Evaluation Frameworks](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/evaluation-frameworks.md) | Choosing between Direct Scoring and Pairwise Comparison. |
| [Bias Mitigation](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/bias-mitigation.md) | Protocols for Position, Length, and Verbosity bias. |
| [Rubric Design](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/rubric-design.md) | Patterns for creating consistent grading standards. |
| [Case Studies](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/case-studies.md) | Implementation examples for Accuracy, Tone, and Readability. |
Core Decision Framework
Direct Scoring
Best for objective criteria (accuracy, format). Rates on a scale (1-5/1-10).
See [Evaluation Frameworks](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/evaluation-frameworks.md) for details.
Pairwise Comparison
Best for subjective preference (style, tone, helpfulness). Compares two outputs directly.
Required: Always use [Bias Mitigation](file:///c:/workspaces/antigravity-meta/.agents/skills/advanced-evaluation/resources/bias-mitigation.md) (Position Swap).
Best Practices
- Justification First: Always require the judge to provide evidence before the score.
- Swap Positions: When using pairwise comparison, run the evaluation twice with roles swapped.
- Calibrate Confidence: Adjust confidence values based on judge consistency and evidence quality.
Related Skills
context-fundamentals: Prompt structure for evaluation.tool-design: Building evaluation infrastructure.
Summary
Build reliable evaluation systems by selecting the correct taxonomy and aggressively mitigating LLM biases.
04 同仓库更多 Skills
wcag-audit-patterns
Comprehensive guide to auditing web content against WCAG 2.2 guidelines with actionable remediation strategies.
unit-testing-test-generate
Generate comprehensive, maintainable unit tests across languages with strong coverage and edge case focus.
testing-qa
Comprehensive testing and QA workflow covering unit testing, integration testing, E2E testing, browser automation, and quality assurance.
test-fixing
Systematically identify and fix all failing tests using smart grouping strategies. Use when explicitly asks to fix tests ("fix these tests", "make tests pass"), reports test failures ("tests are failing", "test suite is broken"), or completes implementation and wants tests passing.
test-driven-development
Use when implementing any feature or bugfix, before writing implementation code
terraform-infrastructure
Terraform infrastructure as code workflow for provisioning cloud resources, creating reusable modules, and managing infrastructure at scale.