search-fail-pivot
Heuristic for detecting search tool failures and pivoting to domain knowledge for document generation
- License 许可
- MIT
- 所属仓库
- HKUDS/AgentPilot
- 最近更新
- 2026-11-12
- Stars
- ★ 8.0k
- Forks
- 944
01 前言 Front Matter
- name
- search-fail-pivot
- description
- Heuristic for detecting search tool failures and pivoting to domain knowledge for document generation
02 资源文件 1
- 其他.skill_id31 B
03 SKILL.md 正文
Search Tool Failure Detection and Pivot Strategy
Purpose
When using search_web or similar external data retrieval tools, repeated failures waste iterations. This skill provides a diagnostic heuristic to recognize when to abandon search attempts and proceed with existing domain knowledge.
Core Heuristic
If search_web fails 2+ consecutive times with 'unknown error' or similar non-recoverable errors, pivot to domain knowledge + run_shell for document generation.
Do not attempt more than 2 retries on the same or similar queries before pivoting.
Step-by-Step Instructions
Step 1: Track Search Failures
Monitor search_web outcomes during your task:
- Count consecutive failures (empty results, 'unknown error', timeout, access denied)
- Note the error type - distinguish between:
- Recoverable: Rate limiting, temporary timeout (retry 1-2 times)
- Non-recoverable: 'unknown error', persistent empty results, access denied
Step 2: Apply the 2-Failure Rule
IF search_web failures >= 2 (same or similar queries)
THEN:
1. Stop attempting search_web
2. Document what information you attempted to retrieve
3. Proceed with existing domain knowledge
Step 3: Pivot to Domain Knowledge
When pivoting:
- Acknowledge the limitation: Note that external verification was attempted but unavailable
- Use established knowledge: Draw on training data for well-known facts, laws, cases, standards
- Generate with run_shell: Create documents using Python/bash scripts rather than waiting for external data
Example pivot workflow:
# Instead of continuing search_web attempts:
# 1. Compile known information from domain knowledge
known_facts = {
"law": "COPPA requirements for children's data",
"precedent": "FTC v. Google/YouTube settlement patterns",
"jurisdiction": "California privacy law framework"
}
# 2. Generate document directly
# Use run_shell with Python to create PDF/DOC
Step 4: Document Generation Pattern
When generating documents after pivoting:
# Use run_shell to execute document generation
# Example: Create legal memo with Python + reportlab
python << 'EOF'
from reportlab.lib.pagesizes import letter
from reportlab.pdfgen import canvas
# Generate document with domain knowledge
c = canvas.Canvas("memo.pdf", pagesize=letter)
c.drawString(100, 750, "Legal Analysis based on established precedents")
# ... continue with known information
c.save()
EOF
When This Applies
This heuristic is particularly useful for:
| Task Type | Why It Applies |
|---|---|
| Legal research | Case law and statutes are well-documented in training |
| Technical documentation | Standards and best practices are established knowledge |
| Historical analysis | Past events and data are in training corpus |
| Regulatory compliance | Major regulations (GDPR, COPPA, HIPAA) are well-known |
When NOT to Pivot
Do NOT apply this heuristic when:
- You need current/time-sensitive data (stock prices, weather, news < 24hrs)
- The task explicitly requires external verification
- You have not yet attempted the search (first attempt should complete)
- Errors are clearly recoverable (rate limit with retry-after header)
Example Decision Flow
Attempt search_web → Success? → Yes → Use results
↓
No
↓
Is this failure #1? → Yes → Retry once with modified query
↓
No (failure #2+)
↓
Pivot to domain knowledge + run_shell document generation
Benefits
- Saves iterations: Avoids wasting 3-4 attempts on unavailable data
- Maintains progress: Task continues instead of stalling on search
- Leverages training: Uses the substantial knowledge already available
- Clear handoff: Explicitly documents the pivot decision for transparency
Anti-Patterns to Avoid
❌ Continuing search_web attempts beyond 2 failures
❌ Not documenting why you pivoted from search
❌ Applying this to tasks requiring real-time data
❌ Abandoning search on first failure without retry
✅ Apply 2-failure rule consistently
✅ Document the pivot decision in your output
✅ Use run_shell efficiently after pivoting
✅ Distinguish recoverable vs non-recoverable errors
04 同仓库更多 Skills
agentpilot
Set up AgentPilot and use its single entry to prepare missing inputs and run configured RSI experiments. Use when the user requests AgentPilot installation or work with AgentPilot.
remember
Review auto-memory entries and propose promotions to OPENSPACE.md, OPENSPACE.local.md, or retained auto-memory topics; detect duplicates, stale facts, conflicts, and ambiguous entries before making changes.
skill-discovery
Search reusable OpenSpace skills locally, and browse cloud packages step by step when a cloud skill may help. Reusing proven skills saves tokens, improves reliability, and extends your capabilities beyond built-in tools.
delegate-task
Delegate tasks to OpenSpace — a full-stack autonomous worker for coding, DevOps, web research, and desktop automation, backed by an extensive MCP tool and skill library. Skills auto-improve through use, reducing token consumption over time. A cloud community lets agents share and collectively evolve reusable skills.
write-special-chars-ts-check
Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, plus a reliable TypeScript type-checking step that filters output to only the newly-written files to avoid being misled by pre-existing errors.
write-special-chars-readback
Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, with mandatory post-write verification via read_file and cleanup of /tmp helper scripts.