[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M70H1HZFCCSBZ9BYBFD3GRNF":3,"skill-same:01M70H1HZFCCSBZ9BYBFD3GRNF":68},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":6,"ownerDescription":52,"updatedAt":53,"install":54,"resources":67},"01M70H1HZFCCSBZ9BYBFD3GRNF","regex-vs-llm-structured-text",null,"Decision framework for parsing structured text (quizzes, forms, invoices, receipts, tables) with a hybrid regex-first pipeline — regex extraction handles 95%+ cheaply, a confidence scorer flags low-confidence items, and an LLM validator fixes only the edge cases. Use when choosing between regex and LLM for text extraction, building a cheap document parser, or optimizing extraction cost and accuracy.","mdnaimul22\u002Fhuman-skills","mdnaimul22","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F64520358?v=4","01M70FH95MH2H796Q01TYSFZ03","Naimul","01M70FH96798CQR4YHA13F17GA",3,"MIT","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fregex-vs-llm-structured-text\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"text","regex vs llm structured text","structured","regex","when","framework","document parser","hybrid","fixes","validator","invoices","tables","forms","flags","only","parser","decision","receipts","parsing","pipeline","2026-11-10 10:12:30",false,"---\nname: regex-vs-llm-structured-text\ndescription: Decision framework for parsing structured text (quizzes, forms, invoices, receipts, tables) with a hybrid regex-first pipeline — regex extraction handles 95%+ cheaply, a confidence scorer flags low-confidence items, and an LLM validator fixes only the edge cases. Use when choosing between regex and LLM for text extraction, building a cheap document parser, or optimizing extraction cost and accuracy.\nmetadata:\n  origin: ECC\n---\n\n# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n","# Regex vs LLM for Structured Text Parsing\n\nA practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.\n\n## When to Activate\n\n- Parsing structured text with repeating patterns (questions, forms, tables)\n- Deciding between regex and LLM for text extraction\n- Building hybrid pipelines that combine both approaches\n- Optimizing cost\u002Faccuracy tradeoffs in text processing\n\n## Decision Framework\n\n```\nIs the text format consistent and repeating?\n├── Yes (>90% follows a pattern) → Start with Regex\n│   ├── Regex handles 95%+ → Done, no LLM needed\n│   └── Regex handles \u003C95% → Add LLM for edge cases only\n└── No (free-form, highly variable) → Use LLM directly\n```\n\n## Architecture Pattern\n\n```\nSource Text\n    │\n    ▼\n[Regex Parser] ─── Extracts structure (95-98% accuracy)\n    │\n    ▼\n[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)\n    │\n    ▼\n[Confidence Scorer] ─── Flags low-confidence extractions\n    │\n    ├── High confidence (≥0.95) → Direct output\n    │\n    └── Low confidence (\u003C0.95) → [LLM Validator] → Output\n```\n\n## Implementation\n\n### 1. Regex Parser (Handles the Majority)\n\n```python\nimport re\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass ParsedItem:\n    id: str\n    text: str\n    choices: tuple[str, ...]\n    answer: str\n    confidence: float = 1.0\n\ndef parse_structured_text(content: str) -> list[ParsedItem]:\n    \"\"\"Parse structured text using regex patterns.\"\"\"\n    pattern = re.compile(\n        r\"(?P\u003Cid>\\d+)\\.\\s*(?P\u003Ctext>.+?)\\n\"\n        r\"(?P\u003Cchoices>(?:[A-D]\\..+?\\n)+)\"\n        r\"Answer:\\s*(?P\u003Canswer>[A-D])\",\n        re.MULTILINE | re.DOTALL,\n    )\n    items = []\n    for match in pattern.finditer(content):\n        choices = tuple(\n            c.strip() for c in re.findall(r\"[A-D]\\.\\s*(.+)\", match.group(\"choices\"))\n        )\n        items.append(ParsedItem(\n            id=match.group(\"id\"),\n            text=match.group(\"text\").strip(),\n            choices=choices,\n            answer=match.group(\"answer\"),\n        ))\n    return items\n```\n\n### 2. Confidence Scoring\n\nFlag items that may need LLM review:\n\n```python\n@dataclass(frozen=True)\nclass ConfidenceFlag:\n    item_id: str\n    score: float\n    reasons: tuple[str, ...]\n\ndef score_confidence(item: ParsedItem) -> ConfidenceFlag:\n    \"\"\"Score extraction confidence and flag issues.\"\"\"\n    reasons = []\n    score = 1.0\n\n    if len(item.choices) \u003C 3:\n        reasons.append(\"few_choices\")\n        score -= 0.3\n\n    if not item.answer:\n        reasons.append(\"missing_answer\")\n        score -= 0.5\n\n    if len(item.text) \u003C 10:\n        reasons.append(\"short_text\")\n        score -= 0.2\n\n    return ConfidenceFlag(\n        item_id=item.id,\n        score=max(0.0, score),\n        reasons=tuple(reasons),\n    )\n\ndef identify_low_confidence(\n    items: list[ParsedItem],\n    threshold: float = 0.95,\n) -> list[ConfidenceFlag]:\n    \"\"\"Return items below confidence threshold.\"\"\"\n    flags = [score_confidence(item) for item in items]\n    return [f for f in flags if f.score \u003C threshold]\n```\n\n### 3. LLM Validator (Edge Cases Only)\n\n```python\ndef validate_with_llm(\n    item: ParsedItem,\n    original_text: str,\n    client,\n) -> ParsedItem:\n    \"\"\"Use LLM to fix low-confidence extractions.\"\"\"\n    response = client.messages.create(\n        model=\"claude-haiku-4-5-20251001\",  # Cheapest model for validation\n        max_tokens=500,\n        messages=[{\n            \"role\": \"user\",\n            \"content\": (\n                f\"Extract the question, choices, and answer from this text.\\n\\n\"\n                f\"Text: {original_text}\\n\\n\"\n                f\"Current extraction: {item}\\n\\n\"\n                f\"Return corrected JSON if needed, or 'CORRECT' if accurate.\"\n            ),\n        }],\n    )\n    # Parse LLM response and return corrected item...\n    return corrected_item\n```\n\n### 4. Hybrid Pipeline\n\n```python\ndef process_document(\n    content: str,\n    *,\n    llm_client=None,\n    confidence_threshold: float = 0.95,\n) -> list[ParsedItem]:\n    \"\"\"Full pipeline: regex -> confidence check -> LLM for edge cases.\"\"\"\n    # Step 1: Regex extraction (handles 95-98%)\n    items = parse_structured_text(content)\n\n    # Step 2: Confidence scoring\n    low_confidence = identify_low_confidence(items, confidence_threshold)\n\n    if not low_confidence or llm_client is None:\n        return items\n\n    # Step 3: LLM validation (only for flagged items)\n    low_conf_ids = {f.item_id for f in low_confidence}\n    result = []\n    for item in items:\n        if item.id in low_conf_ids:\n            result.append(validate_with_llm(item, content, llm_client))\n        else:\n            result.append(item)\n\n    return result\n```\n\n## Real-World Metrics\n\nFrom a production quiz parsing pipeline (410 items):\n\n| Metric | Value |\n|--------|-------|\n| Regex success rate | 98.0% |\n| Low confidence items | 8 (2.0%) |\n| LLM calls needed | ~5 |\n| Cost savings vs all-LLM | ~95% |\n| Test coverage | 93% |\n\n## Best Practices\n\n- **Start with regex** — even imperfect regex gives you a baseline to improve\n- **Use confidence scoring** to programmatically identify what needs LLM help\n- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)\n- **Never mutate** parsed items — return new instances from cleaning\u002Fvalidation steps\n- **TDD works well** for parsers — write tests for known patterns first, then edge cases\n- **Log metrics** (regex success rate, LLM call count) to track pipeline health\n\n## Anti-Patterns to Avoid\n\n- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)\n- Using regex for free-form, highly variable text (LLM is better here)\n- Skipping confidence scoring and hoping regex \"just works\"\n- Mutating parsed objects during cleaning\u002Fvalidation steps\n- Not testing edge cases (malformed input, missing fields, encoding issues)\n\n## When to Use\n\n- Quiz\u002Fexam question parsing\n- Form data extraction\n- Invoice\u002Freceipt processing\n- Document structure parsing (headers, sections, tables)\n- Any structured text with repeating patterns where cost matters\n",{"name":5,"metadata":44,"description":7},{"origin":45},"ECC","main","74715deacdc69119d739f38916f8ee4f0f979ebf","skills\u002Fstorage\u002Faffaan-m_ECC\u002Fregex-vs-llm-structured-text",0,"https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills","https:\u002F\u002Fgithub.com\u002Fmdnaimul22","AI Agents Workflows Automation | Specializing in Python, FastAPI, MCP Agents & Custom ERP.","2026-11-11 21:16:57",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add mdnaimul22\u002Fhuman-skills --skill regex-vs-llm-structured-text",{"tool":60,"command":61},"bunx","bunx skills add mdnaimul22\u002Fhuman-skills --skill regex-vs-llm-structured-text",{"tool":63,"command":64},"pnpm","pnpm dlx skills add mdnaimul22\u002Fhuman-skills --skill regex-vs-llm-structured-text","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fregex-vs-llm-structured-text\u002FSKILL.md",true,[],[69,77,85,92,99,106],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":76},"01M767791C1KRP2ZRW2Z73VZ5Q","system-lifecycle-designer","Universal architectural methodology for designing granular, atomic, and sub-atomic lifecycles and end-to-end workflows for single functionality integrations across any domain (daemons, batch jobs, request transactions, resource pools). Provides patterns for deterministic identity, pre-commit validation, differential reconciliation, sticky leasing, state machines, transactional persistence, and closed-loop feedback.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fmy_skills\u002Fsystem-lifecycle-designer\u002FSKILL.md",[],[],"2026-11-12 13:55:43",{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":84},"01M748ABPN5F1FANHH6HKZQMXT","style-sampling","Make a new episode in a proven video style - Collage Explainer, Finance Explainer, Animated Science Explainer, 3D Brand Explainer, 3D True Crime - on the person's topic. Use for \"an episode of\", \"in the style of\", \"sample this style\", 「用这个风格做一集」. One video to copy is reverse-any-video.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Fstyle-sampling\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":84},"01M748A59V04PKW4HA0XQ81FGP","reverse-any-video","Rebuild any video file as an editable HyperFrames project - motion graphics, talking heads, live action or AI-generated footage - checked against the original frame by frame. Use to reverse, recreate, remake, replicate or copy a video, or 「复刻这个视频」「照着这个视频做一个」.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Freverse-any-video\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":84},"01M7487T50P55CSXMWXZ0G9KRG","typescript-patterns","TypeScript patterns for strict, type-safe, maintainable code in modern TypeScript and JavaScript applications. Covers strict compiler configuration, type design, narrowing, discriminated unions, generics, async and error handling, module boundaries, dependency hygiene, and type-level testing. Use when writing or reviewing TypeScript or JavaScript code.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftypescript-patterns\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":84},"01M7487ERZRGPC5NJ75077Y238","test-collaboration","Inventory and govern a project's test assets. Connect requirements, business rules, risks, bugs, and cross-boundary contracts to stable TEST-IDs and executable evidence in the repository's existing test registry or TESTS.md. Use for test inventory, test gap analysis, unit\u002Fintegration\u002Fcontract\u002FE2E\u002Fsmoke classification, bug-to-test regression protection, consumer\u002Fprovider contract testing, test-necessity review, and pre-delivery evidence review.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftest-collaboration\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":84},"01M748721J3GKR2WS6K0QWFC35","story-lifecycle","Manage a file-based epic → story → sprint development loop without requiring an external project management tool. Creates and tracks epics, breaks them into user stories, assigns them to sprints, and drives implementation story by story. Use when starting a new feature initiative or when the team has no Jira\u002FLinear\u002FTaiga but needs structured delivery tracking.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fstory-lifecycle\u002FSKILL.md",[],[]]