[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MJ10QJWC4Q24NQF7D1HHG7":3,"skill-same:01M6MJ10QJWC4Q24NQF7D1HHG7":65},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":36,"linkOnly":37,"markdownContent":38,"markdownBody":39,"frontMatter":40,"refName":41,"commitSha":42,"skillDirectory":43,"forksCount":44,"repositoryHtmlUrl":45,"repositoryHomepageUrl":46,"ownerHtmlUrl":47,"ownerBlogUrl":48,"ownerDescription":49,"updatedAt":50,"install":51,"resources":64},"01M6MJ10QJWC4Q24NQF7D1HHG7","chunking",null,"Use when splitting extracted text into chunks for LLM context windows or RAG ingestion. Covers chunk size, overlap, markdown\u002Fyaml\u002Fsemantic chunkers, tokenizer-based sizing, and the standalone `chunk` command.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fkreuzberg-dev\u002Fplugins\u002Fplugins\u002Fkreuzberg\u002Fskills\u002Fchunking\u002FSKILL.md",[],[5,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35],"ingestion","when","chunks","rag ingestion","for","standalone","size","semantic","windows","use","context","command","text","markdown","yaml","chunk","rag","2026-11-10 10:12:30",false,"---\nname: chunking\ndescription: Use when splitting extracted text into chunks for LLM context windows or RAG ingestion. Covers chunk size, overlap, markdown\u002Fyaml\u002Fsemantic chunkers, tokenizer-based sizing, and the standalone `chunk` command.\n---\n\n# Chunking\n\nUse this when feeding documents into an LLM context window or a vector\nstore. Kreuzberg chunks two ways: inline during extraction (chunks land on\n`result.chunks`), or standalone via the `chunk` command for text you\nalready have. Sizing is character-based by default, or token-based when a\ntokenizer model is supplied.\n\n## Inline during extraction\n\nTurn on chunking with `--chunk` and the chunks appear on the structured\nresult under `chunks`:\n\n```bash\n# 1000-char chunks, 200-char overlap (defaults when --chunk is on)\nkreuzberg extract report.pdf --chunk --format json | jq '.chunks | length'\n\n# Explicit size + overlap\nkreuzberg extract report.pdf --chunk --chunk-size 1500 --chunk-overlap 300 --format json\n```\n\nOverlap must be smaller than chunk size — the CLI rejects\n`--chunk-overlap >= --chunk-size`. When you set only `--chunk-overlap`\nagainst an existing config, an overlap that exceeds the size is clamped to\n`chunk_size \u002F 4`.\n\n## Standalone `chunk` command\n\nChunk text you already have, from `--text` or stdin. Output defaults to\nJSON:\n\n```bash\n# From a flag\nkreuzberg chunk --text \"long document text ...\" --chunk-size 800 --chunk-overlap 100\n\n# From stdin (pipe extracted content straight in)\nkreuzberg extract notes.md | kreuzberg chunk --chunk-size 500 --format json\n```\n\nJSON output carries `chunks` (array of strings), `chunk_count`, the\nresolved `config` (`max_characters`, `overlap`, `chunker_type`), and\n`input_size_bytes`. Use `--format text` for a human-readable dump with\n`--- chunk N ---` separators.\n\n> Note: in the JSON output, `chunker_type` is rendered capitalized (`\"Text\"`,\n> `\"Markdown\"`, `\"Yaml\"`, `\"Semantic\"`) because it is emitted via Rust's Debug\n> formatting, whereas the `--chunker-type` input flag is lowercase\n> (`text`, `markdown`, `yaml`, `semantic`). Lowercase the value before\n> comparing if you parse it back.\n\n## Chunker types\n\n`--chunker-type` selects the splitting strategy (standalone `chunk`\ncommand):\n\n| Type       | Behavior                                                            |\n| ---------- | ------------------------------------------------------------------- |\n| `text`     | **Default.** Plain character-window splitting with overlap.         |\n| `markdown` | Markdown-aware — splits on structure (headings, blocks) where possible. |\n| `yaml`     | YAML-aware splitting for structured config\u002Fdata documents.          |\n| `semantic` | Topic-boundary splitting driven by `--topic-threshold` (0.0–1.0, default 0.75). |\n\n```bash\n# Markdown-aware chunking keeps headings and blocks intact\nkreuzberg chunk --text \"$(cat README.md)\" --chunker-type markdown\n\n# Semantic chunking — lower threshold = more, smaller topic chunks\nkreuzberg chunk --text \"$(cat transcript.txt)\" --chunker-type semantic --topic-threshold 0.6\n```\n\n## Token-based sizing\n\nBy default `--chunk-size` counts characters. To size chunks by tokens for\na specific model, pass `--chunking-tokenizer` with a HuggingFace tokenizer\nid. On the `extract` command this implicitly enables chunking. Requires the\n`chunking-tokenizers` feature (present in the default CLI build).\n\n```bash\n# Size chunks by GPT-4o tokens during extraction\nkreuzberg extract report.pdf --chunking-tokenizer Xenova\u002Fgpt-4o --format json\n\n# Or on the standalone command\nkreuzberg chunk --text \"$(cat doc.txt)\" --chunking-tokenizer Xenova\u002Fgpt-4o --chunk-size 512\n```\n\nWith a tokenizer set, `--chunk-size` is interpreted in tokens, not\ncharacters.\n\n## Config file alternative\n\nField names in config files are snake_case under `[chunking]`:\n\n```toml\n[chunking]\nmax_characters = 1000\noverlap = 200\nchunker_type = \"markdown\"\n```\n\n```bash\nkreuzberg extract report.pdf --config kreuzberg.toml --format json\n```\n\n> CLI flags map to config fields as `--chunk-size` → `max_characters` and\n> `--chunk-overlap` → `overlap`. In config files use the snake_case names.\n\n## Programmatic access\n\nFrom Python, enable chunking on the config and read `result.chunks`:\n\n```python\nfrom kreuzberg import extract_file_sync, ExtractionConfig, ChunkingConfig\n\nconfig = ExtractionConfig(\n    chunking=ChunkingConfig(max_chars=1000, max_overlap=200),\n)\nresult = extract_file_sync(\"report.pdf\", config=config)\nfor chunk in result.chunks:\n    print(len(chunk))\n```\n\n> Python `ChunkingConfig` uses `max_chars` \u002F `max_overlap`. Rust uses\n> `max_characters` \u002F `overlap`. See `references\u002Fpython-api.md` and\n> `references\u002Frust-api.md` in the sibling `kreuzberg` skill.\n\n## Picking parameters\n\n- **RAG \u002F vector store** — 500–1000 chars (or 256–512 tokens) with\n  10–20% overlap. Use `markdown` chunking for docs to keep sections whole.\n- **LLM summarization** — larger chunks (1500–4000 chars) with small\n  overlap; size by tokens to stay under the model window.\n- **Topic segmentation** — `semantic` chunker; tune `--topic-threshold`\n  down for finer splits, up for coarser ones.\n\n## Common pitfalls\n\n- **Overlap ≥ size** — rejected on `extract`; clamped to `size \u002F 4` when\n  only overlap is changed against an existing config.\n- **Tokenizer without the feature** — `--chunking-tokenizer` errors if the\n  CLI was built without `chunking-tokenizers`. The default build includes it.\n- **Empty input** — the standalone `chunk` command bails on empty text;\n  provide `--text` or pipe non-empty stdin.\n\nSee `references\u002Fconfiguration.md` for the full `[chunking]` schema and\n`references\u002Fcli-reference.md` for every chunk flag.\n","# Chunking\n\nUse this when feeding documents into an LLM context window or a vector\nstore. Kreuzberg chunks two ways: inline during extraction (chunks land on\n`result.chunks`), or standalone via the `chunk` command for text you\nalready have. Sizing is character-based by default, or token-based when a\ntokenizer model is supplied.\n\n## Inline during extraction\n\nTurn on chunking with `--chunk` and the chunks appear on the structured\nresult under `chunks`:\n\n```bash\n# 1000-char chunks, 200-char overlap (defaults when --chunk is on)\nkreuzberg extract report.pdf --chunk --format json | jq '.chunks | length'\n\n# Explicit size + overlap\nkreuzberg extract report.pdf --chunk --chunk-size 1500 --chunk-overlap 300 --format json\n```\n\nOverlap must be smaller than chunk size — the CLI rejects\n`--chunk-overlap >= --chunk-size`. When you set only `--chunk-overlap`\nagainst an existing config, an overlap that exceeds the size is clamped to\n`chunk_size \u002F 4`.\n\n## Standalone `chunk` command\n\nChunk text you already have, from `--text` or stdin. Output defaults to\nJSON:\n\n```bash\n# From a flag\nkreuzberg chunk --text \"long document text ...\" --chunk-size 800 --chunk-overlap 100\n\n# From stdin (pipe extracted content straight in)\nkreuzberg extract notes.md | kreuzberg chunk --chunk-size 500 --format json\n```\n\nJSON output carries `chunks` (array of strings), `chunk_count`, the\nresolved `config` (`max_characters`, `overlap`, `chunker_type`), and\n`input_size_bytes`. Use `--format text` for a human-readable dump with\n`--- chunk N ---` separators.\n\n> Note: in the JSON output, `chunker_type` is rendered capitalized (`\"Text\"`,\n> `\"Markdown\"`, `\"Yaml\"`, `\"Semantic\"`) because it is emitted via Rust's Debug\n> formatting, whereas the `--chunker-type` input flag is lowercase\n> (`text`, `markdown`, `yaml`, `semantic`). Lowercase the value before\n> comparing if you parse it back.\n\n## Chunker types\n\n`--chunker-type` selects the splitting strategy (standalone `chunk`\ncommand):\n\n| Type       | Behavior                                                            |\n| ---------- | ------------------------------------------------------------------- |\n| `text`     | **Default.** Plain character-window splitting with overlap.         |\n| `markdown` | Markdown-aware — splits on structure (headings, blocks) where possible. |\n| `yaml`     | YAML-aware splitting for structured config\u002Fdata documents.          |\n| `semantic` | Topic-boundary splitting driven by `--topic-threshold` (0.0–1.0, default 0.75). |\n\n```bash\n# Markdown-aware chunking keeps headings and blocks intact\nkreuzberg chunk --text \"$(cat README.md)\" --chunker-type markdown\n\n# Semantic chunking — lower threshold = more, smaller topic chunks\nkreuzberg chunk --text \"$(cat transcript.txt)\" --chunker-type semantic --topic-threshold 0.6\n```\n\n## Token-based sizing\n\nBy default `--chunk-size` counts characters. To size chunks by tokens for\na specific model, pass `--chunking-tokenizer` with a HuggingFace tokenizer\nid. On the `extract` command this implicitly enables chunking. Requires the\n`chunking-tokenizers` feature (present in the default CLI build).\n\n```bash\n# Size chunks by GPT-4o tokens during extraction\nkreuzberg extract report.pdf --chunking-tokenizer Xenova\u002Fgpt-4o --format json\n\n# Or on the standalone command\nkreuzberg chunk --text \"$(cat doc.txt)\" --chunking-tokenizer Xenova\u002Fgpt-4o --chunk-size 512\n```\n\nWith a tokenizer set, `--chunk-size` is interpreted in tokens, not\ncharacters.\n\n## Config file alternative\n\nField names in config files are snake_case under `[chunking]`:\n\n```toml\n[chunking]\nmax_characters = 1000\noverlap = 200\nchunker_type = \"markdown\"\n```\n\n```bash\nkreuzberg extract report.pdf --config kreuzberg.toml --format json\n```\n\n> CLI flags map to config fields as `--chunk-size` → `max_characters` and\n> `--chunk-overlap` → `overlap`. In config files use the snake_case names.\n\n## Programmatic access\n\nFrom Python, enable chunking on the config and read `result.chunks`:\n\n```python\nfrom kreuzberg import extract_file_sync, ExtractionConfig, ChunkingConfig\n\nconfig = ExtractionConfig(\n    chunking=ChunkingConfig(max_chars=1000, max_overlap=200),\n)\nresult = extract_file_sync(\"report.pdf\", config=config)\nfor chunk in result.chunks:\n    print(len(chunk))\n```\n\n> Python `ChunkingConfig` uses `max_chars` \u002F `max_overlap`. Rust uses\n> `max_characters` \u002F `overlap`. See `references\u002Fpython-api.md` and\n> `references\u002Frust-api.md` in the sibling `kreuzberg` skill.\n\n## Picking parameters\n\n- **RAG \u002F vector store** — 500–1000 chars (or 256–512 tokens) with\n  10–20% overlap. Use `markdown` chunking for docs to keep sections whole.\n- **LLM summarization** — larger chunks (1500–4000 chars) with small\n  overlap; size by tokens to stay under the model window.\n- **Topic segmentation** — `semantic` chunker; tune `--topic-threshold`\n  down for finer splits, up for coarser ones.\n\n## Common pitfalls\n\n- **Overlap ≥ size** — rejected on `extract`; clamped to `size \u002F 4` when\n  only overlap is changed against an existing config.\n- **Tokenizer without the feature** — `--chunking-tokenizer` errors if the\n  CLI was built without `chunking-tokenizers`. The default build includes it.\n- **Empty input** — the standalone `chunk` command bails on empty text;\n  provide `--text` or pipe non-empty stdin.\n\nSee `references\u002Fconfiguration.md` for the full `[chunking]` schema and\n`references\u002Fcli-reference.md` for every chunk flag.\n",{"name":5,"description":7},"main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002Fkreuzberg-dev\u002Fplugins\u002Fplugins\u002Fkreuzberg\u002Fskills\u002Fchunking",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":52,"prompt":62,"zipAvailable":63},[53,56,59],{"tool":54,"command":55},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill chunking",{"tool":57,"command":58},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill chunking",{"tool":60,"command":61},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill chunking","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fkreuzberg-dev\u002Fplugins\u002Fplugins\u002Fkreuzberg\u002Fskills\u002Fchunking\u002FSKILL.md",true,[],[66,76,84,92,100,108],{"skillNo":67,"name":68,"nameZh":6,"description":69,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":71,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":75},"01M7DV332YQMGD8WTRJ9DZQXTB","worktable-review-with-annotations","Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.","MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-review-with-annotations\u002FSKILL.md",[],[],"2026-11-15 14:19:17",{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":80,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":75},"01M7DV2YGHCZVBEB1AHYB6CGHE","worktable-find-and-synthesize-context","Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-find-and-synthesize-context\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":88,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":75},"01M7DV2TQN1H11ME2SAF9MXN7V","worktable-create-or-update-docs","Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-update-docs\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":96,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":75},"01M7DV2NE452NEA8TXATKSNAJH","worktable-create-or-manage-records","Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-records\u002FSKILL.md",[],[],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":104,"sourceHtmlUrl":105,"categories":106,"keywords":107,"publishedAt":75},"01M7DV2HENRBR2KSCVM064WH0Y","worktable-create-or-manage-interactive-html","Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-create-or-manage-interactive-html\u002FSKILL.md",[],[],{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":70,"licenseFileUrl":112,"sourceHtmlUrl":113,"categories":114,"keywords":115,"publishedAt":75},"01M7DV2CADR5R7RQFFYPNAGR9Y","worktable-collaborate-threads","Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fworktable\u002Fworktable-plugin\u002Fskills\u002Fworktable-collaborate-threads\u002FSKILL.md",[],[]]