extracting-keywords
Use when extracting keywords (YAKE/RAKE) from documents — and, secondarily, when detecting document language or generating embeddings for RAG and search. Covers the keyword config (and its feature gating), `--detect-language`, and the standalone `embed` command with real flags.
- License 许可
- Apache-2.0
- 最近更新
- 2026-11-14
- Stars
- ★ 1.6k
- Forks
- 491
01 前言 Front Matter
- name
- extracting-keywords
- description
- Use when extracting keywords (YAKE/RAKE) from documents — and, secondarily, when detecting document language or generating embeddings for RAG and search. Covers the keyword config (and its feature gating), `--detect-language`, and the standalone `embed` command with real flags.
03 SKILL.md 正文
Extracting keywords, language, and embeddings
Use this for the enrichment surface around extraction: statistical keyword
extraction, language detection, and vector embeddings. Keywords and
language detection ride along with extraction and land on the result;
embeddings are produced by a dedicated embed command.
Keywords (YAKE / RAKE)
Keyword extraction is configured via the [keywords] config block (or
inline JSON) — there is no single --keywords CLI flag. When enabled,
extracted keywords appear on result.keywords. Two algorithms are
available:
- YAKE (
"yake") — statistical, unsupervised single-document extraction. Good general default. - RAKE (
"rake") — co-occurrence / phrase-based. Favors multi-word key phrases.
Feature-gated: keyword extraction requires the CLI to be built with the
keywords-yakeand/orkeywords-rakeCargo features (both are in the default/fullbuild). If the CLI was built without them, the[keywords]config block is silently ignored —result.keywordssimply stays empty rather than erroring. The"yake"algorithm needskeywords-yake;"rake"needskeywords-rake.
Enable via inline JSON on the CLI:
kreuzberg extract paper.pdf --format json \
--config-json '{"keywords":{"algorithm":"yake","max_keywords":15,"language":"en"}}' \
| jq '.keywords'
Or in a config file:
[keywords]
algorithm = "rake" # "yake" or "rake"
max_keywords = 10 # default 10
min_score = 0.0 # filter below this score (ranges differ per algorithm)
ngram_range = [1, 3] # unigrams..trigrams (default)
language = "en" # stopword language; omit to skip stopword filtering
kreuzberg extract report.pdf --config kreuzberg.toml --format json | jq '.keywords'
Field notes:
max_keywordscaps how many keywords are returned (default 10).min_scorefilters low-scoring keywords; note YAKE scores are lower-is-better while RAKE scores are higher-is-better, so a single threshold behaves differently per algorithm.ngram_rangeis[min, max]:[1,1]unigrams only,[1,2]adds bigrams,[1,3](default) adds trigrams.languageenables stopword filtering for that language; omit it to disable stopword filtering entirely.
Language detection
Language detection is a real CLI flag: --detect-language. Detected
languages appear on result.detected_languages:
kreuzberg extract multilingual.pdf --detect-language true --format json \
| jq '.detected_languages'
In a config file it lives under [language_detection]:
[language_detection]
enabled = true
min_confidence = 0.8
detect_multiple = false
The CLI flag enables detection with min_confidence = 0.8 and
single-language mode; use the config block to detect multiple languages or
tune confidence.
Embeddings (embed command)
The standalone embed command produces vector embeddings for text from
--text (repeatable) or stdin. It does not run extraction — pipe
extracted content in if you want document embeddings.
# Local ONNX preset model (default provider)
kreuzberg embed --text "first passage" --text "second passage" --preset balanced
# Embed extracted document text
kreuzberg extract report.pdf | kreuzberg embed --preset quality
Presets for the local provider: fast, balanced (default), quality,
multilingual. Output defaults to JSON (--format json).
--provider selects the embedding source:
| Provider | Flag | Notes |
|---|---|---|
local |
--preset <fast|balanced|quality|multilingual> |
Default. ONNX model, no API key. |
llm |
--model <id> --api-key <key> |
liter-llm routing, e.g. openai/text-embedding-3-small. |
plugin |
--plugin <name> |
A backend pre-registered in-process via the plugin API. |
# Provider-hosted embeddings via an LLM
kreuzberg embed --text "query text" \
--provider llm --model openai/text-embedding-3-small --api-key "$OPENAI_API_KEY"
Local embedding presets must be downloaded first if not cached. Pre-warm them with the cache command:
kreuzberg cache warm --embedding-model balanced # one preset
kreuzberg cache warm --all-embeddings # all four presets
Programmatic access
Keywords and detected languages live on the extraction result:
from kreuzberg import extract_file_sync, ExtractionConfig
result = extract_file_sync(
"paper.pdf",
config=ExtractionConfig(), # configure keywords/language_detection on the config
)
print(result.keywords) # extracted keywords (when enabled)
print(result.detected_languages) # detected languages (when enabled)
See references/python-api.md and references/configuration.md in the
sibling kreuzberg skill for the keyword / language-detection config
classes and the embedding presets.
Common pitfalls
- No
--keywordsflag — keyword extraction is config-only. Use--config-json '{"keywords":{...}}'or a[keywords]config block. min_scoredirection — lower is better for YAKE, higher is better for RAKE; pick the threshold to match the algorithm.- Embeddings ≠ extraction —
embedonly takes raw text. Pipekreuzberg extractoutput into it for document vectors. - Cold embedding models — first local run downloads the preset; run
kreuzberg cache warm --all-embeddingsto pre-populate.
See references/advanced-features.md for the embeddings pipeline and
references/cli-reference.md for the embed and cache warm flag sets.
04 同仓库更多 Skills
worktable-review-with-annotations
Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.
worktable-find-and-synthesize-context
Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.
worktable-create-or-update-docs
Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.
worktable-create-or-manage-records
Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.
worktable-create-or-manage-interactive-html
Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.
worktable-collaborate-threads
Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.