data-catalog
Create and enrich durable data catalogs using the native DS_CATALOG_V1 Markdown contract, declared entity relationships, privacy citation fields, and stable relationship IDs. Use when inventorying engagement data, recording semantic relationships, or preparing a catalog for ERD rendering.
- License 许可
- CC-BY-4.0
- 最近更新
- 2026-11-14
- Stars
- ★ 1.6k
- Forks
- 355
01 前言 Front Matter
- name
- data-catalog
- license
- CC-BY-4.0
- metadata
- { "authors": "Microsoft (planning synthesis)", "last_updated": "2026-08-03", "spec_version": "1.0", "content_based_on": "https://www.w3.org/TR/vocab-dcat-3/; https://www.w3.org/TR/prov-o/; https://www.dublincore.org/specifications/dublin-core/dcmi-terms/; https://datapackage.org/standard/table-schema/" }
- description
- Create and enrich durable data catalogs using the native DS_CATALOG_V1 Markdown contract, declared entity relationships, privacy citation fields, and stable relationship IDs. Use when inventorying engagement data, recording semantic relationships, or preparing a catalog for ERD rendering.
- user-invocable
- false
02 资源文件 17
- 资产文件assets/ds-catalog-v1.schema.json5.9 KB
- 参考文档examples/northwind-catalog.md8.4 KB
- 其他pyproject.toml698 B
- 参考文档references/catalog-contract.md11.1 KB
- 参考文档references/dcat-crosswalk.md5.4 KB
- 参考文档references/provenance.md3.1 KB
- 脚本scripts/validate_catalog.py15.0 KB
- 参考文档templates/ds-catalog-v1.md5.3 KB
- 其他tests/corpus/0_valid_frontmatter90 B
- 其他tests/corpus/1_empty_frontmatter8 B
- 其他tests/corpus/2_unclosed_sequence20 B
- 其他tests/corpus/3_duplicate_key24 B
- 其他tests/corpus/4_no_frontmatter15 B
- 参考文档tests/corpus/README.md1.2 KB
- 脚本tests/fuzz_harness.py1.2 KB
- 脚本tests/test_validate_catalog.py24.0 KB
- 其他uv.lock107.7 KB
03 SKILL.md 正文
Data Catalog Workflow
Goal
Produce a customer-readable Markdown catalog whose YAML frontmatter is a valid DS_CATALOG_V1 machine contract. Preserve uncertainty explicitly so inferred or assumed relationships never appear confirmed.
Flow
- Confirm the engagement name and the caller-approved durable output path.
- Inventory entities at business grain. Record source access, tier, volume, profile pointer, classification, lineage, and open questions without copying column-level profile data.
- Assign every relationship a stable
rel-*identifier. Record endpoints, maximum cardinality, both endpoint minimums, one or more paired join-key fields, confidence, and evidence basis. - Reconcile coverage counts with the entity and relationship records.
- Render the human-readable sections from the YAML facts, ending with the canonical Data Science and Engineering Coaching disclaimer footer. Narrative can explain facts but cannot redefine them.
- Validate the artifact with
scripts/validate_catalog.pybefore treating it as ready for review.
Inputs
- Engagement context and a caller-approved output path
- Data source inventory and access status
- Business entity names, grain, and declared relationships
- Existing per-dataset profile paths, when available
- Privacy classifications or standards citations produced by the owning privacy workflow
Success criteria
- The frontmatter declares exactly
catalog_version: DS_CATALOG_V1and validates againstassets/ds-catalog-v1.schema.json. - Entity IDs and relationship IDs are unique and stable. Every endpoint and lineage reference resolves.
- Every relationship declares
cardinalityas its maximum multiplicity plusfrom_minimumandto_minimumaszeroorone. - Join keys use one string on both sides or paired arrays of equal length, and record field names only without primary-key, foreign-key, or uniqueness roles.
- Relationship confidence is one of
confirmed,inferred, orassumed, and every relationship records its basis. - Classification uses the
privacy-standardscitation-field names. The catalog does not invent standards identifiers. - Column statistics and feature metadata remain behind
profile_refrather than being copied into the catalog. - Every customer-facing catalog ends with the canonical Data Science and Engineering Coaching disclaimer footer.
Constraints
- Treat catalog enrichment as user-driven. Offer enrichment when a source is missing, but do not change the catalog silently.
- Record source locations as paths or connection references, never embedded credentials.
- Keep DCAT alignment non-binding. The catalog is native YAML, not RDF, and makes no DCAT conformance claim.
- Keep tier recording separate from tier behavior.
data-catalogrecords the tier;dataopsowns what that tier means. - Keep rendering separate from semantic authority. Diagram tools consume declared relationships and do not infer new ones.
- End every customer-facing catalog with the canonical Data Science and Engineering Coaching disclaimer from the
disclaimer-languageskill.
Stop rules
- Stop and request clarification when an entity grain, relationship endpoint, join-key pairing, or endpoint minimum is ambiguous.
- Stop and retain
inferredorassumedconfidence when evidence does not supportconfirmed. - Stop before a durable write when the caller has not confirmed the destination.
- Stop and route privacy interpretation to
privacy-standardsor the Privacy Planner when citation values or DPIA status are unknown.
Package resources
| Resource | Use |
|---|---|
| catalog-contract.md | Read for the authoritative field, identity, multiplicity, and body-generation rules |
| dcat-crosswalk.md | Read when planning a DCAT, DCAT-AP, or catalog-platform export |
| provenance.md | Read for standards selection, licensing, and non-conformance boundaries |
| ds-catalog-v1.md | Copy when starting a new catalog |
| northwind-catalog.md | Read as a complete valid example with scalar and composite join keys |
assets/ds-catalog-v1.schema.json |
Use as the structural JSON Schema for parsed frontmatter |
scripts/validate_catalog.py |
Execute with uv run python scripts/validate_catalog.py <catalog.md> to validate a catalog |
Attribution
The native contract, workflow, schema, template, examples, and validator are repository-original content licensed CC BY 4.0.
The crosswalk paraphrases selected concepts from W3C DCAT 3, W3C PROV-O, and Frictionless Table Schema v2. Standard names and term identifiers are factual citations, the crosswalk prose and the paired-array join-key contract are independently authored, and no upstream schema, example, table, or substantial excerpt is reproduced. Because no upstream expression is redistributed, the package remains solely CC BY 4.0. See provenance.md.
04 同仓库更多 Skills
statistical-hypothesis-testing
Compare binary rates or average values across affected and unaffected cohorts during root cause analysis. Use when root-cause-analysis selects an applicable affected-versus-unaffected cohort comparison.
root-cause-analysis
Investigate incidents, software defects, and data or process failures by forming falsifiable hypotheses, executing tests against available evidence, and iterating to a verified causal explanation or an explicit evidence blocker. Use when performing root cause analysis, incident investigation, or postmortem analysis, with cost and model choice confirmed before a full investigation.
causal-evidence-likelihood
Assess how one evidence item changes an RCA hypothesis using an uncalibrated, model-elicited likelihood-ratio band. Use when root-cause-analysis selects an eligible hypothesis-evidence pair.
untrusted-content-boundary
Untrusted-content boundary: treat ingested external content as data, not instructions, and refuse embedded authority changes. Preloaded by agents that read external, tracker, or user-supplied content.
telemetry-foundations
Declarative OpenTelemetry-aligned telemetry vocabulary and instrumentation conventions for traces, metrics, logs, and PII handling
pr-reference
Generates PR reference XML with commit history and unified diffs between branches, with extension and path filtering. Use when creating pull request descriptions, preparing code reviews, analyzing branch changes, discovering work items from diffs, or generating structured diff summaries.