AGENT SKILL
eda/engineering
EDA engineering practices
- License 许可
- MIT
- 最近更新
- 2026-03-01
- Stars
- ★ 3
- Forks
- 2
01 前言 Front Matter
- name
- eda/engineering
- license
- MIT
- metadata
- { "audience": "data-engineers", "category": "data-science" }
- description
- EDA engineering practices
- compatibility
- opencode
02 资源文件 5
- 参考文档automation_proposal.md613 B
- 其他codex_triage.toml488 B
- 参考文档mapping_note.md2.9 KB
- 参考文档README.md49 B
- 参考文档workflow_map.json6.3 KB
03 SKILL.md 正文
What I do
- Build scalable EDA pipelines
- Automate data profiling
- Create reproducible data analysis workflows
- Implement data quality monitoring
- Design data exploration tools
- Build self-service analytics infrastructure
When to use me
Use me when:
- Building data infrastructure
- Automating EDA for large datasets
- Creating data quality dashboards
- Implementing data governance
- Building analytics platforms
Key Concepts
Data Profiling Pipeline
from great_expectations import GreatExpectations
import pandas as pd
# Define expectations
expectations = [
{"expectation_type": "expect_column_values_to_not_be_null",
"kwargs": {"column": "id"}},
{"expectation_type": "expect_column_values_to_be_unique",
"kwargs": {"column": "id"}},
{"expectation_type": "expect_column_values_to_be_between",
"kwargs": {"column": "age", "min_value": 0, "max_value": 120}},
{"expectation_type": "expect_column_distinct_values_to_be_in_set",
"kwargs": {"column": "status", "value_set": ["active", "inactive"]}}
]
# Validate data
ge = GreatExpectations()
batch = ge.get_batch("data.csv", "default")
results = ge.validate(batch, expectations=expectations)
Automated Profiling
from pandas_profiling import ProfileReport
# Generate comprehensive EDA report
profile = ProfileReport(
df,
title="Data Profiling Report",
explorative=True,
missing_diagrams=True,
correlations={"high_cardinality": "include"}
)
profile.to_file("eda_report.html")
Data Quality Metrics
- Completeness: % non-null values
- Uniqueness: % unique values
- Validity: % values in valid range
- Consistency: Cross-field validation
- Timeliness: Data freshness
Monitoring Setup
- Great Expectations for validation
- Datafold for diff analysis
- Great Expectations Cloud
- Amundsen for data catalog
04 同仓库更多 Skills
xamarin
Xamarin cross-platform development
★ 3
wrangling
Data wrangling fundamentals
★ 3
windows
Windows operating system administration and scripting
★ 3
wide-column
Wide-column data model
★ 3
wide-column-stores
Wide-column database stores
★ 3
websockets
WebSocket implementation and best practices
★ 3