[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7165P0FVVXJFS1KK777EWB3":3,"skill-same:01M7165P0FVVXJFS1KK777EWB3":119},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":31,"publishedAt":52,"linkOnly":53,"markdownContent":54,"markdownBody":55,"frontMatter":56,"refName":61,"commitSha":62,"skillDirectory":63,"forksCount":14,"repositoryHtmlUrl":64,"repositoryHomepageUrl":6,"ownerHtmlUrl":65,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":66,"install":67,"resources":80},"01M7165P0FVVXJFS1KK777EWB3","exploratory-data-analysis",null,"Perform comprehensive exploratory data analysis on scientific data files across 200+ file formats. This skill should be used when analyzing any scientific data file to understand its structure, content, quality, and characteristics. Automatically detects file type and generates detailed markdown reports with format-specific analysis, quality metrics, and downstream analysis recommendations. Covers chemistry, bioinformatics, microscopy, spectroscopy, proteomics, metabolomics, and general scientific data formats.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002FSKILL.md",[18,23,26],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51],"data","exploratory data analysis","analysis","data analysis","general","when","formats","with","scientific","metabolomics","type","recommendations","across","structure","files","reports","markdown","analyzing","file","metrics","2026-11-10 10:12:30",false,"---\nname: exploratory-data-analysis\ndescription: Perform comprehensive exploratory data analysis on scientific data files across 200+ file formats. This skill should be used when analyzing any scientific data file to understand its structure, content, quality, and characteristics. Automatically detects file type and generates detailed markdown reports with format-specific analysis, quality metrics, and downstream analysis recommendations. Covers chemistry, bioinformatics, microscopy, spectroscopy, proteomics, metabolomics, and general scientific data formats.\nlicense: MIT license\nmetadata: {\"version\": \"1.0\", \"skill-author\": \"K-Dense Inc.\"}\n---\n\n# Exploratory Data Analysis\n\n## Overview\n\nPerform comprehensive exploratory data analysis (EDA) on scientific data files across multiple domains. This skill provides automated file type detection, format-specific analysis, data quality assessment, and generates detailed markdown reports suitable for documentation and downstream analysis planning.\n\n**Key Capabilities:**\n- Automatic detection and analysis of 200+ scientific file formats\n- Comprehensive format-specific metadata extraction\n- Data quality and integrity assessment\n- Statistical summaries and distributions\n- Visualization recommendations\n- Downstream analysis suggestions\n- Markdown report generation\n\n## When to Use This Skill\n\nUse this skill when:\n- User provides a path to a scientific data file for analysis\n- User asks to \"explore\", \"analyze\", or \"summarize\" a data file\n- User wants to understand the structure and content of scientific data\n- User needs a comprehensive report of a dataset before analysis\n- User wants to assess data quality or completeness\n- User asks what type of analysis is appropriate for a file\n\n## Supported File Categories\n\nThe skill has comprehensive coverage of scientific file formats organized into six major categories:\n\n### 1. Chemistry and Molecular Formats (60+ extensions)\nStructure files, computational chemistry outputs, molecular dynamics trajectories, and chemical databases.\n\n**File types include:** `.pdb`, `.cif`, `.mol`, `.mol2`, `.sdf`, `.xyz`, `.smi`, `.gro`, `.log`, `.fchk`, `.cube`, `.dcd`, `.xtc`, `.trr`, `.prmtop`, `.psf`, and more.\n\n**Reference file:** `references\u002Fchemistry_molecular_formats.md`\n\n### 2. Bioinformatics and Genomics Formats (50+ extensions)\nSequence data, alignments, annotations, variants, and expression data.\n\n**File types include:** `.fasta`, `.fastq`, `.sam`, `.bam`, `.vcf`, `.bed`, `.gff`, `.gtf`, `.bigwig`, `.h5ad`, `.loom`, `.counts`, `.mtx`, and more.\n\n**Reference file:** `references\u002Fbioinformatics_genomics_formats.md`\n\n### 3. Microscopy and Imaging Formats (45+ extensions)\nMicroscopy images, medical imaging, whole slide imaging, and electron microscopy.\n\n**File types include:** `.tif`, `.nd2`, `.lif`, `.czi`, `.ims`, `.dcm`, `.nii`, `.mrc`, `.dm3`, `.vsi`, `.svs`, `.ome.tiff`, and more.\n\n**Reference file:** `references\u002Fmicroscopy_imaging_formats.md`\n\n### 4. Spectroscopy and Analytical Chemistry Formats (35+ extensions)\nNMR, mass spectrometry, IR\u002FRaman, UV-Vis, X-ray, chromatography, and other analytical techniques.\n\n**File types include:** `.fid`, `.mzML`, `.mzXML`, `.raw`, `.mgf`, `.spc`, `.jdx`, `.xy`, `.cif` (crystallography), `.wdf`, and more.\n\n**Reference file:** `references\u002Fspectroscopy_analytical_formats.md`\n\n### 5. Proteomics and Metabolomics Formats (30+ extensions)\nMass spec proteomics, metabolomics, lipidomics, and multi-omics data.\n\n**File types include:** `.mzML`, `.pepXML`, `.protXML`, `.mzid`, `.mzTab`, `.sky`, `.mgf`, `.msp`, `.h5ad`, and more.\n\n**Reference file:** `references\u002Fproteomics_metabolomics_formats.md`\n\n### 6. General Scientific Data Formats (30+ extensions)\nArrays, tables, hierarchical data, compressed archives, and common scientific formats.\n\n**File types include:** `.npy`, `.npz`, `.csv`, `.xlsx`, `.json`, `.hdf5`, `.zarr`, `.parquet`, `.mat`, `.fits`, `.nc`, `.xml`, and more.\n\n**Reference file:** `references\u002Fgeneral_scientific_formats.md`\n\n## Workflow\n\n### Step 1: File Type Detection\n\nWhen a user provides a file path, first identify the file type:\n\n1. Extract the file extension\n2. Look up the extension in the appropriate reference file\n3. Identify the file category and format description\n4. Load format-specific information\n\n**Example:**\n```\nUser: \"Analyze data.fastq\"\n→ Extension: .fastq\n→ Category: bioinformatics_genomics\n→ Format: FASTQ Format (sequence data with quality scores)\n→ Reference: references\u002Fbioinformatics_genomics_formats.md\n```\n\n### Step 2: Load Format-Specific Information\n\nBased on the file type, read the corresponding reference file to understand:\n- **Typical Data:** What kind of data this format contains\n- **Use Cases:** Common applications for this format\n- **Python Libraries:** How to read the file in Python\n- **EDA Approach:** What analyses are appropriate for this data type\n\nSearch the reference file for the specific extension (e.g., search for \"### .fastq\" in `bioinformatics_genomics_formats.md`).\n\n### Step 3: Perform Data Analysis\n\nUse the `scripts\u002Feda_analyzer.py` script OR implement custom analysis:\n\n**Option A: Use the analyzer script**\n```python\n# The script automatically:\n# 1. Detects file type\n# 2. Loads reference information\n# 3. Performs format-specific analysis\n# 4. Generates markdown report\n\npython scripts\u002Feda_analyzer.py \u003Cfilepath> [output.md]\n```\n\n**Option B: Custom analysis in the conversation**\nBased on the format information from the reference file, perform appropriate analysis:\n\nFor tabular data (CSV, TSV, Excel):\n- Load with pandas\n- Check dimensions, data types\n- Analyze missing values\n- Calculate summary statistics\n- Identify outliers\n- Check for duplicates\n\nFor sequence data (FASTA, FASTQ):\n- Count sequences\n- Analyze length distributions\n- Calculate GC content\n- Assess quality scores (FASTQ)\n\nFor images (TIFF, ND2, CZI):\n- Check dimensions (X, Y, Z, C, T)\n- Analyze bit depth and value range\n- Extract metadata (channels, timestamps, spatial calibration)\n- Calculate intensity statistics\n\nFor arrays (NPY, HDF5):\n- Check shape and dimensions\n- Analyze data type\n- Calculate statistical summaries\n- Check for missing\u002Finvalid values\n\n### Step 4: Generate Comprehensive Report\n\nCreate a markdown report with the following sections:\n\n#### Required Sections:\n1. **Title and Metadata**\n   - Filename and timestamp\n   - File size and location\n\n2. **Basic Information**\n   - File properties\n   - Format identification\n\n3. **File Type Details**\n   - Format description from reference\n   - Typical data content\n   - Common use cases\n   - Python libraries for reading\n\n4. **Data Analysis**\n   - Structure and dimensions\n   - Statistical summaries\n   - Quality assessment\n   - Data characteristics\n\n5. **Key Findings**\n   - Notable patterns\n   - Potential issues\n   - Quality metrics\n\n6. **Recommendations**\n   - Preprocessing steps\n   - Appropriate analyses\n   - Tools and methods\n   - Visualization approaches\n\n#### Template Location\nUse `assets\u002Freport_template.md` as a guide for report structure.\n\n### Step 5: Save Report\n\nSave the markdown report with a descriptive filename:\n- Pattern: `{original_filename}_eda_report.md`\n- Example: `experiment_data.fastq` → `experiment_data_eda_report.md`\n\n## Detailed Format References\n\nEach reference file contains comprehensive information for dozens of file types. To find information about a specific format:\n\n1. Identify the category from the extension\n2. Read the appropriate reference file\n3. Search for the section heading matching the extension (e.g., \"### .pdb\")\n4. Extract the format information\n\n### Reference File Structure\n\nEach format entry includes:\n- **Description:** What the format is\n- **Typical Data:** What it contains\n- **Use Cases:** Common applications\n- **Python Libraries:** How to read it (with code examples)\n- **EDA Approach:** Specific analyses to perform\n\n**Example lookup:**\n```markdown\n### .pdb - Protein Data Bank\n**Description:** Standard format for 3D structures of biological macromolecules\n**Typical Data:** Atomic coordinates, residue information, secondary structure\n**Use Cases:** Protein structure analysis, molecular visualization, docking\n**Python Libraries:**\n- `Biopython`: `Bio.PDB`\n- `MDAnalysis`: `MDAnalysis.Universe('file.pdb')`\n**EDA Approach:**\n- Structure validation (bond lengths, angles)\n- B-factor distribution\n- Missing residues detection\n- Ramachandran plots\n```\n\n## Best Practices\n\n### Reading Reference Files\n\nReference files are large (10,000+ words each). To efficiently use them:\n\n1. **Search by extension:** Use grep to find the specific format\n   ```python\n   import re\n   with open('references\u002Fchemistry_molecular_formats.md', 'r') as f:\n       content = f.read()\n       pattern = r'### \\.pdb[^#]*?(?=###|\\Z)'\n       match = re.search(pattern, content, re.IGNORECASE | re.DOTALL)\n   ```\n\n2. **Extract relevant sections:** Don't load entire reference files into context unnecessarily\n\n3. **Cache format info:** If analyzing multiple files of the same type, reuse the format information\n\n### Data Analysis\n\n1. **Sample large files:** For files with millions of records, analyze a representative sample\n2. **Handle errors gracefully:** Many scientific formats require specific libraries; provide clear installation instructions\n3. **Validate metadata:** Cross-check metadata consistency (e.g., stated dimensions vs actual data)\n4. **Consider data provenance:** Note instrument, software versions, processing steps\n\n### Report Generation\n\n1. **Be comprehensive:** Include all relevant information for downstream analysis\n2. **Be specific:** Provide concrete recommendations based on the file type\n3. **Be actionable:** Suggest specific next steps and tools\n4. **Include code examples:** Show how to load and work with the data\n\n## Examples\n\n### Example 1: Analyzing a FASTQ file\n\n```python\n# User provides: \"Analyze reads.fastq\"\n\n# 1. Detect file type\nextension = '.fastq'\ncategory = 'bioinformatics_genomics'\n\n# 2. Read reference info\n# Search references\u002Fbioinformatics_genomics_formats.md for \"### .fastq\"\n\n# 3. Perform analysis\nfrom Bio import SeqIO\nsequences = list(SeqIO.parse('reads.fastq', 'fastq'))\n# Calculate: read count, length distribution, quality scores, GC content\n\n# 4. Generate report\n# Include: format description, analysis results, QC recommendations\n\n# 5. Save as: reads_eda_report.md\n```\n\n### Example 2: Analyzing a CSV dataset\n\n```python\n# User provides: \"Explore experiment_results.csv\"\n\n# 1. Detect: .csv → general_scientific\n\n# 2. Load reference for CSV format\n\n# 3. Analyze\nimport pandas as pd\ndf = pd.read_csv('experiment_results.csv')\n# Dimensions, dtypes, missing values, statistics, correlations\n\n# 4. Generate report with:\n# - Data structure\n# - Missing value patterns\n# - Statistical summaries\n# - Correlation matrix\n# - Outlier detection results\n\n# 5. Save report\n```\n\n### Example 3: Analyzing microscopy data\n\n```python\n# User provides: \"Analyze cells.nd2\"\n\n# 1. Detect: .nd2 → microscopy_imaging (Nikon format)\n\n# 2. Read reference for ND2 format\n# Learn: multi-dimensional (XYZCT), requires nd2reader\n\n# 3. Analyze\nfrom nd2reader import ND2Reader\nwith ND2Reader('cells.nd2') as images:\n    # Extract: dimensions, channels, timepoints, metadata\n    # Calculate: intensity statistics, frame info\n\n# 4. Generate report with:\n# - Image dimensions (XY, Z-stacks, time, channels)\n# - Channel wavelengths\n# - Pixel size and calibration\n# - Recommendations for image analysis\n\n# 5. Save report\n```\n\n## Troubleshooting\n\n### Missing Libraries\n\nMany scientific formats require specialized libraries:\n\n**Problem:** Import error when trying to read a file\n\n**Solution:** Provide clear installation instructions\n```python\ntry:\n    from Bio import SeqIO\nexcept ImportError:\n    print(\"Install Biopython: uv pip install biopython\")\n```\n\nCommon requirements by category:\n- **Bioinformatics:** `biopython`, `pysam`, `pyBigWig`\n- **Chemistry:** `rdkit`, `mdanalysis`, `cclib`\n- **Microscopy:** `tifffile`, `nd2reader`, `aicsimageio`, `pydicom`\n- **Spectroscopy:** `nmrglue`, `pymzml`, `pyteomics`\n- **General:** `pandas`, `numpy`, `h5py`, `scipy`\n\n### Unknown File Types\n\nIf a file extension is not in the references:\n\n1. Ask the user about the file format\n2. Check if it's a vendor-specific variant\n3. Attempt generic analysis based on file structure (text vs binary)\n4. Provide general recommendations\n\n### Large Files\n\nFor very large files:\n\n1. Use sampling strategies (first N records)\n2. Use memory-mapped access (for HDF5, NPY)\n3. Process in chunks (for CSV, FASTQ)\n4. Provide estimates based on samples\n\n## Script Usage\n\nThe `scripts\u002Feda_analyzer.py` can be used directly:\n\n```bash\n# Basic usage\npython scripts\u002Feda_analyzer.py data.csv\n\n# Specify output file\npython scripts\u002Feda_analyzer.py data.csv output_report.md\n\n# The script will:\n# 1. Auto-detect file type\n# 2. Load format references\n# 3. Perform appropriate analysis\n# 4. Generate markdown report\n```\n\nThe script supports automatic analysis for many common formats, but custom analysis in the conversation provides more flexibility and domain-specific insights.\n\n## Advanced Usage\n\n### Multi-File Analysis\n\nWhen analyzing multiple related files:\n1. Perform individual EDA on each file\n2. Create a summary comparison report\n3. Identify relationships and dependencies\n4. Suggest integration strategies\n\n### Quality Control\n\nFor data quality assessment:\n1. Check format compliance\n2. Validate metadata consistency\n3. Assess completeness\n4. Identify outliers and anomalies\n5. Compare to expected ranges\u002Fdistributions\n\n### Preprocessing Recommendations\n\nBased on data characteristics, recommend:\n1. Normalization strategies\n2. Missing value imputation\n3. Outlier handling\n4. Batch correction\n5. Format conversions\n\n## Resources\n\n### scripts\u002F\n- `eda_analyzer.py`: Comprehensive analysis script that can be run directly or imported\n\n### references\u002F\n- `chemistry_molecular_formats.md`: 60+ chemistry\u002Fmolecular file formats\n- `bioinformatics_genomics_formats.md`: 50+ bioinformatics formats\n- `microscopy_imaging_formats.md`: 45+ imaging formats\n- `spectroscopy_analytical_formats.md`: 35+ spectroscopy formats\n- `proteomics_metabolomics_formats.md`: 30+ omics formats\n- `general_scientific_formats.md`: 30+ general formats\n\n### assets\u002F\n- `report_template.md`: Comprehensive markdown template for EDA reports\n\n","# Exploratory Data Analysis\n\n## Overview\n\nPerform comprehensive exploratory data analysis (EDA) on scientific data files across multiple domains. This skill provides automated file type detection, format-specific analysis, data quality assessment, and generates detailed markdown reports suitable for documentation and downstream analysis planning.\n\n**Key Capabilities:**\n- Automatic detection and analysis of 200+ scientific file formats\n- Comprehensive format-specific metadata extraction\n- Data quality and integrity assessment\n- Statistical summaries and distributions\n- Visualization recommendations\n- Downstream analysis suggestions\n- Markdown report generation\n\n## When to Use This Skill\n\nUse this skill when:\n- User provides a path to a scientific data file for analysis\n- User asks to \"explore\", \"analyze\", or \"summarize\" a data file\n- User wants to understand the structure and content of scientific data\n- User needs a comprehensive report of a dataset before analysis\n- User wants to assess data quality or completeness\n- User asks what type of analysis is appropriate for a file\n\n## Supported File Categories\n\nThe skill has comprehensive coverage of scientific file formats organized into six major categories:\n\n### 1. Chemistry and Molecular Formats (60+ extensions)\nStructure files, computational chemistry outputs, molecular dynamics trajectories, and chemical databases.\n\n**File types include:** `.pdb`, `.cif`, `.mol`, `.mol2`, `.sdf`, `.xyz`, `.smi`, `.gro`, `.log`, `.fchk`, `.cube`, `.dcd`, `.xtc`, `.trr`, `.prmtop`, `.psf`, and more.\n\n**Reference file:** `references\u002Fchemistry_molecular_formats.md`\n\n### 2. Bioinformatics and Genomics Formats (50+ extensions)\nSequence data, alignments, annotations, variants, and expression data.\n\n**File types include:** `.fasta`, `.fastq`, `.sam`, `.bam`, `.vcf`, `.bed`, `.gff`, `.gtf`, `.bigwig`, `.h5ad`, `.loom`, `.counts`, `.mtx`, and more.\n\n**Reference file:** `references\u002Fbioinformatics_genomics_formats.md`\n\n### 3. Microscopy and Imaging Formats (45+ extensions)\nMicroscopy images, medical imaging, whole slide imaging, and electron microscopy.\n\n**File types include:** `.tif`, `.nd2`, `.lif`, `.czi`, `.ims`, `.dcm`, `.nii`, `.mrc`, `.dm3`, `.vsi`, `.svs`, `.ome.tiff`, and more.\n\n**Reference file:** `references\u002Fmicroscopy_imaging_formats.md`\n\n### 4. Spectroscopy and Analytical Chemistry Formats (35+ extensions)\nNMR, mass spectrometry, IR\u002FRaman, UV-Vis, X-ray, chromatography, and other analytical techniques.\n\n**File types include:** `.fid`, `.mzML`, `.mzXML`, `.raw`, `.mgf`, `.spc`, `.jdx`, `.xy`, `.cif` (crystallography), `.wdf`, and more.\n\n**Reference file:** `references\u002Fspectroscopy_analytical_formats.md`\n\n### 5. Proteomics and Metabolomics Formats (30+ extensions)\nMass spec proteomics, metabolomics, lipidomics, and multi-omics data.\n\n**File types include:** `.mzML`, `.pepXML`, `.protXML`, `.mzid`, `.mzTab`, `.sky`, `.mgf`, `.msp`, `.h5ad`, and more.\n\n**Reference file:** `references\u002Fproteomics_metabolomics_formats.md`\n\n### 6. General Scientific Data Formats (30+ extensions)\nArrays, tables, hierarchical data, compressed archives, and common scientific formats.\n\n**File types include:** `.npy`, `.npz`, `.csv`, `.xlsx`, `.json`, `.hdf5`, `.zarr`, `.parquet`, `.mat`, `.fits`, `.nc`, `.xml`, and more.\n\n**Reference file:** `references\u002Fgeneral_scientific_formats.md`\n\n## Workflow\n\n### Step 1: File Type Detection\n\nWhen a user provides a file path, first identify the file type:\n\n1. Extract the file extension\n2. Look up the extension in the appropriate reference file\n3. Identify the file category and format description\n4. Load format-specific information\n\n**Example:**\n```\nUser: \"Analyze data.fastq\"\n→ Extension: .fastq\n→ Category: bioinformatics_genomics\n→ Format: FASTQ Format (sequence data with quality scores)\n→ Reference: references\u002Fbioinformatics_genomics_formats.md\n```\n\n### Step 2: Load Format-Specific Information\n\nBased on the file type, read the corresponding reference file to understand:\n- **Typical Data:** What kind of data this format contains\n- **Use Cases:** Common applications for this format\n- **Python Libraries:** How to read the file in Python\n- **EDA Approach:** What analyses are appropriate for this data type\n\nSearch the reference file for the specific extension (e.g., search for \"### .fastq\" in `bioinformatics_genomics_formats.md`).\n\n### Step 3: Perform Data Analysis\n\nUse the `scripts\u002Feda_analyzer.py` script OR implement custom analysis:\n\n**Option A: Use the analyzer script**\n```python\n# The script automatically:\n# 1. Detects file type\n# 2. Loads reference information\n# 3. Performs format-specific analysis\n# 4. Generates markdown report\n\npython scripts\u002Feda_analyzer.py \u003Cfilepath> [output.md]\n```\n\n**Option B: Custom analysis in the conversation**\nBased on the format information from the reference file, perform appropriate analysis:\n\nFor tabular data (CSV, TSV, Excel):\n- Load with pandas\n- Check dimensions, data types\n- Analyze missing values\n- Calculate summary statistics\n- Identify outliers\n- Check for duplicates\n\nFor sequence data (FASTA, FASTQ):\n- Count sequences\n- Analyze length distributions\n- Calculate GC content\n- Assess quality scores (FASTQ)\n\nFor images (TIFF, ND2, CZI):\n- Check dimensions (X, Y, Z, C, T)\n- Analyze bit depth and value range\n- Extract metadata (channels, timestamps, spatial calibration)\n- Calculate intensity statistics\n\nFor arrays (NPY, HDF5):\n- Check shape and dimensions\n- Analyze data type\n- Calculate statistical summaries\n- Check for missing\u002Finvalid values\n\n### Step 4: Generate Comprehensive Report\n\nCreate a markdown report with the following sections:\n\n#### Required Sections:\n1. **Title and Metadata**\n   - Filename and timestamp\n   - File size and location\n\n2. **Basic Information**\n   - File properties\n   - Format identification\n\n3. **File Type Details**\n   - Format description from reference\n   - Typical data content\n   - Common use cases\n   - Python libraries for reading\n\n4. **Data Analysis**\n   - Structure and dimensions\n   - Statistical summaries\n   - Quality assessment\n   - Data characteristics\n\n5. **Key Findings**\n   - Notable patterns\n   - Potential issues\n   - Quality metrics\n\n6. **Recommendations**\n   - Preprocessing steps\n   - Appropriate analyses\n   - Tools and methods\n   - Visualization approaches\n\n#### Template Location\nUse `assets\u002Freport_template.md` as a guide for report structure.\n\n### Step 5: Save Report\n\nSave the markdown report with a descriptive filename:\n- Pattern: `{original_filename}_eda_report.md`\n- Example: `experiment_data.fastq` → `experiment_data_eda_report.md`\n\n## Detailed Format References\n\nEach reference file contains comprehensive information for dozens of file types. To find information about a specific format:\n\n1. Identify the category from the extension\n2. Read the appropriate reference file\n3. Search for the section heading matching the extension (e.g., \"### .pdb\")\n4. Extract the format information\n\n### Reference File Structure\n\nEach format entry includes:\n- **Description:** What the format is\n- **Typical Data:** What it contains\n- **Use Cases:** Common applications\n- **Python Libraries:** How to read it (with code examples)\n- **EDA Approach:** Specific analyses to perform\n\n**Example lookup:**\n```markdown\n### .pdb - Protein Data Bank\n**Description:** Standard format for 3D structures of biological macromolecules\n**Typical Data:** Atomic coordinates, residue information, secondary structure\n**Use Cases:** Protein structure analysis, molecular visualization, docking\n**Python Libraries:**\n- `Biopython`: `Bio.PDB`\n- `MDAnalysis`: `MDAnalysis.Universe('file.pdb')`\n**EDA Approach:**\n- Structure validation (bond lengths, angles)\n- B-factor distribution\n- Missing residues detection\n- Ramachandran plots\n```\n\n## Best Practices\n\n### Reading Reference Files\n\nReference files are large (10,000+ words each). To efficiently use them:\n\n1. **Search by extension:** Use grep to find the specific format\n   ```python\n   import re\n   with open('references\u002Fchemistry_molecular_formats.md', 'r') as f:\n       content = f.read()\n       pattern = r'### \\.pdb[^#]*?(?=###|\\Z)'\n       match = re.search(pattern, content, re.IGNORECASE | re.DOTALL)\n   ```\n\n2. **Extract relevant sections:** Don't load entire reference files into context unnecessarily\n\n3. **Cache format info:** If analyzing multiple files of the same type, reuse the format information\n\n### Data Analysis\n\n1. **Sample large files:** For files with millions of records, analyze a representative sample\n2. **Handle errors gracefully:** Many scientific formats require specific libraries; provide clear installation instructions\n3. **Validate metadata:** Cross-check metadata consistency (e.g., stated dimensions vs actual data)\n4. **Consider data provenance:** Note instrument, software versions, processing steps\n\n### Report Generation\n\n1. **Be comprehensive:** Include all relevant information for downstream analysis\n2. **Be specific:** Provide concrete recommendations based on the file type\n3. **Be actionable:** Suggest specific next steps and tools\n4. **Include code examples:** Show how to load and work with the data\n\n## Examples\n\n### Example 1: Analyzing a FASTQ file\n\n```python\n# User provides: \"Analyze reads.fastq\"\n\n# 1. Detect file type\nextension = '.fastq'\ncategory = 'bioinformatics_genomics'\n\n# 2. Read reference info\n# Search references\u002Fbioinformatics_genomics_formats.md for \"### .fastq\"\n\n# 3. Perform analysis\nfrom Bio import SeqIO\nsequences = list(SeqIO.parse('reads.fastq', 'fastq'))\n# Calculate: read count, length distribution, quality scores, GC content\n\n# 4. Generate report\n# Include: format description, analysis results, QC recommendations\n\n# 5. Save as: reads_eda_report.md\n```\n\n### Example 2: Analyzing a CSV dataset\n\n```python\n# User provides: \"Explore experiment_results.csv\"\n\n# 1. Detect: .csv → general_scientific\n\n# 2. Load reference for CSV format\n\n# 3. Analyze\nimport pandas as pd\ndf = pd.read_csv('experiment_results.csv')\n# Dimensions, dtypes, missing values, statistics, correlations\n\n# 4. Generate report with:\n# - Data structure\n# - Missing value patterns\n# - Statistical summaries\n# - Correlation matrix\n# - Outlier detection results\n\n# 5. Save report\n```\n\n### Example 3: Analyzing microscopy data\n\n```python\n# User provides: \"Analyze cells.nd2\"\n\n# 1. Detect: .nd2 → microscopy_imaging (Nikon format)\n\n# 2. Read reference for ND2 format\n# Learn: multi-dimensional (XYZCT), requires nd2reader\n\n# 3. Analyze\nfrom nd2reader import ND2Reader\nwith ND2Reader('cells.nd2') as images:\n    # Extract: dimensions, channels, timepoints, metadata\n    # Calculate: intensity statistics, frame info\n\n# 4. Generate report with:\n# - Image dimensions (XY, Z-stacks, time, channels)\n# - Channel wavelengths\n# - Pixel size and calibration\n# - Recommendations for image analysis\n\n# 5. Save report\n```\n\n## Troubleshooting\n\n### Missing Libraries\n\nMany scientific formats require specialized libraries:\n\n**Problem:** Import error when trying to read a file\n\n**Solution:** Provide clear installation instructions\n```python\ntry:\n    from Bio import SeqIO\nexcept ImportError:\n    print(\"Install Biopython: uv pip install biopython\")\n```\n\nCommon requirements by category:\n- **Bioinformatics:** `biopython`, `pysam`, `pyBigWig`\n- **Chemistry:** `rdkit`, `mdanalysis`, `cclib`\n- **Microscopy:** `tifffile`, `nd2reader`, `aicsimageio`, `pydicom`\n- **Spectroscopy:** `nmrglue`, `pymzml`, `pyteomics`\n- **General:** `pandas`, `numpy`, `h5py`, `scipy`\n\n### Unknown File Types\n\nIf a file extension is not in the references:\n\n1. Ask the user about the file format\n2. Check if it's a vendor-specific variant\n3. Attempt generic analysis based on file structure (text vs binary)\n4. Provide general recommendations\n\n### Large Files\n\nFor very large files:\n\n1. Use sampling strategies (first N records)\n2. Use memory-mapped access (for HDF5, NPY)\n3. Process in chunks (for CSV, FASTQ)\n4. Provide estimates based on samples\n\n## Script Usage\n\nThe `scripts\u002Feda_analyzer.py` can be used directly:\n\n```bash\n# Basic usage\npython scripts\u002Feda_analyzer.py data.csv\n\n# Specify output file\npython scripts\u002Feda_analyzer.py data.csv output_report.md\n\n# The script will:\n# 1. Auto-detect file type\n# 2. Load format references\n# 3. Perform appropriate analysis\n# 4. Generate markdown report\n```\n\nThe script supports automatic analysis for many common formats, but custom analysis in the conversation provides more flexibility and domain-specific insights.\n\n## Advanced Usage\n\n### Multi-File Analysis\n\nWhen analyzing multiple related files:\n1. Perform individual EDA on each file\n2. Create a summary comparison report\n3. Identify relationships and dependencies\n4. Suggest integration strategies\n\n### Quality Control\n\nFor data quality assessment:\n1. Check format compliance\n2. Validate metadata consistency\n3. Assess completeness\n4. Identify outliers and anomalies\n5. Compare to expected ranges\u002Fdistributions\n\n### Preprocessing Recommendations\n\nBased on data characteristics, recommend:\n1. Normalization strategies\n2. Missing value imputation\n3. Outlier handling\n4. Batch correction\n5. Format conversions\n\n## Resources\n\n### scripts\u002F\n- `eda_analyzer.py`: Comprehensive analysis script that can be run directly or imported\n\n### references\u002F\n- `chemistry_molecular_formats.md`: 60+ chemistry\u002Fmolecular file formats\n- `bioinformatics_genomics_formats.md`: 50+ bioinformatics formats\n- `microscopy_imaging_formats.md`: 45+ imaging formats\n- `spectroscopy_analytical_formats.md`: 35+ spectroscopy formats\n- `proteomics_metabolomics_formats.md`: 30+ omics formats\n- `general_scientific_formats.md`: 30+ general formats\n\n### assets\u002F\n- `report_template.md`: Comprehensive markdown template for EDA reports\n\n",{"name":5,"license":57,"metadata":58,"description":7},"MIT license",{"version":59,"skill-author":60},"1.0","K-Dense Inc.","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","scientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":68,"prompt":78,"zipAvailable":79},[69,72,75],{"tool":70,"command":71},"npx","npx skills add fadrienne\u002FMyGitHub --skill exploratory-data-analysis",{"tool":73,"command":74},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill exploratory-data-analysis",{"tool":76,"command":77},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill exploratory-data-analysis","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002FSKILL.md",true,[81,87,93,97,101,105,109,113],{"path":82,"resourceType":83,"resourceTypeDesc":84,"sizeBytes":85,"sourceHtmlUrl":86},"assets\u002Freport_template.md","ASSET","资产文件",3488,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Fassets\u002Freport_template.md",{"path":88,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":91,"sourceHtmlUrl":92},"references\u002Fbioinformatics_genomics_formats.md","REFERENCE","参考文档",21163,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fbioinformatics_genomics_formats.md",{"path":94,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":95,"sourceHtmlUrl":96},"references\u002Fchemistry_molecular_formats.md",22029,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fchemistry_molecular_formats.md",{"path":98,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":99,"sourceHtmlUrl":100},"references\u002Fgeneral_scientific_formats.md",15287,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fgeneral_scientific_formats.md",{"path":102,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":103,"sourceHtmlUrl":104},"references\u002Fmicroscopy_imaging_formats.md",18549,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fmicroscopy_imaging_formats.md",{"path":106,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":107,"sourceHtmlUrl":108},"references\u002Fproteomics_metabolomics_formats.md",15543,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fproteomics_metabolomics_formats.md",{"path":110,"resourceType":89,"resourceTypeDesc":90,"sizeBytes":111,"sourceHtmlUrl":112},"references\u002Fspectroscopy_analytical_formats.md",18073,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Freferences\u002Fspectroscopy_analytical_formats.md",{"path":114,"resourceType":115,"resourceTypeDesc":116,"sizeBytes":117,"sourceHtmlUrl":118},"scripts\u002Feda_analyzer.py","SCRIPT","脚本",21318,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fexploratory-data-analysis\u002Fscripts\u002Feda_analyzer.py",[120,146,171,201,226,253],{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":127,"publishedAt":52},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[126],{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[128,129,130,131,132,133,134,135,37,136,137,34,138,139,140,141,142,143,144,145],"power","statistical power","effect","scipy","regression","choose","design","uv","Bash","collection","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":147,"name":148,"nameZh":6,"description":149,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":150,"categories":151,"keywords":155,"publishedAt":52},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[152],{"categoryNo":153,"categoryName":154,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[34,156,128,157,158,159,37,160,161,32,162,163,164,39,165,166,167,168,169,170],"statistical analysis","academic research","implementing","selection","you","reporting","for","tests","use","test","research","results","statsmodels","models","help",{"skillNo":172,"name":173,"nameZh":6,"description":174,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":175,"categories":176,"keywords":184,"publishedAt":52},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[177,180,181],{"categoryNo":178,"categoryName":179,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":182,"categoryName":183,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000036","内容创作者",[185,40,186,187,188,189,190,191,192,193,136,194,195,45,196,197,198,199,161,200],"writing","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","openrouter","Outlines","core skill","read","features",{"skillNo":202,"name":203,"nameZh":6,"description":204,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":205,"categories":206,"keywords":209,"publishedAt":52},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[207,208],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[210,40,211,212,213,214,215,216,217,192,37,218,219,220,221,222,223,224,162,225],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":227,"name":228,"nameZh":6,"description":229,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":230,"categories":231,"keywords":238,"publishedAt":52},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[232,235],{"categoryNo":233,"categoryName":234,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":236,"categoryName":237,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000030","开发工程师",[239,240,166,241,242,243,244,135,245,37,136,34,246,247,248,249,250,251,199,252],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":254,"name":255,"nameZh":6,"description":256,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":257,"licenseFileUrl":6,"sourceHtmlUrl":258,"categories":259,"keywords":266,"publishedAt":52},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[260,263,264,265],{"categoryNo":261,"categoryName":262,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":178,"categoryName":179,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":182,"categoryName":183,"taxonomyType":29,"taxonomyTypeDesc":30},[185,267,268,48,269,270,210,271,272,37,136,34,273,274,275,276,277,278,40,199],"markdown mermaid writing","mermaid","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]