[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6KJ6WGAATAVP395YDW7VT9R":3,"skill-same:01M6KJ6WGAATAVP395YDW7VT9R":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":37,"linkOnly":38,"markdownContent":39,"markdownBody":40,"frontMatter":41,"refName":42,"commitSha":43,"skillDirectory":44,"forksCount":13,"repositoryHtmlUrl":45,"repositoryHomepageUrl":6,"ownerHtmlUrl":46,"ownerBlogUrl":6,"ownerDescription":47,"updatedAt":48,"install":49,"resources":62},"01M6KJ6WGAATAVP395YDW7VT9R","irregular-excel-parsing",null,"Handle Excel files with irregular headers, merged cells, and unknown header row positions using pattern-matching and index-based extraction.","AG064\u002FOpenSpace","AG064","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F184930299?v=4","01M6KHVFG7TB73NHX3RDYSTMH2","01M6KHVFMM7XA2PZA9Z83PHS3V",0,"MIT","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000004","电子表格","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36],"xlsx","parsing","excel parsing","irregular excel parsing","index","extraction","pattern-matching","files","pattern","positions","matching","with","headers","unknown","2026-11-10 10:12:30",false,"---\nname: irregular-excel-parsing\ndescription: Handle Excel files with irregular headers, merged cells, and unknown header row positions using pattern-matching and index-based extraction.\n---\n\n# Irregular Excel File Parsing\n\nUse this skill when you encounter Excel files where standard `pandas.read_excel()` fails due to:\n- Headers not in row 0 (6-9+ header rows common)\n- Merged cells in header area\n- Unknown or inconsistent header row positions\n- Multiple title\u002Fmetadata rows before actual data\n\n## Step-by-Step Instructions\n\n### Step 1: Read Excel Without Headers\n\nFirst, read the entire sheet with `header=None` to get raw data:\n\n```python\nimport pandas as pd\n\n# Read all data without assuming header position\ndf_raw = pd.read_excel('filename.xlsx', sheet_name='Sheet1', header=None)\n```\n\n### Step 2: Scan Rows to Find Header Pattern\n\nSearch for the actual header row by looking for distinctive patterns:\n\n```python\ndef find_header_row(df):\n    \"\"\"Find header row by pattern-matching common column identifiers.\"\"\"\n    header_patterns = [\n        r'Store ID',\n        r'ID\\d{4}',  # ID followed by 4 digits\n        r'Week \\d+',\n        r'Date',\n        r'Store',\n        r'Product',\n        r'ID'\n    ]\n    \n    for row_idx in range(len(df)):\n        row_values = df.iloc[row_idx].astype(str).str.lower()\n        for pattern in header_patterns:\n            if row_values.str.contains(pattern, case=False, regex=True).any():\n                return row_idx\n    \n    # Fallback: return first non-empty row\n    for row_idx in range(len(df)):\n        if df.iloc[row_idx].notna().sum() > 0:\n            return row_idx\n    \n    return 0\n\nheader_row = find_header_row(df_raw)\n```\n\n### Step 3: Extract and Clean Headers\n\nExtract the header row and clean column names:\n\n```python\n# Extract header row\nheaders = df_raw.iloc[header_row].tolist()\n\n# Clean headers: convert to string, strip whitespace, handle NaN\nclean_headers = []\nfor h in headers:\n    if pd.isna(h) or str(h).strip() == '':\n        clean_headers.append(f'col_{len(clean_headers)}')\n    else:\n        clean_headers.append(str(h).strip())\n\n# Handle duplicate headers by adding suffix\nfrom collections import Counter\nheader_counts = Counter(clean_headers)\nfinal_headers = []\nfor h in clean_headers:\n    if header_counts[h] > 1:\n        final_headers.append(f\"{h}_{header_counts[h]}\")\n        header_counts[h] -= 1\n    else:\n        final_headers.append(h)\n```\n\n### Step 4: Extract Data Rows\n\nExtract data starting from the row after headers:\n\n```python\n# Get data rows (everything after header)\ndata_df = df_raw.iloc[header_row + 1:].copy()\ndata_df.columns = final_headers\n\n# Reset index\ndata_df = data_df.reset_index(drop=True)\n\n# Remove completely empty rows\ndata_df = data_df.dropna(how='all')\n```\n\n### Step 5: Validate and Clean Data\n\nPerform basic validation and type conversion:\n\n```python\n# Identify ID columns and preserve as string\nfor col in data_df.columns:\n    if 'id' in col.lower() or 'code' in col.lower():\n        data_df[col] = data_df[col].astype(str).str.strip()\n\n# Convert numeric columns\nnumeric_cols = data_df.select_dtypes(include=['float64', 'int64']).columns\nfor col in numeric_cols:\n    data_df[col] = pd.to_numeric(data_df[col], errors='coerce')\n\n# Remove rows with invalid critical data\nif 'Store ID' in data_df.columns:\n    data_df = data_df[data_df['Store ID'].notna() & (data_df['Store ID'] != '')]\n```\n\n## Complete Example Function\n\n```python\ndef parse_irregular_excel(filepath, sheet_name=0):\n    \"\"\"Parse Excel file with unknown\u002Firregular header structure.\"\"\"\n    import pandas as pd\n    import re\n    from collections import Counter\n    \n    # Step 1: Read raw\n    df_raw = pd.read_excel(filepath, sheet_name=sheet_name, header=None)\n    \n    # Step 2: Find header row\n    patterns = [r'Store ID', r'ID\\d{4}', r'Week', r'Date', r'Store']\n    header_row = 0\n    for idx in range(min(15, len(df_raw))):  # Check first 15 rows\n        row_str = ' '.join(df_raw.iloc[idx].astype(str))\n        for pattern in patterns:\n            if re.search(pattern, row_str, re.IGNORECASE):\n                header_row = idx\n                break\n    \n    # Step 3: Extract headers\n    headers = [str(h).strip() if pd.notna(h) else f'col_{i}' \n               for i, h in enumerate(df_raw.iloc[header_row])]\n    \n    # Handle duplicates\n    counts = Counter(headers)\n    final_headers = []\n    for h in headers:\n        if counts[h] > 1:\n            final_headers.append(f\"{h}_{counts[h]}\")\n            counts[h] -= 1\n        else:\n            final_headers.append(h)\n    \n    # Step 4: Extract data\n    data_df = df_raw.iloc[header_row + 1:].copy()\n    data_df.columns = final_headers\n    data_df = data_df.dropna(how='all').reset_index(drop=True)\n    \n    return data_df\n```\n\n## When to Use This Pattern\n\n- ✅ Excel files with 6-9+ title\u002Fmetadata rows before data\n- ✅ Merged cells in header area causing misalignment\n- ✅ Headers not in predictable positions\n- ✅ Multiple sheets with inconsistent structures\n\n## When NOT to Use\n\n- ❌ Standard Excel files with headers in row 0 (use `read_excel()` directly)\n- ❌ Files with consistent, known structure (use explicit `header=` parameter)\n- ❌ When you know exact header row position (specify it directly)\n\n## Tips\n\n1. **Always inspect first**: Use `df_raw.head(20)` to visualize structure before parsing\n2. **Pattern flexibility**: Adjust regex patterns based on your specific column naming conventions\n3. **Handle merged cells**: Merged cells often result in NaN values - fill strategically if needed\n4. **Save for reuse**: Once you determine the correct header row for a file type, cache this for future runs","# Irregular Excel File Parsing\n\nUse this skill when you encounter Excel files where standard `pandas.read_excel()` fails due to:\n- Headers not in row 0 (6-9+ header rows common)\n- Merged cells in header area\n- Unknown or inconsistent header row positions\n- Multiple title\u002Fmetadata rows before actual data\n\n## Step-by-Step Instructions\n\n### Step 1: Read Excel Without Headers\n\nFirst, read the entire sheet with `header=None` to get raw data:\n\n```python\nimport pandas as pd\n\n# Read all data without assuming header position\ndf_raw = pd.read_excel('filename.xlsx', sheet_name='Sheet1', header=None)\n```\n\n### Step 2: Scan Rows to Find Header Pattern\n\nSearch for the actual header row by looking for distinctive patterns:\n\n```python\ndef find_header_row(df):\n    \"\"\"Find header row by pattern-matching common column identifiers.\"\"\"\n    header_patterns = [\n        r'Store ID',\n        r'ID\\d{4}',  # ID followed by 4 digits\n        r'Week \\d+',\n        r'Date',\n        r'Store',\n        r'Product',\n        r'ID'\n    ]\n    \n    for row_idx in range(len(df)):\n        row_values = df.iloc[row_idx].astype(str).str.lower()\n        for pattern in header_patterns:\n            if row_values.str.contains(pattern, case=False, regex=True).any():\n                return row_idx\n    \n    # Fallback: return first non-empty row\n    for row_idx in range(len(df)):\n        if df.iloc[row_idx].notna().sum() > 0:\n            return row_idx\n    \n    return 0\n\nheader_row = find_header_row(df_raw)\n```\n\n### Step 3: Extract and Clean Headers\n\nExtract the header row and clean column names:\n\n```python\n# Extract header row\nheaders = df_raw.iloc[header_row].tolist()\n\n# Clean headers: convert to string, strip whitespace, handle NaN\nclean_headers = []\nfor h in headers:\n    if pd.isna(h) or str(h).strip() == '':\n        clean_headers.append(f'col_{len(clean_headers)}')\n    else:\n        clean_headers.append(str(h).strip())\n\n# Handle duplicate headers by adding suffix\nfrom collections import Counter\nheader_counts = Counter(clean_headers)\nfinal_headers = []\nfor h in clean_headers:\n    if header_counts[h] > 1:\n        final_headers.append(f\"{h}_{header_counts[h]}\")\n        header_counts[h] -= 1\n    else:\n        final_headers.append(h)\n```\n\n### Step 4: Extract Data Rows\n\nExtract data starting from the row after headers:\n\n```python\n# Get data rows (everything after header)\ndata_df = df_raw.iloc[header_row + 1:].copy()\ndata_df.columns = final_headers\n\n# Reset index\ndata_df = data_df.reset_index(drop=True)\n\n# Remove completely empty rows\ndata_df = data_df.dropna(how='all')\n```\n\n### Step 5: Validate and Clean Data\n\nPerform basic validation and type conversion:\n\n```python\n# Identify ID columns and preserve as string\nfor col in data_df.columns:\n    if 'id' in col.lower() or 'code' in col.lower():\n        data_df[col] = data_df[col].astype(str).str.strip()\n\n# Convert numeric columns\nnumeric_cols = data_df.select_dtypes(include=['float64', 'int64']).columns\nfor col in numeric_cols:\n    data_df[col] = pd.to_numeric(data_df[col], errors='coerce')\n\n# Remove rows with invalid critical data\nif 'Store ID' in data_df.columns:\n    data_df = data_df[data_df['Store ID'].notna() & (data_df['Store ID'] != '')]\n```\n\n## Complete Example Function\n\n```python\ndef parse_irregular_excel(filepath, sheet_name=0):\n    \"\"\"Parse Excel file with unknown\u002Firregular header structure.\"\"\"\n    import pandas as pd\n    import re\n    from collections import Counter\n    \n    # Step 1: Read raw\n    df_raw = pd.read_excel(filepath, sheet_name=sheet_name, header=None)\n    \n    # Step 2: Find header row\n    patterns = [r'Store ID', r'ID\\d{4}', r'Week', r'Date', r'Store']\n    header_row = 0\n    for idx in range(min(15, len(df_raw))):  # Check first 15 rows\n        row_str = ' '.join(df_raw.iloc[idx].astype(str))\n        for pattern in patterns:\n            if re.search(pattern, row_str, re.IGNORECASE):\n                header_row = idx\n                break\n    \n    # Step 3: Extract headers\n    headers = [str(h).strip() if pd.notna(h) else f'col_{i}' \n               for i, h in enumerate(df_raw.iloc[header_row])]\n    \n    # Handle duplicates\n    counts = Counter(headers)\n    final_headers = []\n    for h in headers:\n        if counts[h] > 1:\n            final_headers.append(f\"{h}_{counts[h]}\")\n            counts[h] -= 1\n        else:\n            final_headers.append(h)\n    \n    # Step 4: Extract data\n    data_df = df_raw.iloc[header_row + 1:].copy()\n    data_df.columns = final_headers\n    data_df = data_df.dropna(how='all').reset_index(drop=True)\n    \n    return data_df\n```\n\n## When to Use This Pattern\n\n- ✅ Excel files with 6-9+ title\u002Fmetadata rows before data\n- ✅ Merged cells in header area causing misalignment\n- ✅ Headers not in predictable positions\n- ✅ Multiple sheets with inconsistent structures\n\n## When NOT to Use\n\n- ❌ Standard Excel files with headers in row 0 (use `read_excel()` directly)\n- ❌ Files with consistent, known structure (use explicit `header=` parameter)\n- ❌ When you know exact header row position (specify it directly)\n\n## Tips\n\n1. **Always inspect first**: Use `df_raw.head(20)` to visualize structure before parsing\n2. **Pattern flexibility**: Adjust regex patterns based on your specific column naming conventions\n3. **Handle merged cells**: Merged cells often result in NaN values - fill strategically if needed\n4. **Save for reuse**: Once you determine the correct header row for a file type, cache this for future runs",{"name":5,"description":7},"main","ff671ef61ed6587c49bd54c81cdc751c0b99e363","gdpval_bench\u002Fskills\u002Firregular-excel-parsing","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace","https:\u002F\u002Fgithub.com\u002FAG064","AG064, E.Mi.\r\n\r\nA person who thinks all the time, yet all they have to think about is nothing. I want to improve the world and work on open source projects","2026-10-08 06:53:48",{"source":8,"skillName":5,"commands":50,"prompt":60,"zipAvailable":61},[51,54,57],{"tool":52,"command":53},"npx","npx skills add AG064\u002FOpenSpace --skill irregular-excel-parsing",{"tool":55,"command":56},"bunx","bunx skills add AG064\u002FOpenSpace --skill irregular-excel-parsing",{"tool":58,"command":59},"pnpm","pnpm dlx skills add AG064\u002FOpenSpace --skill irregular-excel-parsing","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002FSKILL.md",true,[63],{"path":64,"resourceType":65,"resourceTypeDesc":66,"sizeBytes":67,"sourceHtmlUrl":68},".skill_id","OTHER","其他",37,"https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002F.skill_id",[70,93,109,118,148,164],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":37},"01M6KKMPWA0B5SCCH766F8M5EA","write-special-chars-ts-check","Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, plus a reliable TypeScript type-checking step that filters output to only the newly-written files to avoid being misled by pre-existing errors.","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fwrite-special-chars-ts-check\u002FSKILL.md",[],[77,78,79,80,81,82,83,84,85,86,87,30,88,89,90,91,92],"write special chars ts check","check","write","filters","Existing","type-checking","for","errors","script","output","type","only","template","file","python","typescript",{"skillNo":94,"name":95,"nameZh":6,"description":96,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":37},"01M6KKMJXVBNVEY3TRK055F4A7","write-special-chars-readback","Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, with mandatory post-write verification via read_file and cleanup of \u002Ftmp helper scripts.","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fwrite-special-chars-readback\u002FSKILL.md",[],[100,101,79,89,102,103,90,91,104,105,106,107,108,83,85,34],"readback","write special chars readback","cleanup","read","scripts","verification","via","post","helper",{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":115,"publishedAt":37},"01M6KKMEGX9M9NC105J0W3Q5B9","write-special-chars-heredoc","Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) by using a Python heredoc script executed via shell.","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fwrite-special-chars-heredoc\u002FSKILL.md",[],[79,116,91,83,106,89,117,90,85],"write special chars heredoc","shell",{"skillNo":119,"name":120,"nameZh":6,"description":121,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":122,"categories":123,"keywords":129,"publishedAt":37},"01M6KKM8EJ2AATVHZBNB916WT0","worldmonitor-reference","Curated architecture index of the WorldMonitor open-source dashboard. Reference for extracting patterns — vanilla TypeScript, Panel class hierarchy, service layer, CSS grid, API edge functions, refresh scheduler, and 60+ data source integrations.","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fworldmonitor-reference\u002FSKILL.md",[124],{"categoryNo":125,"categoryName":126,"taxonomyType":127,"taxonomyTypeDesc":128},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[130,131,132,133,92,134,135,136,137,138,83,139,140,141,142,143,144,145,146,147],"reference","worldmonitor","worldmonitor reference","dashboard","css","architecture","extracting","hierarchy","source","layer","panel","patterns","edge functions","integrations","open","api","data","scheduler",{"skillNo":149,"name":150,"nameZh":6,"description":151,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":152,"categories":153,"keywords":154,"publishedAt":37},"01M6KKM39PHYK627F19GE2B9XN","virtual-list","Implement efficient virtual scrolling for rendering large lists with DOM recycling, chunk-based rendering, and performance optimizations","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fvirtual-list\u002FSKILL.md",[],[155,156,157,158,83,159,34,160,161,162,163],"virtual list","list","virtual","chunk","dom","rendering","recycling","implement","performance",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":170,"publishedAt":37},"01M6KKKY7T2P61Y0JH54CDEJKQ","verify-directory-structure","Diagnose and resolve file access issues by first verifying the directory structure.","https:\u002F\u002Fgithub.com\u002FAG064\u002FOpenSpace\u002Fblob\u002Fmain\u002Fshowcase\u002Fskills\u002Fverify-directory-structure\u002FSKILL.md",[],[171,172,173,174,175,176,90,177,178,179],"directory","verify","structure","verify directory structure","access","first","issues","diagnose","file access"]