[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6WWSCH4QGCZ4KQHDEB3PRR8":3,"skill-same:01M6WWSCH4QGCZ4KQHDEB3PRR8":84},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":57,"commitSha":58,"skillDirectory":59,"forksCount":14,"repositoryHtmlUrl":60,"repositoryHomepageUrl":61,"ownerHtmlUrl":62,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":63,"install":64,"resources":77},"01M6WWSCH4QGCZ4KQHDEB3PRR8","pandas-polars",null,"DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.","projectious-work\u002Fprocesskit","projectious-work","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F263889726?v=4","01M6N97TJJ118WQ35CHRV8RD0E","projectious.work","01M6N97TPWJPF0BMKYQ4PBY5W8",0,"MIT","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.10.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"polars","pandas","pandas polars","performance","libraries","code","dataframe","data","work","operations","use","with","v1","tabular","when","2026-11-10 10:12:30",false,"---\nname: pandas-polars\ndescription: |\n  DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.\nmetadata:\n  processkit:\n    apiVersion: processkit.projectious.work\u002Fv1\n    id: SKILL-pandas-polars\n    version: \"1.1.0\"\n    created: 2026-04-06T00:00:00Z\n    category: data-ai\n---\n\n# Pandas and Polars\n\n## Intro\n\nPandas is the mature, broad-ecosystem default for exploratory work\nunder ~1 GB; polars is faster, leaner, and lazy-evaluated for\nproduction pipelines and larger data. Both read the same formats —\nwhen in doubt, match what the project already uses.\n\n## Overview\n\n### Choosing between them\n\n- **Pandas** — mature ecosystem, broader library compatibility\n  (sklearn, plotly), familiar API, best for EDA and \u003C 1 GB datasets\n- **Polars** — faster execution, lower memory, lazy evaluation,\n  better for production pipelines and 1 GB+ datasets\n- Polars has no index — use explicit columns instead\n- If the project already uses one, prefer consistency over switching\n\n### I\u002FO\n\nPrefer Parquet over CSV for performance and type preservation.\nSpecify `dtype` \u002F `schema` on read to enforce types upfront. For\nlarge CSVs, use chunked reads (pandas) or lazy scans (polars).\nAlways specify `parse_dates` or date columns explicitly —\nauto-detection is unreliable. Use `usecols` \u002F `columns` to read only\nthe columns you need.\n\n### Selection and filtering\n\nSelect columns by name, not position. In polars, use expressions:\n`df.select(pl.col(\"name\"), pl.col(\"age\"))`. In pandas, use `.loc[]`\nfor label-based and `.iloc[]` for position-based. Chain filters\nwith `&` and `|`, wrapping conditions in parentheses. Polars\n`filter()` is preferred over boolean indexing for clarity.\n\n### GroupBy and aggregation\n\nAlways name aggregated columns explicitly:\n\n```python\n# Polars\ndf.group_by(\"category\").agg(pl.col(\"price\").mean().alias(\"avg_price\"))\n\n# Pandas\ndf.groupby(\"category\")[\"price\"].mean().reset_index(name=\"avg_price\")\n```\n\nUse multiple aggregations in one pass for efficiency. Polars\nsupports `over()` for window functions without groupby. Avoid\niterating over groups — use vectorized aggregations.\n\n### Joins and merges\n\nSpecify join type explicitly: `inner`, `left`, `outer`, `cross`,\n`anti`, `semi`. Use `on=` for same-named keys, `left_on=`\u002F\n`right_on=` otherwise. Check for duplicates before joining to avoid\nrow explosion. Polars anti-join is efficient for \"rows not in\nother table.\" Use `validate=\"m:1\"` in pandas to catch unexpected\nduplicates.\n\n### Reshaping\n\nPivot to wide format for summary tables; melt to long for analysis\nand plotting:\n\n```python\n# Polars\ndf.pivot(on=\"category\", index=\"date\", values=\"sales\")\n\n# Pandas\ndf.pivot_table(index=\"date\", columns=\"category\", values=\"sales\", aggfunc=\"sum\")\n```\n\nUse `stack()`\u002F`unstack()` in pandas for multi-level index reshaping.\n\n### Missing data, strings, datetimes\n\nCheck missing with `df.null_count()` (polars) or `df.isna().sum()`\n(pandas). Fill with `fill_null` \u002F `fillna`, or forward\u002Fbackward\nfill. Drop only when appropriate. Strip whitespace and lowercase\nstrings early. Parse dates on read with explicit format. Always be\nexplicit about timezones — naive datetimes cause subtle bugs.\n\n### Performance\n\nUse lazy evaluation in polars: `df.lazy().filter(...).select(...).\ncollect()`. Polars optimizes the query plan, pushing filters and\nprojections down. In pandas, avoid `apply()` with Python functions —\nuse vectorized operations. Use categorical dtype for low-cardinality\nstrings. Profile with `%timeit` and check memory with\n`df.estimated_size()` (polars).\n\n## Gotchas\n\nAgent-specific failure modes — provider-neutral pause-and-self-check items:\n\n- **Using `apply()` with a Python function in pandas for performance-sensitive code.** `apply()` drops to a Python loop, which is 10–100x slower than vectorized operations. Use built-in string methods, `map()`, `np.where()`, or Polars expressions instead.\n- **Joining without checking for duplicates first.** A join on a key that has duplicates in either table causes a row explosion (M × N rows for M and N matching rows). Check `df[\"key\"].is_unique` before joining; validate with `validate=\"m:1\"` in pandas.\n- **Auto-detecting date formats or dtypes on read.** Pandas' auto-detection of dates and dtypes is unreliable across file versions and locales. Always specify `dtype=` and `parse_dates=` (pandas) or `schema=` (polars) explicitly on read. A column that auto-parsed as `datetime64` in development may parse as `object` on a slightly different CSV in production.\n- **Ignoring naive datetimes and timezone bugs.** A naive datetime (no timezone) compared to a tz-aware datetime raises an error in polars and silently produces wrong results in some pandas operations. Always be explicit about timezones — parse with an explicit `time_zone=` and convert to UTC at the boundary.\n- **Using polars with pandas `apply()` mental model.** Polars expressions are lazy and operate on the whole column — there is no row-by-row loop. Translating pandas `apply()` to polars `map_elements()` (element-wise Python) negates polars' performance advantage. Rewrite in terms of polars expression API.\n- **Not using lazy evaluation in polars for large datasets.** Calling `pl.read_csv(...)` eagerly loads the entire file into memory. Use `pl.scan_csv(...).lazy()` and chain `.filter()`, `.select()`, `.collect()` so polars can push projections and filters down and avoid materializing intermediate results.\n- **Storing multiple logical tables in one DataFrame.** Mixing two different entities (orders and line items) in one DataFrame with repeated parent-level columns produces confusing GroupBy results and joins. Separate into clean normalized DataFrames and join when needed.\n\n## Full reference\n\n### API comparison — reading data\n\n```python\n# Pandas\ndf = pd.read_csv(\"data.csv\", usecols=[\"a\", \"b\"], dtype={\"a\": \"int64\"})\ndf = pd.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars\ndf = pl.read_csv(\"data.csv\", columns=[\"a\", \"b\"], schema={\"a\": pl.Int64})\ndf = pl.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars lazy (recommended for large files)\nlf = pl.scan_csv(\"data.csv\")\ndf = lf.filter(...).select(...).collect()\n```\n\n### Selecting columns\n\n```python\n# Pandas\ndf[[\"name\", \"age\"]]\ndf.loc[:, \"name\":\"age\"]\n\n# Polars\ndf.select(\"name\", \"age\")\ndf.select(pl.col(\"name\"), pl.col(\"age\"))\ndf.select(pl.col(\"^sales_.*$\"))  # regex column selection\n```\n\n### Filtering rows\n\n```python\n# Pandas\ndf[df[\"age\"] > 30]\ndf[(df[\"age\"] > 30) & (df[\"city\"] == \"Berlin\")]\ndf.query(\"age > 30 and city == 'Berlin'\")\n\n# Polars\ndf.filter(pl.col(\"age\") > 30)\ndf.filter((pl.col(\"age\") > 30) & (pl.col(\"city\") == \"Berlin\"))\n```\n\n### Adding \u002F transforming columns\n\n```python\n# Pandas\ndf[\"total\"] = df[\"price\"] * df[\"qty\"]\ndf = df.assign(total=lambda d: d[\"price\"] * d[\"qty\"])\n\n# Polars\ndf = df.with_columns((pl.col(\"price\") * pl.col(\"qty\")).alias(\"total\"))\ndf = df.with_columns(\n    pl.col(\"name\").str.to_uppercase().alias(\"name_upper\"),\n    (pl.col(\"price\") * 1.1).alias(\"price_with_tax\"),\n)\n```\n\n### GroupBy and aggregation\n\n```python\n# Pandas\ndf.groupby(\"category\").agg(\n    avg_price=(\"price\", \"mean\"),\n    total_qty=(\"qty\", \"sum\"),\n    count=(\"id\", \"count\"),\n).reset_index()\n\n# Polars\ndf.group_by(\"category\").agg(\n    pl.col(\"price\").mean().alias(\"avg_price\"),\n    pl.col(\"qty\").sum().alias(\"total_qty\"),\n    pl.col(\"id\").count().alias(\"count\"),\n)\n```\n\n### Window functions\n\n```python\n# Pandas\ndf[\"rank\"] = df.groupby(\"category\")[\"price\"].rank(ascending=False)\ndf[\"avg_in_group\"] = df.groupby(\"category\")[\"price\"].transform(\"mean\")\n\n# Polars\ndf = df.with_columns(\n    pl.col(\"price\").rank(descending=True).over(\"category\").alias(\"rank\"),\n    pl.col(\"price\").mean().over(\"category\").alias(\"avg_in_group\"),\n)\n```\n\n### Joins\n\n```python\n# Pandas\npd.merge(left, right, on=\"id\", how=\"left\")\npd.merge(left, right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Polars\nleft.join(right, on=\"id\", how=\"left\")\nleft.join(right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Anti-join (rows in left NOT in right)\nleft[~left[\"id\"].isin(right[\"id\"])]            # pandas\nleft.join(right, on=\"id\", how=\"anti\")          # polars\n```\n\n### Pivot and melt\n\n```python\n# Pivot — Pandas\ndf.pivot_table(index=\"date\", columns=\"product\", values=\"sales\", aggfunc=\"sum\")\n# Pivot — Polars\ndf.pivot(on=\"product\", index=\"date\", values=\"sales\", aggregate_function=\"sum\")\n\n# Melt — Pandas\ndf.melt(id_vars=[\"date\"], value_vars=[\"product_a\", \"product_b\"],\n        var_name=\"product\", value_name=\"sales\")\n# Melt — Polars\ndf.unpivot(index=\"date\", on=[\"product_a\", \"product_b\"],\n           variable_name=\"product\", value_name=\"sales\")\n```\n\n### Missing data\n\n```python\n# Pandas\ndf.isna().sum()\ndf.fillna({\"price\": 0, \"name\": \"Unknown\"})\ndf.dropna(subset=[\"critical_col\"])\ndf[\"value\"].ffill()\n\n# Polars\ndf.null_count()\ndf.with_columns(\n    pl.col(\"price\").fill_null(0),\n    pl.col(\"name\").fill_null(\"Unknown\"),\n)\ndf.drop_nulls(subset=[\"critical_col\"])\ndf.with_columns(pl.col(\"value\").forward_fill())\n```\n\n### When to use which\n\n| Scenario | Recommendation |\n|---|---|\n| Exploratory data analysis | Pandas (Jupyter, wider ecosystem) |\n| Dataset > 1 GB | Polars (lower memory, parallel) |\n| Production data pipeline | Polars (lazy eval, deterministic) |\n| sklearn \u002F statsmodels integration | Pandas (native support) |\n| Streaming \u002F chunked processing | Polars lazy or pandas chunks |\n| Simple one-off scripts | Either — use what the team knows |\n| Multi-threaded workloads | Polars (releases GIL, built-in parallelism) |\n\n### Worked examples\n\n- **Large CSV → filter → group stats:** Polars lazy mode, scan the\n  CSV, filter before collect, group with multiple aggregations in\n  one `.agg()`, sort, write Parquet.\n- **Pandas → polars conversion:** Replace `.loc[]` with\n  `filter()`\u002F`select()`, convert `groupby().agg()` to polars\n  `group_by().agg()` with `pl.col()` expressions, drop index-based\n  ops, add `.lazy()`\u002F`.collect()` for optimization.\n- **Messy dataset cleanup:** Drop exact duplicates, identify\n  high-null columns, fill or drop based on context, lowercase and\n  strip strings, parse dates with explicit format, validate value\n  ranges via filter expressions.\n\n### References\n\n- [Pandas Documentation](https:\u002F\u002Fpandas.pydata.org\u002Fdocs\u002F)\n- [Polars User Guide](https:\u002F\u002Fdocs.pola.rs\u002F)\n","# Pandas and Polars\n\n## Intro\n\nPandas is the mature, broad-ecosystem default for exploratory work\nunder ~1 GB; polars is faster, leaner, and lazy-evaluated for\nproduction pipelines and larger data. Both read the same formats —\nwhen in doubt, match what the project already uses.\n\n## Overview\n\n### Choosing between them\n\n- **Pandas** — mature ecosystem, broader library compatibility\n  (sklearn, plotly), familiar API, best for EDA and \u003C 1 GB datasets\n- **Polars** — faster execution, lower memory, lazy evaluation,\n  better for production pipelines and 1 GB+ datasets\n- Polars has no index — use explicit columns instead\n- If the project already uses one, prefer consistency over switching\n\n### I\u002FO\n\nPrefer Parquet over CSV for performance and type preservation.\nSpecify `dtype` \u002F `schema` on read to enforce types upfront. For\nlarge CSVs, use chunked reads (pandas) or lazy scans (polars).\nAlways specify `parse_dates` or date columns explicitly —\nauto-detection is unreliable. Use `usecols` \u002F `columns` to read only\nthe columns you need.\n\n### Selection and filtering\n\nSelect columns by name, not position. In polars, use expressions:\n`df.select(pl.col(\"name\"), pl.col(\"age\"))`. In pandas, use `.loc[]`\nfor label-based and `.iloc[]` for position-based. Chain filters\nwith `&` and `|`, wrapping conditions in parentheses. Polars\n`filter()` is preferred over boolean indexing for clarity.\n\n### GroupBy and aggregation\n\nAlways name aggregated columns explicitly:\n\n```python\n# Polars\ndf.group_by(\"category\").agg(pl.col(\"price\").mean().alias(\"avg_price\"))\n\n# Pandas\ndf.groupby(\"category\")[\"price\"].mean().reset_index(name=\"avg_price\")\n```\n\nUse multiple aggregations in one pass for efficiency. Polars\nsupports `over()` for window functions without groupby. Avoid\niterating over groups — use vectorized aggregations.\n\n### Joins and merges\n\nSpecify join type explicitly: `inner`, `left`, `outer`, `cross`,\n`anti`, `semi`. Use `on=` for same-named keys, `left_on=`\u002F\n`right_on=` otherwise. Check for duplicates before joining to avoid\nrow explosion. Polars anti-join is efficient for \"rows not in\nother table.\" Use `validate=\"m:1\"` in pandas to catch unexpected\nduplicates.\n\n### Reshaping\n\nPivot to wide format for summary tables; melt to long for analysis\nand plotting:\n\n```python\n# Polars\ndf.pivot(on=\"category\", index=\"date\", values=\"sales\")\n\n# Pandas\ndf.pivot_table(index=\"date\", columns=\"category\", values=\"sales\", aggfunc=\"sum\")\n```\n\nUse `stack()`\u002F`unstack()` in pandas for multi-level index reshaping.\n\n### Missing data, strings, datetimes\n\nCheck missing with `df.null_count()` (polars) or `df.isna().sum()`\n(pandas). Fill with `fill_null` \u002F `fillna`, or forward\u002Fbackward\nfill. Drop only when appropriate. Strip whitespace and lowercase\nstrings early. Parse dates on read with explicit format. Always be\nexplicit about timezones — naive datetimes cause subtle bugs.\n\n### Performance\n\nUse lazy evaluation in polars: `df.lazy().filter(...).select(...).\ncollect()`. Polars optimizes the query plan, pushing filters and\nprojections down. In pandas, avoid `apply()` with Python functions —\nuse vectorized operations. Use categorical dtype for low-cardinality\nstrings. Profile with `%timeit` and check memory with\n`df.estimated_size()` (polars).\n\n## Gotchas\n\nAgent-specific failure modes — provider-neutral pause-and-self-check items:\n\n- **Using `apply()` with a Python function in pandas for performance-sensitive code.** `apply()` drops to a Python loop, which is 10–100x slower than vectorized operations. Use built-in string methods, `map()`, `np.where()`, or Polars expressions instead.\n- **Joining without checking for duplicates first.** A join on a key that has duplicates in either table causes a row explosion (M × N rows for M and N matching rows). Check `df[\"key\"].is_unique` before joining; validate with `validate=\"m:1\"` in pandas.\n- **Auto-detecting date formats or dtypes on read.** Pandas' auto-detection of dates and dtypes is unreliable across file versions and locales. Always specify `dtype=` and `parse_dates=` (pandas) or `schema=` (polars) explicitly on read. A column that auto-parsed as `datetime64` in development may parse as `object` on a slightly different CSV in production.\n- **Ignoring naive datetimes and timezone bugs.** A naive datetime (no timezone) compared to a tz-aware datetime raises an error in polars and silently produces wrong results in some pandas operations. Always be explicit about timezones — parse with an explicit `time_zone=` and convert to UTC at the boundary.\n- **Using polars with pandas `apply()` mental model.** Polars expressions are lazy and operate on the whole column — there is no row-by-row loop. Translating pandas `apply()` to polars `map_elements()` (element-wise Python) negates polars' performance advantage. Rewrite in terms of polars expression API.\n- **Not using lazy evaluation in polars for large datasets.** Calling `pl.read_csv(...)` eagerly loads the entire file into memory. Use `pl.scan_csv(...).lazy()` and chain `.filter()`, `.select()`, `.collect()` so polars can push projections and filters down and avoid materializing intermediate results.\n- **Storing multiple logical tables in one DataFrame.** Mixing two different entities (orders and line items) in one DataFrame with repeated parent-level columns produces confusing GroupBy results and joins. Separate into clean normalized DataFrames and join when needed.\n\n## Full reference\n\n### API comparison — reading data\n\n```python\n# Pandas\ndf = pd.read_csv(\"data.csv\", usecols=[\"a\", \"b\"], dtype={\"a\": \"int64\"})\ndf = pd.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars\ndf = pl.read_csv(\"data.csv\", columns=[\"a\", \"b\"], schema={\"a\": pl.Int64})\ndf = pl.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars lazy (recommended for large files)\nlf = pl.scan_csv(\"data.csv\")\ndf = lf.filter(...).select(...).collect()\n```\n\n### Selecting columns\n\n```python\n# Pandas\ndf[[\"name\", \"age\"]]\ndf.loc[:, \"name\":\"age\"]\n\n# Polars\ndf.select(\"name\", \"age\")\ndf.select(pl.col(\"name\"), pl.col(\"age\"))\ndf.select(pl.col(\"^sales_.*$\"))  # regex column selection\n```\n\n### Filtering rows\n\n```python\n# Pandas\ndf[df[\"age\"] > 30]\ndf[(df[\"age\"] > 30) & (df[\"city\"] == \"Berlin\")]\ndf.query(\"age > 30 and city == 'Berlin'\")\n\n# Polars\ndf.filter(pl.col(\"age\") > 30)\ndf.filter((pl.col(\"age\") > 30) & (pl.col(\"city\") == \"Berlin\"))\n```\n\n### Adding \u002F transforming columns\n\n```python\n# Pandas\ndf[\"total\"] = df[\"price\"] * df[\"qty\"]\ndf = df.assign(total=lambda d: d[\"price\"] * d[\"qty\"])\n\n# Polars\ndf = df.with_columns((pl.col(\"price\") * pl.col(\"qty\")).alias(\"total\"))\ndf = df.with_columns(\n    pl.col(\"name\").str.to_uppercase().alias(\"name_upper\"),\n    (pl.col(\"price\") * 1.1).alias(\"price_with_tax\"),\n)\n```\n\n### GroupBy and aggregation\n\n```python\n# Pandas\ndf.groupby(\"category\").agg(\n    avg_price=(\"price\", \"mean\"),\n    total_qty=(\"qty\", \"sum\"),\n    count=(\"id\", \"count\"),\n).reset_index()\n\n# Polars\ndf.group_by(\"category\").agg(\n    pl.col(\"price\").mean().alias(\"avg_price\"),\n    pl.col(\"qty\").sum().alias(\"total_qty\"),\n    pl.col(\"id\").count().alias(\"count\"),\n)\n```\n\n### Window functions\n\n```python\n# Pandas\ndf[\"rank\"] = df.groupby(\"category\")[\"price\"].rank(ascending=False)\ndf[\"avg_in_group\"] = df.groupby(\"category\")[\"price\"].transform(\"mean\")\n\n# Polars\ndf = df.with_columns(\n    pl.col(\"price\").rank(descending=True).over(\"category\").alias(\"rank\"),\n    pl.col(\"price\").mean().over(\"category\").alias(\"avg_in_group\"),\n)\n```\n\n### Joins\n\n```python\n# Pandas\npd.merge(left, right, on=\"id\", how=\"left\")\npd.merge(left, right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Polars\nleft.join(right, on=\"id\", how=\"left\")\nleft.join(right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Anti-join (rows in left NOT in right)\nleft[~left[\"id\"].isin(right[\"id\"])]            # pandas\nleft.join(right, on=\"id\", how=\"anti\")          # polars\n```\n\n### Pivot and melt\n\n```python\n# Pivot — Pandas\ndf.pivot_table(index=\"date\", columns=\"product\", values=\"sales\", aggfunc=\"sum\")\n# Pivot — Polars\ndf.pivot(on=\"product\", index=\"date\", values=\"sales\", aggregate_function=\"sum\")\n\n# Melt — Pandas\ndf.melt(id_vars=[\"date\"], value_vars=[\"product_a\", \"product_b\"],\n        var_name=\"product\", value_name=\"sales\")\n# Melt — Polars\ndf.unpivot(index=\"date\", on=[\"product_a\", \"product_b\"],\n           variable_name=\"product\", value_name=\"sales\")\n```\n\n### Missing data\n\n```python\n# Pandas\ndf.isna().sum()\ndf.fillna({\"price\": 0, \"name\": \"Unknown\"})\ndf.dropna(subset=[\"critical_col\"])\ndf[\"value\"].ffill()\n\n# Polars\ndf.null_count()\ndf.with_columns(\n    pl.col(\"price\").fill_null(0),\n    pl.col(\"name\").fill_null(\"Unknown\"),\n)\ndf.drop_nulls(subset=[\"critical_col\"])\ndf.with_columns(pl.col(\"value\").forward_fill())\n```\n\n### When to use which\n\n| Scenario | Recommendation |\n|---|---|\n| Exploratory data analysis | Pandas (Jupyter, wider ecosystem) |\n| Dataset > 1 GB | Polars (lower memory, parallel) |\n| Production data pipeline | Polars (lazy eval, deterministic) |\n| sklearn \u002F statsmodels integration | Pandas (native support) |\n| Streaming \u002F chunked processing | Polars lazy or pandas chunks |\n| Simple one-off scripts | Either — use what the team knows |\n| Multi-threaded workloads | Polars (releases GIL, built-in parallelism) |\n\n### Worked examples\n\n- **Large CSV → filter → group stats:** Polars lazy mode, scan the\n  CSV, filter before collect, group with multiple aggregations in\n  one `.agg()`, sort, write Parquet.\n- **Pandas → polars conversion:** Replace `.loc[]` with\n  `filter()`\u002F`select()`, convert `groupby().agg()` to polars\n  `group_by().agg()` with `pl.col()` expressions, drop index-based\n  ops, add `.lazy()`\u002F`.collect()` for optimization.\n- **Messy dataset cleanup:** Drop exact duplicates, identify\n  high-null columns, fill or drop based on context, lowercase and\n  strip strings, parse dates with explicit format, validate value\n  ranges via filter expressions.\n\n### References\n\n- [Pandas Documentation](https:\u002F\u002Fpandas.pydata.org\u002Fdocs\u002F)\n- [Polars User Guide](https:\u002F\u002Fdocs.pola.rs\u002F)\n",{"name":5,"metadata":49,"description":56},{"processkit":50},{"id":51,"created":52,"version":53,"category":54,"apiVersion":55},"SKILL-pandas-polars",1775433600000,"1.1.0","data-ai","processkit.projectious.work\u002Fv1","DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.\n","main","4876a92620ee792aac2ac9a1a806322d8a572367","archive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.10.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit","https:\u002F\u002Fprojectious-work.github.io\u002Fprocesskit\u002F","https:\u002F\u002Fgithub.com\u002Fprojectious-work","2026-11-11 23:24:18",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add projectious-work\u002Fprocesskit --skill pandas-polars",{"tool":70,"command":71},"bunx","bunx skills add projectious-work\u002Fprocesskit --skill pandas-polars",{"tool":73,"command":74},"pnpm","pnpm dlx skills add projectious-work\u002Fprocesskit --skill pandas-polars","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.10.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002FSKILL.md",true,[78],{"path":79,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fapi-comparison.md","REFERENCE","参考文档",4106,"https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.10.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002Freferences\u002Fapi-comparison.md",[85,92,99,105,129,149],{"skillNo":86,"name":87,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":91},"01M7665G2ED3624R3X93304265","standard-ui-design","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fstandard-ui-design\u002FSKILL.md",[],[],"2026-11-12 13:55:43",{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":91},"01M76651QTC6W0HJ4AFW2MMR3V","standard-agent-use","Company standard for working with AI agents. Use when deciding what to delegate to agents, capturing open work, keeping agent memory or knowledge notes, improving an agent workflow, or introducing agent work in a team or for a client.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fstandard-agent-use\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":102,"categories":103,"keywords":104,"publishedAt":91},"01M76650NN4T8P241021NBR64S","standard-agent-building","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fstandard-agent-building\u002FSKILL.md",[],[],{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":114,"publishedAt":44},"01M6X5MTC89B4ZGXW3JVSQCPF8","skill-management","Manage Skill package manifests as v1 ontology entities.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fskill-management\u002FSKILL.md",[111],{"categoryNo":112,"categoryName":113,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[115,116,117,41,118,119,37,120,121,122,123,124,107,125,126,127,128],"management","skill management","ontology","create","as","transition","get","manage","update","alpha","list","package","skill package","manifests",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":136,"publishedAt":44},"01M6X5MCV74NHW5MEDNHZRF2C8","schema-management","Manage generated processkit schema contracts through deterministic MCP rebuild and inspection workflows.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fschema-management\u002FSKILL.md",[135],{"categoryNo":112,"categoryName":113,"taxonomyType":21,"taxonomyTypeDesc":22},[137,138,115,139,140,141,142,143,144,145,146,147,37,121,122,124,148],"schema","schema management","contract","rebuild","contracts","workflows","wed","generated","MCP","validation","deterministic","mode",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":44},"01M6X5M5G6YJ96AMZGSCZJ1D13","proposition-management","Record claims, risks, beliefs, world facts, WSJF estimates, and assumptions through the Proposition interface.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fproposition-management\u002FSKILL.md",[],[156,115,157,158,118,159,160,161,37,121,162,163,123,164,124,165,166,167,168,169],"proposition management","proposition","interface","claims","record","world","creation","event","WSJF","index","updates","query","facts","log"]