[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7166VXJWQSSC3RQZP60WBGR":3,"skill-same:01M7166VXJWQSSC3RQZP60WBGR":80},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":49,"linkOnly":50,"markdownContent":51,"markdownBody":52,"frontMatter":53,"refName":60,"commitSha":61,"skillDirectory":62,"forksCount":14,"repositoryHtmlUrl":63,"repositoryHomepageUrl":6,"ownerHtmlUrl":64,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":65,"install":66,"resources":79},"01M7166VXJWQSSC3RQZP60WBGR","research-code",null,"Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000021","资料检索与研究","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48],"research","code","research code","read","when","which","errors","scripts","line","data-analysis","uv","write","review","common","Bash","analysis","data pipelines","run","debugging","memory","2026-11-10 10:12:30",false,"---\nname: research-code\ndescription: \"Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.\"\nallowed-tools: Read Write Edit Bash\ncompatibility: \"Python >=3.10. Profiling: cProfile (stdlib), line_profiler, memory_profiler. Install extras with uv.\"\nlicense: MIT license\nmetadata: {\"version\": \"1.0\", \"skill-author\": \"K-Dense Inc.\"}\n---\n\n# Research Code\n\n## Overview\n\nScientific code fails differently from production software: bugs are often silent (wrong numbers instead of crashes), reproducibility is an afterthought, and the codebase is shaped by one person writing fast rather than a team writing carefully. This skill addresses the code layer of research — not the science, not the data, but the scripts that connect them.\n\n**Scope:**\n- **Debugging** — crashes, wrong results, unexpected behavior\n- **Code review for correctness** — common numerical and statistical coding errors\n- **Profiling** — slow pipelines, memory blowups\n- **Reproducibility** — making runs deterministic and shareable\n- **Structure** — organizing scripts for clarity and reuse\n\n---\n\n## Debugging\n\n### Step 1: Read the full traceback\n\nPython tracebacks are read bottom-up: the bottom line is the error; the lines above are the call stack. Look at:\n- The **error type** (`TypeError`, `ValueError`, `KeyError`, `IndexError`, `AttributeError`)\n- The **error message** — often tells you exactly what went wrong\n- The **file and line number** where the error occurred\n- The **local variables** at that line (add a `print()` or use `pdb`)\n\n### Step 2: Reproduce minimally\n\nStrip away everything not needed to reproduce the bug:\n\n```python\n# Before — large real dataset, complex pipeline\n# After — minimal reproducible example (MRE)\nimport numpy as np\n\nX = np.array([[1, 2], [3, 4]])\n# Does the bug happen here?\nresult = my_function(X)\n```\n\n**Rules for a good MRE:**\n- Smallest dataset that triggers the bug (synthetic is fine)\n- No external files or database connections\n- No irrelevant imports\n- Self-contained in one file\n\n### Step 3: Form and test hypotheses\n\nDon't randomly change code. Form a specific hypothesis about the cause, then test it:\n\n```python\n# Hypothesis: the bug is in the preprocessing, not the model\n# Test: bypass preprocessing, feed raw data directly\nresult = model.predict(X_raw)   # does it still fail?\n```\n\n### Common debugging patterns\n\n**Shape mismatches (numpy\u002Ftorch)**\n```python\n# Always print shapes at intermediate steps\nprint(f\"X: {X.shape}, y: {y.shape}, weights: {weights.shape}\")\n# Expected: (100, 10), (100,), (10,)\n```\n\n**Off-by-one in slicing \u002F indexing**\n```python\n# Python slices exclude the end: a[0:5] gives indices 0,1,2,3,4\n# pandas .iloc vs .loc — iloc is position-based, loc is label-based\ndf.iloc[0:5]   # rows 0–4\ndf.loc[0:5]    # rows where index label is 0 through 5 (inclusive!)\n```\n\n**Silent NaN propagation**\n```python\nimport numpy as np\nimport pandas as pd\n\n# Check for NaNs after every transformation\nassert not np.isnan(X).any(), f\"NaN in X after preprocessing\"\nassert not df.isnull().values.any(), f\"NaN in DataFrame: {df.isnull().sum()}\"\n\n# Find where NaNs came from\ndf.isnull().sum()           # count per column\ndf[df['col'].isnull()]      # rows with NaN in a column\n```\n\n**Wrong axis in numpy operations**\n```python\n# axis=0 operates along rows (across rows, per column)\n# axis=1 operates along columns (across columns, per row)\nX = np.array([[1, 2, 3], [4, 5, 6]])\n\nX.mean(axis=0)  # [2.5, 3.5, 4.5] — mean of each column\nX.mean(axis=1)  # [2.0, 5.0]       — mean of each row\n\n# Normalize each sample (row) — common mistake: wrong axis\nX_norm = (X - X.mean(axis=1, keepdims=True)) \u002F X.std(axis=1, keepdims=True)\n# Without keepdims=True, broadcasting fails\n```\n\n**Random state inconsistency**\n```python\nimport numpy as np\nimport random\n\n# Set all seeds at the top of the script\nSEED = 42\nnp.random.seed(SEED)\nrandom.seed(SEED)\n\n# PyTorch\nimport torch\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n\n# sklearn\nfrom sklearn.model_selection import train_test_split\nX_train, X_test = train_test_split(X, random_state=SEED)\n```\n\n**Using `pdb` for interactive debugging**\n```python\nimport pdb; pdb.set_trace()   # breakpoint (Python 3.7+: just `breakpoint()`)\n# In pdb: n=next, s=step into, c=continue, p \u003Cvar>=print, l=list code, q=quit\n```\n\n---\n\n## Code Review for Correctness\n\n### Statistical coding errors\n\n**Data leakage — fitting on test data**\n```python\n# WRONG: scaler sees test data during fit\nscaler = StandardScaler()\nX_scaled = scaler.fit_transform(X)            # fits on ALL data\nX_train, X_test = train_test_split(X_scaled)\n\n# CORRECT: fit only on train, transform both\nX_train, X_test = train_test_split(X)\nscaler = StandardScaler()\nX_train = scaler.fit_transform(X_train)        # fit on train only\nX_test = scaler.transform(X_test)              # apply same transform\n```\n\n**Target leakage — features that encode the outcome**\n```python\n# Example: 'diagnosis_date' is derived from the outcome — leakage\n# Rule: no feature column should be causally downstream of the target\n```\n\n**Incorrect cross-validation with grouped data**\n```python\n# WRONG: rows from the same patient can appear in both train and test\ncv = KFold(n_splits=5)\n\n# CORRECT for grouped data (patients, subjects, locations):\nfrom sklearn.model_selection import GroupKFold\ncv = GroupKFold(n_splits=5)\nfor train_idx, test_idx in cv.split(X, y, groups=patient_ids):\n    ...\n```\n\n**Wrong metric for imbalanced classes**\n```python\n# Accuracy is misleading when classes are imbalanced (95% negative → 95% by predicting all negative)\n# Use instead:\nfrom sklearn.metrics import roc_auc_score, average_precision_score, f1_score\nauc = roc_auc_score(y_test, proba[:, 1])\nap = average_precision_score(y_test, proba[:, 1])   # area under PR curve\nf1 = f1_score(y_test, preds, average='macro')\n```\n\n**Multiple comparisons without correction**\n```python\nfrom statsmodels.stats.multitest import multipletests\n\np_values = [0.03, 0.04, 0.001, 0.07, 0.02]\nreject, p_corrected, _, _ = multipletests(p_values, alpha=0.05, method='fdr_bh')\n# Benjamini-Hochberg FDR correction\n```\n\n**Numerical instability — log of zero or division by zero**\n```python\n# WRONG: log(0) = -inf\nlog_proba = np.log(proba)\n\n# CORRECT: use log1p for values near 0, or clip\nimport numpy as np\nlog_proba = np.log(np.clip(proba, 1e-10, 1.0))\n\n# Or use scipy which handles edge cases:\nfrom scipy.special import xlogy\n# xlogy(x, y) = x * log(y), returns 0 when x=0 (avoiding 0 * -inf)\n```\n\n**Integer overflow in numpy**\n```python\n# Default numpy int is int64; but some operations can overflow\ncounts = np.array([1000000], dtype=np.int32)\nresult = counts * counts    # overflows int32!\n\n# Fix: use int64 or float\ncounts = counts.astype(np.int64)\n```\n\n---\n\n## Profiling\n\n### Find the bottleneck first\n\n```bash\n# cProfile — built-in, profile the whole script\npython -m cProfile -s cumulative my_script.py 2>&1 | head -30\n\n# Or profile a specific function:\nimport cProfile\ncProfile.run('my_slow_function(data)', sort='cumulative')\n```\n\n### Line-level profiling\n\n```bash\nuv pip install line_profiler\n\n# Decorate the function you suspect:\n@profile\ndef my_slow_function(data):\n    ...\n\n# Run:\nkernprof -l -v my_script.py\n```\n\n### Memory profiling\n\n```bash\nuv pip install memory_profiler\n\n@profile\ndef memory_hungry_function(data):\n    ...\n\npython -m memory_profiler my_script.py\n```\n\n### Common bottlenecks and fixes\n\n**Python loop over pandas rows**\n```python\n# SLOW: iterrows is ~1000x slower than vectorized\nfor idx, row in df.iterrows():\n    result.append(row['a'] * row['b'])\n\n# FAST: vectorized operation\nresult = df['a'] * df['b']\n\n# FAST: apply (slower than vectorized but faster than iterrows)\nresult = df.apply(lambda row: row['a'] * row['b'], axis=1)\n```\n\n**Loading large CSV repeatedly**\n```python\n# Cache as parquet — 10-100x faster reads, much smaller files\ndf = pd.read_csv('large_data.csv')\ndf.to_parquet('large_data.parquet')\n\n# Next time:\ndf = pd.read_parquet('large_data.parquet')\n```\n\n**Recomputing expensive results**\n```python\nimport joblib\n\ndef expensive_computation(data):\n    ...\n\n# Cache with joblib.Memory\nmemory = joblib.Memory('.\u002Fcache', verbose=0)\ncached_fn = memory.cache(expensive_computation)\nresult = cached_fn(data)   # recomputes only when data changes\n```\n\n**Unnecessary copies of large arrays**\n```python\n# Check if an array is a view or a copy\nx = arr[:100]\nx.base is arr    # True → view (no copy); False → copy\n\n# Prefer in-place operations when possible\narr += 1         # in-place, no copy\narr = arr + 1    # creates a new array\n```\n\n---\n\n## Reproducibility\n\n### Minimal reproducibility checklist\n\n```python\n# 1. Pin all random seeds (see debugging section above)\n\n# 2. Log the environment\nimport subprocess, sys\nprint(sys.version)\nprint(subprocess.check_output([\"pip\", \"freeze\"]).decode())\n\n# 3. Log the git commit\nimport subprocess\ncommit = subprocess.check_output([\"git\", \"rev-parse\", \"HEAD\"]).decode().strip()\nprint(f\"Git commit: {commit}\")\n\n# 4. Log input data hashes\nimport hashlib, pandas as pd\ndef file_hash(path):\n    return hashlib.md5(open(path, 'rb').read()).hexdigest()\n\nprint(f\"Data hash: {file_hash('data\u002Ftrain.csv')}\")\n\n# 5. Save outputs with timestamps or version tags, not by overwriting\nimport datetime\nts = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\ndf.to_csv(f\"results_{ts}.csv\", index=False)\n```\n\n### Environment management\n\n```bash\n# Capture current environment\nuv pip freeze > requirements.txt\n\n# Recreate it\nuv pip install -r requirements.txt\n\n# For full isolation (recommended for sharing)\nuv venv .venv\nsource .venv\u002Fbin\u002Factivate\nuv pip install -r requirements.txt\n```\n\n---\n\n## Structuring Analysis Scripts\n\n### Recommended layout for a single analysis\n\n```\nproject\u002F\n├── data\u002F\n│   ├── raw\u002F          # original, never modified\n│   └── processed\u002F    # outputs of preprocessing\n├── notebooks\u002F        # exploratory, named with date: 20240601_eda.ipynb\n├── scripts\u002F\n│   ├── preprocess.py\n│   ├── train.py\n│   └── evaluate.py\n├── outputs\u002F\n│   ├── figures\u002F\n│   └── results\u002F\n├── requirements.txt\n└── README.md         # how to reproduce from scratch\n```\n\n### Make scripts runnable as CLI tools\n\n```python\nimport argparse\n\ndef parse_args():\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"--data\", required=True, help=\"Path to input CSV\")\n    parser.add_argument(\"--output\", default=\"results.csv\")\n    parser.add_argument(\"--seed\", type=int, default=42)\n    return parser.parse_args()\n\ndef main():\n    args = parse_args()\n    # ... use args.data, args.output, args.seed\n\nif __name__ == \"__main__\":\n    main()\n```\n\n---\n\n## Integration\n\n- **`\u002Fexploratory-data-analysis`** — understand what's in the data before debugging why analysis produces unexpected results\n- **`\u002Fstatistical-analysis`** — choose the right test; this skill fixes the code that runs it\n- **`\u002Fexperiment-tracking`** — once the code is correct, track runs with MLflow or W&B\n- **`\u002Fpolars`** — replace slow pandas pipelines with Polars for large datasets\n","# Research Code\n\n## Overview\n\nScientific code fails differently from production software: bugs are often silent (wrong numbers instead of crashes), reproducibility is an afterthought, and the codebase is shaped by one person writing fast rather than a team writing carefully. This skill addresses the code layer of research — not the science, not the data, but the scripts that connect them.\n\n**Scope:**\n- **Debugging** — crashes, wrong results, unexpected behavior\n- **Code review for correctness** — common numerical and statistical coding errors\n- **Profiling** — slow pipelines, memory blowups\n- **Reproducibility** — making runs deterministic and shareable\n- **Structure** — organizing scripts for clarity and reuse\n\n---\n\n## Debugging\n\n### Step 1: Read the full traceback\n\nPython tracebacks are read bottom-up: the bottom line is the error; the lines above are the call stack. Look at:\n- The **error type** (`TypeError`, `ValueError`, `KeyError`, `IndexError`, `AttributeError`)\n- The **error message** — often tells you exactly what went wrong\n- The **file and line number** where the error occurred\n- The **local variables** at that line (add a `print()` or use `pdb`)\n\n### Step 2: Reproduce minimally\n\nStrip away everything not needed to reproduce the bug:\n\n```python\n# Before — large real dataset, complex pipeline\n# After — minimal reproducible example (MRE)\nimport numpy as np\n\nX = np.array([[1, 2], [3, 4]])\n# Does the bug happen here?\nresult = my_function(X)\n```\n\n**Rules for a good MRE:**\n- Smallest dataset that triggers the bug (synthetic is fine)\n- No external files or database connections\n- No irrelevant imports\n- Self-contained in one file\n\n### Step 3: Form and test hypotheses\n\nDon't randomly change code. Form a specific hypothesis about the cause, then test it:\n\n```python\n# Hypothesis: the bug is in the preprocessing, not the model\n# Test: bypass preprocessing, feed raw data directly\nresult = model.predict(X_raw)   # does it still fail?\n```\n\n### Common debugging patterns\n\n**Shape mismatches (numpy\u002Ftorch)**\n```python\n# Always print shapes at intermediate steps\nprint(f\"X: {X.shape}, y: {y.shape}, weights: {weights.shape}\")\n# Expected: (100, 10), (100,), (10,)\n```\n\n**Off-by-one in slicing \u002F indexing**\n```python\n# Python slices exclude the end: a[0:5] gives indices 0,1,2,3,4\n# pandas .iloc vs .loc — iloc is position-based, loc is label-based\ndf.iloc[0:5]   # rows 0–4\ndf.loc[0:5]    # rows where index label is 0 through 5 (inclusive!)\n```\n\n**Silent NaN propagation**\n```python\nimport numpy as np\nimport pandas as pd\n\n# Check for NaNs after every transformation\nassert not np.isnan(X).any(), f\"NaN in X after preprocessing\"\nassert not df.isnull().values.any(), f\"NaN in DataFrame: {df.isnull().sum()}\"\n\n# Find where NaNs came from\ndf.isnull().sum()           # count per column\ndf[df['col'].isnull()]      # rows with NaN in a column\n```\n\n**Wrong axis in numpy operations**\n```python\n# axis=0 operates along rows (across rows, per column)\n# axis=1 operates along columns (across columns, per row)\nX = np.array([[1, 2, 3], [4, 5, 6]])\n\nX.mean(axis=0)  # [2.5, 3.5, 4.5] — mean of each column\nX.mean(axis=1)  # [2.0, 5.0]       — mean of each row\n\n# Normalize each sample (row) — common mistake: wrong axis\nX_norm = (X - X.mean(axis=1, keepdims=True)) \u002F X.std(axis=1, keepdims=True)\n# Without keepdims=True, broadcasting fails\n```\n\n**Random state inconsistency**\n```python\nimport numpy as np\nimport random\n\n# Set all seeds at the top of the script\nSEED = 42\nnp.random.seed(SEED)\nrandom.seed(SEED)\n\n# PyTorch\nimport torch\ntorch.manual_seed(SEED)\ntorch.cuda.manual_seed_all(SEED)\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n\n# sklearn\nfrom sklearn.model_selection import train_test_split\nX_train, X_test = train_test_split(X, random_state=SEED)\n```\n\n**Using `pdb` for interactive debugging**\n```python\nimport pdb; pdb.set_trace()   # breakpoint (Python 3.7+: just `breakpoint()`)\n# In pdb: n=next, s=step into, c=continue, p \u003Cvar>=print, l=list code, q=quit\n```\n\n---\n\n## Code Review for Correctness\n\n### Statistical coding errors\n\n**Data leakage — fitting on test data**\n```python\n# WRONG: scaler sees test data during fit\nscaler = StandardScaler()\nX_scaled = scaler.fit_transform(X)            # fits on ALL data\nX_train, X_test = train_test_split(X_scaled)\n\n# CORRECT: fit only on train, transform both\nX_train, X_test = train_test_split(X)\nscaler = StandardScaler()\nX_train = scaler.fit_transform(X_train)        # fit on train only\nX_test = scaler.transform(X_test)              # apply same transform\n```\n\n**Target leakage — features that encode the outcome**\n```python\n# Example: 'diagnosis_date' is derived from the outcome — leakage\n# Rule: no feature column should be causally downstream of the target\n```\n\n**Incorrect cross-validation with grouped data**\n```python\n# WRONG: rows from the same patient can appear in both train and test\ncv = KFold(n_splits=5)\n\n# CORRECT for grouped data (patients, subjects, locations):\nfrom sklearn.model_selection import GroupKFold\ncv = GroupKFold(n_splits=5)\nfor train_idx, test_idx in cv.split(X, y, groups=patient_ids):\n    ...\n```\n\n**Wrong metric for imbalanced classes**\n```python\n# Accuracy is misleading when classes are imbalanced (95% negative → 95% by predicting all negative)\n# Use instead:\nfrom sklearn.metrics import roc_auc_score, average_precision_score, f1_score\nauc = roc_auc_score(y_test, proba[:, 1])\nap = average_precision_score(y_test, proba[:, 1])   # area under PR curve\nf1 = f1_score(y_test, preds, average='macro')\n```\n\n**Multiple comparisons without correction**\n```python\nfrom statsmodels.stats.multitest import multipletests\n\np_values = [0.03, 0.04, 0.001, 0.07, 0.02]\nreject, p_corrected, _, _ = multipletests(p_values, alpha=0.05, method='fdr_bh')\n# Benjamini-Hochberg FDR correction\n```\n\n**Numerical instability — log of zero or division by zero**\n```python\n# WRONG: log(0) = -inf\nlog_proba = np.log(proba)\n\n# CORRECT: use log1p for values near 0, or clip\nimport numpy as np\nlog_proba = np.log(np.clip(proba, 1e-10, 1.0))\n\n# Or use scipy which handles edge cases:\nfrom scipy.special import xlogy\n# xlogy(x, y) = x * log(y), returns 0 when x=0 (avoiding 0 * -inf)\n```\n\n**Integer overflow in numpy**\n```python\n# Default numpy int is int64; but some operations can overflow\ncounts = np.array([1000000], dtype=np.int32)\nresult = counts * counts    # overflows int32!\n\n# Fix: use int64 or float\ncounts = counts.astype(np.int64)\n```\n\n---\n\n## Profiling\n\n### Find the bottleneck first\n\n```bash\n# cProfile — built-in, profile the whole script\npython -m cProfile -s cumulative my_script.py 2>&1 | head -30\n\n# Or profile a specific function:\nimport cProfile\ncProfile.run('my_slow_function(data)', sort='cumulative')\n```\n\n### Line-level profiling\n\n```bash\nuv pip install line_profiler\n\n# Decorate the function you suspect:\n@profile\ndef my_slow_function(data):\n    ...\n\n# Run:\nkernprof -l -v my_script.py\n```\n\n### Memory profiling\n\n```bash\nuv pip install memory_profiler\n\n@profile\ndef memory_hungry_function(data):\n    ...\n\npython -m memory_profiler my_script.py\n```\n\n### Common bottlenecks and fixes\n\n**Python loop over pandas rows**\n```python\n# SLOW: iterrows is ~1000x slower than vectorized\nfor idx, row in df.iterrows():\n    result.append(row['a'] * row['b'])\n\n# FAST: vectorized operation\nresult = df['a'] * df['b']\n\n# FAST: apply (slower than vectorized but faster than iterrows)\nresult = df.apply(lambda row: row['a'] * row['b'], axis=1)\n```\n\n**Loading large CSV repeatedly**\n```python\n# Cache as parquet — 10-100x faster reads, much smaller files\ndf = pd.read_csv('large_data.csv')\ndf.to_parquet('large_data.parquet')\n\n# Next time:\ndf = pd.read_parquet('large_data.parquet')\n```\n\n**Recomputing expensive results**\n```python\nimport joblib\n\ndef expensive_computation(data):\n    ...\n\n# Cache with joblib.Memory\nmemory = joblib.Memory('.\u002Fcache', verbose=0)\ncached_fn = memory.cache(expensive_computation)\nresult = cached_fn(data)   # recomputes only when data changes\n```\n\n**Unnecessary copies of large arrays**\n```python\n# Check if an array is a view or a copy\nx = arr[:100]\nx.base is arr    # True → view (no copy); False → copy\n\n# Prefer in-place operations when possible\narr += 1         # in-place, no copy\narr = arr + 1    # creates a new array\n```\n\n---\n\n## Reproducibility\n\n### Minimal reproducibility checklist\n\n```python\n# 1. Pin all random seeds (see debugging section above)\n\n# 2. Log the environment\nimport subprocess, sys\nprint(sys.version)\nprint(subprocess.check_output([\"pip\", \"freeze\"]).decode())\n\n# 3. Log the git commit\nimport subprocess\ncommit = subprocess.check_output([\"git\", \"rev-parse\", \"HEAD\"]).decode().strip()\nprint(f\"Git commit: {commit}\")\n\n# 4. Log input data hashes\nimport hashlib, pandas as pd\ndef file_hash(path):\n    return hashlib.md5(open(path, 'rb').read()).hexdigest()\n\nprint(f\"Data hash: {file_hash('data\u002Ftrain.csv')}\")\n\n# 5. Save outputs with timestamps or version tags, not by overwriting\nimport datetime\nts = datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\")\ndf.to_csv(f\"results_{ts}.csv\", index=False)\n```\n\n### Environment management\n\n```bash\n# Capture current environment\nuv pip freeze > requirements.txt\n\n# Recreate it\nuv pip install -r requirements.txt\n\n# For full isolation (recommended for sharing)\nuv venv .venv\nsource .venv\u002Fbin\u002Factivate\nuv pip install -r requirements.txt\n```\n\n---\n\n## Structuring Analysis Scripts\n\n### Recommended layout for a single analysis\n\n```\nproject\u002F\n├── data\u002F\n│   ├── raw\u002F          # original, never modified\n│   └── processed\u002F    # outputs of preprocessing\n├── notebooks\u002F        # exploratory, named with date: 20240601_eda.ipynb\n├── scripts\u002F\n│   ├── preprocess.py\n│   ├── train.py\n│   └── evaluate.py\n├── outputs\u002F\n│   ├── figures\u002F\n│   └── results\u002F\n├── requirements.txt\n└── README.md         # how to reproduce from scratch\n```\n\n### Make scripts runnable as CLI tools\n\n```python\nimport argparse\n\ndef parse_args():\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"--data\", required=True, help=\"Path to input CSV\")\n    parser.add_argument(\"--output\", default=\"results.csv\")\n    parser.add_argument(\"--seed\", type=int, default=42)\n    return parser.parse_args()\n\ndef main():\n    args = parse_args()\n    # ... use args.data, args.output, args.seed\n\nif __name__ == \"__main__\":\n    main()\n```\n\n---\n\n## Integration\n\n- **`\u002Fexploratory-data-analysis`** — understand what's in the data before debugging why analysis produces unexpected results\n- **`\u002Fstatistical-analysis`** — choose the right test; this skill fixes the code that runs it\n- **`\u002Fexperiment-tracking`** — once the code is correct, track runs with MLflow or W&B\n- **`\u002Fpolars`** — replace slow pandas pipelines with Polars for large datasets\n",{"name":5,"license":54,"metadata":55,"description":7,"allowed-tools":58,"compatibility":59},"MIT license",{"version":56,"skill-author":57},"1.0","K-Dense Inc.","Read Write Edit Bash","Python >=3.10. Profiling: cProfile (stdlib), line_profiler, memory_profiler. Install extras with uv.","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","scientific-agent-skills\u002Fskills\u002Fresearch-code","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":67,"prompt":77,"zipAvailable":78},[68,71,74],{"tool":69,"command":70},"npx","npx skills add fadrienne\u002FMyGitHub --skill research-code",{"tool":72,"command":73},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill research-code",{"tool":75,"command":76},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill research-code","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",true,[],[81,107,133,164,191,218],{"skillNo":82,"name":83,"nameZh":6,"description":84,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":90,"publishedAt":49},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[87],{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[91,92,93,94,95,96,97,39,33,43,98,44,99,100,101,102,103,104,105,106],"power","statistical power","effect","scipy","regression","choose","design","collection","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":111,"categories":112,"keywords":116,"publishedAt":49},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[113],{"categoryNo":114,"categoryName":115,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量",[44,117,91,118,119,120,33,121,122,123,124,125,126,127,128,29,129,130,131,132],"statistical analysis","academic research","implementing","selection","you","reporting","data","for","tests","use","with","test","results","statsmodels","models","help",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":146,"publishedAt":49},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[139,142,143],{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":144,"categoryName":145,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000036","内容创作者",[147,148,149,150,151,152,153,154,155,156,43,157,158,159,160,161,162,32,122,163],"writing","scientific","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","structure","openrouter","Outlines","core skill","features",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":174,"publishedAt":49},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[170,173],{"categoryNo":171,"categoryName":172,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},[175,148,176,177,178,179,180,181,182,155,33,183,184,185,186,187,188,189,124,190],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":192,"name":193,"nameZh":6,"description":194,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":195,"licenseFileUrl":6,"sourceHtmlUrl":196,"categories":197,"keywords":204,"publishedAt":49},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[198,201,202,203],{"categoryNo":199,"categoryName":200,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":140,"categoryName":141,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":144,"categoryName":145,"taxonomyType":26,"taxonomyTypeDesc":27},[147,205,206,207,208,209,175,210,211,33,43,44,212,213,214,215,216,217,148,32],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram",{"skillNo":219,"name":220,"nameZh":6,"description":221,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":222,"categories":223,"keywords":232,"publishedAt":49},"01M716618J5EDTN4RW41JZ4W4Z","hypothesis-generation","Structured hypothesis formulation from observations. Use when you have experimental observations or data and need to formulate testable hypotheses with predictions, propose mechanisms, and design experiments to test them. Follows scientific method framework. For open-ended ideation use scientific-brainstorming; for automated LLM-driven hypothesis testing on datasets use hypogenic.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fhypothesis-generation\u002FSKILL.md",[224,225,228,229],{"categoryNo":114,"categoryName":115,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":226,"categoryName":227,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":88,"categoryName":89,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":230,"categoryName":231,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000032","设计师",[233,234,235,236,237,97,156,33,238,121,43,157,160,148,239,32,163,40,240,123],"hypothesis generation","generation","hypothesis","testing","open","predictions","hypothesis formulation","edit"]