AGENT SKILL

kaggle-grind

Autonomous, disciplined Kaggle competition workflow — the loop that took Playground S6E6 to 19th/2742 (top 0.7%). Use when working a Kaggle competition (Playground/tabular especially): building a model zoo, stacking, deciding what to submit, choosing final submissions, or reacting to a public leaderboard that looks suspiciously high. Encodes the hard-won rules: trust CV not a flip-farmable public LB, hunt and stack genuine public OOF corpora, gate every candidate on the statistical noise floor, hedge two genuine decorrelated finals (never a flip), verify finals every pass, and run a resume-surviving scheduled loop. Triggers: "kaggle", "leaderboard", "OOF", "stack", "which submission", "final selection", "the LB shuffled", "public score", "should I submit".

License 许可
MIT
所属仓库
caezium/skills
最近更新
2026-08-24
Stars
★ 3
Forks
0

01 前言 Front Matter

name
kaggle-grind
description
Autonomous, disciplined Kaggle competition workflow — the loop that took Playground S6E6 to 19th/2742 (top 0.7%). Use when working a Kaggle competition (Playground/tabular especially): building a model zoo, stacking, deciding what to submit, choosing final submissions, or reacting to a public leaderboard that looks suspiciously high. Encodes the hard-won rules: trust CV not a flip-farmable public LB, hunt and stack genuine public OOF corpora, gate every candidate on the statistical noise floor, hedge two genuine decorrelated finals (never a flip), verify finals every pass, and run a resume-surviving scheduled loop. Triggers: "kaggle", "leaderboard", "OOF", "stack", "which submission", "final selection", "the LB shuffled", "public score", "should I submit".

02 资源文件 2

03 SKILL.md 正文

kaggle-grind — the autonomous competition loop

This skill encodes a process, not a model. It reliably converts "disciplined grind" into medal-zone finishes by refusing the traps that sink most teams. It was validated on Playground S6E6: 19th / 2742 (top 0.7%), ▲202 places in the private shakeup, above a Grandmaster — by doing exactly the below.

The one idea that matters most

Public LB ≠ the target. Many comps (especially Playground) score the public LB on a small sample (e.g. 20% of test). Under fragile metrics (balanced accuracy, small-sample AUC) the public top fills with public-LB-probing / "flip" notebooks that reverse-engineer public test-row labels. Those are a no-op on the private set and collapse in the shakeup. Optimize CV. Treat the public LB as one noisy sample, never the objective. (A Grandmaster confirmed this post-S6E6, verbatim.)

The loop (run each of these as a pass; a cron can drive it — see below)

0. Day-1 recon (do this FIRST, not on day 20)

  • Read the LB description: what % is public? Note the metric and whether it's shakeup-prone.
  • Pull the #1 public notebook (kaggle kernels list -c <comp> --sort-by scoreDescending, kaggle kernels pull). Classify it: genuine model vs public-LB-probe. Fingerprints of a flip/probe: many teams at the identical 5-decimal score; 1–2-submission teams at the top; code that edits specific test_row_id -> label using known public scores; hardcoded "bad id" lists.
  • Compute the noise floor: for balanced accuracy on n rows, SE ≈ sqrt(sum_k r_k(1-r_k)/n_k)/K ≈ ~0.0003 at n~500k. Any OOF/LB Δ below ~2·SE (~0.0006) is noise. Write it down; gate everything on it.

1. Build a shared-fold, leak-free OOF zoo

  • ONE fold split across every model: StratifiedKFold(k, shuffle=True, random_state=42). Non-negotiable — it's what makes OOFs stackable. Save each model's OOF (n_train,C) + test (n_test,C) as .npy.
  • Diverse families: LGBM/XGB/CatBoost/HistGB/ExtraTrees + a couple NN families + logreg/kNN. But expect trees to dominate tabular; NNs are often soft. Diversity for the stack, not for standalone strength.
  • Leakage paranoia: fit every y-dependent transform (target encoding, scaler, pseudo-labels) on train[~fold] ONLY. Verify: a shuffled-label control must give ~chance.

2. ★ Hunt and stack GENUINE public OOF corpora (the highest-EV lever)

  • This is usually what breaks your own-zoo ceiling. Search early: kaggle datasets list -s "<comp> oof", -s "<comp> stacker", and scan top notebooks for shared OOF/test-pred datasets. Grandmasters publish these (e.g. cdeotte-style OOF+test dumps).
  • RIGOR before trusting any external OOF: id-align it to your train/test order, then verify its OOF argmax balanced-accuracy on your y lands in the sane model range (e.g. 0.95–0.97), NOT ~chance. A file that scores chance is misaligned — reject it. (A naive misaligned blend cost us a submission once.)
  • Stacking public OOFs (train-side, out-of-fold, id-aligned) is legitimate and private-valid — categorically different from copying a flip submission.

3. Meta-stack + decision rule

  • Multinomial Logistic Regression on log-prob (logit) features of all OOFs. Regularized LR generalizes where Caruana hill-climbing overfits. Sweep C.
  • Correlation-prune first: keep a model only if its argmax labels disagree with every selected model on

    ~0.8% of rows. Diversity, not count, drives the gain.

  • Decision rule (metric-specific, often the biggest single lever): for balanced accuracy, tune a per-class log-offset [0,a,b] added to log-probs before argmax — argmax under-predicts minorities.

4. Gate, probe, accept

  • Accept a candidate only if OOF Δ exceeds the noise floor AND (ideally) an LB probe confirms. Many "higher OOF" candidates LOSE on LB — trust LB-held gains, distrust 4th-decimal OOF nudges.
  • BUT on the final day with free submission budget, PROBE every distinct genuine candidate anyway — the noise floor bounds confidence, not the value of a cheap real data point. (Our final #1 was "within noise" on OOF yet +0.0011 on LB.)

5. Finals (the medal decision — rule #13)

  • Pick two GENUINE, decorrelated submissions. NEVER a flip as a final (vanity on public, no-op/risk on private). Submit strong candidates EARLY (dense plateaus tiebreak by submission time).
  • Verify the selected finals EVERY pass once submissions exist — assert both are genuine. (A flip had auto-slotted into final #1; caught on deadline day. That check alone can save the medal.)
  • Manually pin — don't let Kaggle auto-pick two correlated public-bests.

Anti-patterns (proven to waste time / lose)

  • Treating a flip-farmed public LB as the ceiling. (We wasted ~10 days on this once.)
  • Writing an unqualified "ceiling" conclusion. Scope it: "ceiling of OUR models" + always ask "what would a stronger external model add?" Our "genuine ceiling" was wrong twice — public OOFs broke it.
  • Chasing 4th-decimal OOF gains, more tree-FE, or more correlated GBDTs. Cats that are binned color-indices, external data with high adversarial AUC, iterative pseudo — usually ≤+0.0002 or negative.
  • Local-eval "improvements" that aren't verified to generalize (they overfit your exact local set).

Running it as a durable loop (optional)

  • Use the host harness's recurring-monitor or scheduled-task mechanism to fire a self-contained pass prompt. In Claude Code this may be a cron integration; in Codex/T3 use the product's recurring monitoring flow. Guard with a lock file (/tmp/<comp>_loop.lock, epoch + TTL) so passes don't overlap. /tmp is wiped on reboot — re-download public OOF datasets each pass. Use nohup python -u ... for long jobs so they survive session resumes.
  • Convergence downgrade: if N consecutive passes find no candidate above the noise floor AND no new public OOF source, drop cron frequency (hourly → 6h) — don't burn tokens re-confirming a ceiling.
  • CLI/submit caveat: Kaggle's competitions.*/kernels.* API can 401 for hours while datasets.* works. Confirm your real submit path EARLY (CLI vs browser-upload vs user-drag). The browser file-upload tool is sandboxed to session-shared paths — it can't upload arbitrary repo files, so a human may need to drag-drop.

Reference

See references/s6e6_retrospective.md for the full worked example (every lever, the exact numbers, the 7 workflow improvements) and references/repro_stack.py for the winning stack recipe.

04 同仓库更多 Skills

caezium
caezium/skills

writing-for-agents

Writing documents for agents. Use when creating or editing skills, or modifying AGENTS.md or CLAUDE.md.

内容创作者写作与文案
★ 3
caezium
caezium/skills

wrangler

Cloudflare Workers CLI for deploying, developing, and managing Workers, KV, R2, D1, Vectorize, Hyperdrive, Workers AI, Containers, Queues, Workflows, Pipelines, and Secrets Store. Load before running wrangler commands to ensure correct syntax and best practices. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.

后端与接口
★ 3
caezium
caezium/skills

workers-best-practices

Reviews and authors Cloudflare Workers code against production best practices. Load when writing new Workers, reviewing Worker code, configuring wrangler.jsonc, or checking for common Workers anti-patterns (streaming, floating promises, global state, secrets, bindings, observability). Biases towards retrieval from Cloudflare docs over pre-trained knowledge.

★ 3
caezium
caezium/skills

wizard

Generate an interactive bash wizard that walks a human through steps only they can perform. Use when provisioning infrastructure, setting up credentials or CI secrets, walking an unfamiliar third-party dashboard, or running a one-off migration or cutover. Don't invoke this for steps the agent can perform itself.

★ 3
caezium
caezium/skills

why

Use for 'why does X work this way', 'why we picked Y', design rationale, regressions, postmortems, or data-backed thresholds. Discovers available evidence sources, searches each relevant category, and returns a cited read on decisions and tradeoffs. Use how for runtime behavior.

★ 3
caezium
caezium/skills

week-prep

Generate a briefing with calendar, tasks, and attention flags. Use for "week prep", "briefing", or "what's on this week".

Skill与智能体开发
★ 3