AGENT SKILL

benchmarking

暂无简介

License 许可
MIT
所属仓库
nubjs/nub
最近更新
2026-11-14
Stars
★ 4.4k
Forks
71

03 SKILL.md 正文

benchmarking

Honest, reproducible comparative install benchmarks of nub install against npm / pnpm / bun — the external, wall-clock-and-file-count method. For decomposing where the time goes INSIDE a single nub install, use pm-perf-tracing instead.

Applies to: benchmarking against another PM, refreshing homepage/blog install numbers, verifying a perf claim before it ships. A single non-genuine cell discredits the whole table.

The tool: hyperfine

/opt/homebrew/bin/hyperfine is the canonical timer. The load-bearing flag is --prepare, which runs setup BEFORE each timed run, untimed. Never time setup.

# COLD: empty the tool's REAL cache + wipe node_modules before each run (untimed), then time the install.
hyperfine --warmup 0 --runs 5 \
  --prepare 'rm -rf node_modules && d="$(bun pm cache)" && rm -rf "${d:?}"' \
  'bun install --ignore-scripts'

# WARM-RELINK: cache populated, wipe ONLY node_modules before each run.
hyperfine --warmup 1 --runs 5 \
  --prepare 'rm -rf node_modules' \
  'nub install --ignore-scripts'

# WARM-SAT: node_modules already present (idempotency path) — no prepare wipe.
hyperfine --warmup 1 --runs 5 'nub install --ignore-scripts'

Prefer median + spread (min–max) over the mean — contention skews the mean.

The cold / warm protocol

node_modules is deleted before EVERY timed run, cold and warm, always in --prepare. The cold/warm axis differs only in global-cache state:

  • cold — the tool's real cache is EMPTY (genuine download).
  • warm-relink — cache populated, node_modules wiped (the link-from-store path; the homepage number).
  • warm-sat — node_modules already present; a separate, clearly-labeled scenario, not the headline.

GENUINE-COLD per tool — the cache is on DISK

A "cold" run is only cold if the tool's real on-disk cache is gone. Setting a cache-dir env var does not guarantee that — bun ignores BUN_INSTALL_CACHE_DIR and $HOME, resolving its cache via the OS passwd home. You must wipe the disk path.

tool real cache path clear command for cold
nub its store (NUB_CACHE_DIR + XDG_DATA_HOME/XDG_CACHE_HOME) rm -rf "$NUB_CACHE_DIR" "$XDG_DATA_HOME" "$XDG_CACHE_HOME"
npm ~/.npm/_cacache (or --cache <dir>) rm -rf <cache> (the --cache dir you pass)
pnpm pnpm store path (or --store-dir <dir>) rm -rf <store> (or pnpm store prune)
bun bun pm cache = the real ~/.bun/install/cache rm -rf "$(bun pm cache)" — env vars do NOT relocate it

VERIFY each cold is genuine — the offline-fails check

After wiping a cache, an --offline install MUST FAIL. A pass means the cache wasn't cleared (wrong disk path), and any "cold" number from it is a warm-link artifact. Run this for every tool before trusting a cold number:

tool expected after cache wipe genuine?
nub rc≠0, "not available in the local cache" FAIL → genuine
pnpm rc=1, ERR_PNPM_NO_OFFLINE_TARBALL FAIL → genuine
npm rc≠0, cache-miss error FAIL → genuine
bun if bun install --offline SUCCEEDS → cache NOT cleared → bun cold is NOT genuine

A true bun cold needs a clean container, or wiping the user's real cache (destructive — don't, unless in Docker).

Apples-to-apples isolation

  • Each tool gets its OWN cache dir.
  • --ignore-scripts for ALL tools (or --allow-scripts for all — same on both sides).
  • Identical fixture, identical lockfile throughout (use a --frozen-lockfile/--frozen equivalent where the tool offers one).
  • Interleave tool order round-robin (nub → npm → bun → pnpm, repeat) — never all-of-one-then-the-other — so drift in host load hits every tool equally.

Load discipline — measure only on a quiet machine

  • Check the load average before measuring and only proceed when the machine is quiet. A dedicated quiet box or a CI runner near zero load is ideal for anything that will be published.
  • On a shared dev host, WAIT for load to fall below a threshold — don't measure above it. A practical gate is a 1-minute load average under ~40 (pick a threshold the machine actually reaches; below ~5 is ideal on a dedicated box). Poll, wait, then run; abort and retry if it spikes mid-run.
  • Report the load that held during the run alongside the numbers.
  • File counts and store-entry counts are exact and load-independent — a robust cross-check, and the clearest way to tell the dedup story. They complement a properly-measured time, not replace it.

File-count forensics (the load-independent crux)

find node_modules -type f -o -type l | wc -l          # total materialized entries
ls -d node_modules/**/core-js node_modules/core-js    # physical copies of a duplicated dep (dedup story)

Reach for this first — exact, reproducible, immune to load.

The honesty bar (anti-juicing)

  • VERIFY every cold is genuine (offline-fails check) BEFORE citing it.
  • NEVER compare one tool's genuine-cold to another's warm-link — the exact misleading comparison the offline check exists to prevent.
  • Report what was actually measured, caveats included (which colds are genuine, host load, sample size).
  • The homepage cites the WARM number because it is the honest, reproducible one.
  • A single non-genuine cell discredits the whole table. When in doubt, exclude the cell and say why.

Process hygiene (this runs on the maintainer's machine)

Short-lived install/measurement processes reap themselves. The hazard is a long-lived process a bench starts and forgets:

  • Never leave a dev server running. If a bench starts one, tear it down in the same run (trap '…kill…' EXIT INT TERM).
  • Docker: docker run --rm, and confirm docker ps is empty when done.

Reference template

/tmp/cs-bench-final.sh is a working 4-tool harness (nub / npm / bun / pnpm) — per-tool isolated cache dirs, --ignore-scripts for all, interleaved round-robin order, node_modules wiped before every run, median/spread helpers. Read its structure before hand-rolling a new one; adapt the cache paths and fixture, keep the protocol.

Internal decomposition

When a comparative number raises "WHY is nub's phase X slow?", switch to pm-perf-tracing: RUST_LOG=debug nub install for the phase:resolve/fetch/link split, and the gated AUBE_DIAG_FILE per-file linker strategy tally.

04 同仓库更多 Skills

nubjs
nubjs/nub

nub

暂无简介

★ 4.4k
nubjs
nubjs/nub

nub

This project uses nub, a single Rust CLI that runs on the project's installed Node and replaces the everyday Node toolchain. ALWAYS prefer nub over node, bun, npm, npx, pnpm, and yarn for: executing JS/TS files, running package.json scripts, the npx-equivalent, installing dependencies, watch mode, and Node version management. Surfaces whenever you would otherwise reach for node/bun/npm/npx or a package manager.

★ 4.4k
nubjs
nubjs/nub

worktree

Create and manage isolated git worktrees for parallel build/test/landing work on the nub repo. Invoke (via the Skill tool) whenever you need a fresh worktree to land a change, when you want to know what `.worktreeinclude` does or how to add an entry, or when cleaning up after a merge. Encodes the one-command setup (`nub scripts/new-worktree.ts <slug>` or `node …`) that bakes in the proven recipe — worktree off origin/main (vendor/aube is plain in-tree files now, no submodule init), the shared CARGO_TARGET_DIR (`~/.cache/nub/shared-target`) that all worktrees reuse, and applying `.worktreeinclude` — plus the eagerly-pull-the-shared-tree discipline and the safe cleanup path. Pairs with the `dev-loop` build skill.

★ 4.4k
nubjs
nubjs/nub

windows-vm-test

Run ad-hoc Nub tests and debugging probes in the maintainer's existing local Windows 11 ARM64 QEMU/HVF VM when its external disk and runbook are present. Use for interactive win32-arm64 or Windows-on-ARM x64-emulation checks; use ci-adhoc-test for true x64-native behavior or when the local VM is unavailable.

★ 4.4k
nubjs
nubjs/nub

visual-review

Verify UI/layout/styling changes are correct by computing occlusion, clipping, and alignment from the browser's resolved paint order via the chrome-devtools MCP `evaluate_script` tool — instead of eyeballing a flat screenshot. Invoke BEFORE declaring any UI, site, or styling/layout change correct. Screenshots have no depth buffer, so z-index/occlusion/clip bugs are exactly where "just look at it" fails; the `evaluate_script` routines below turn those fuzzy visual judgments into deterministic measurements — including optical center-of-mass, which measures the glyph ink's true visual center so differently-sized labels can be aligned by more than eye.

★ 4.4k
nubjs
nubjs/nub

type-declarations

Add, audit, or update @nubjs/types declarations for Nub runtime APIs. Use whenever Nub adds or changes a user-visible global, built-in method, module loader type, or TC39 proposal; when TypeScript or @types/node gains an overlapping declaration; and before every release to reconcile runtime changes since the previous tag. Covers conflict-safe global augmentation, compiler-version routing, fixture matrices, and package verification.

★ 4.4k