since-cutoff
Find which dependency APIs this project's Python code uses changed after the model's training cutoff (no model calls), write short AGENTS.md or CLAUDE.md notes from the API diff, each with its source, and keep them in step with the lockfile; optionally measure which of the changes the model gets wrong. Use when the user asks whether the model knows their library versions, when code keeps failing on renamed or removed library APIs, or after upgrading dependencies. Trigger with requests such as "does the model know my openai version", "scan my dependencies for API changes" or "check whether the library notes are up to date".
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 1.6k
- Forks
- 491
01 前言 Front Matter
- name
- since-cutoff
- license
- MIT
- metadata
- { "author": "Mohammad Hijjawi", "version": "0.5.0" }
- description
- Find which dependency APIs this project's Python code uses changed after the model's training cutoff (no model calls), write short AGENTS.md or CLAUDE.md notes from the API diff, each with its source, and keep them in step with the lockfile; optionally measure which of the changes the model gets wrong. Use when the user asks whether the model knows their library versions, when code keeps failing on renamed or removed library APIs, or after upgrading dependencies. Trigger with requests such as "does the model know my openai version", "scan my dependencies for API changes" or "check whether the library notes are up to date".
- allowed-tools
- Bash(since-cutoff scan:*), Bash(since-cutoff sync:*), Bash(since-cutoff status:*), Bash(since-cutoff run:*), Bash(since-cutoff models:*), Bash(since-cutoff unapply:*), Bash(uvx since-cutoff:*), Bash(uvx --with basedpyright since-cutoff:*), Bash(pipx run since-cutoff:*), Read
- compatibility
- Needs since-cutoff on PATH, or uv or pipx to run it, and network access to PyPI. The run command also needs model access (the claude CLI or a provider API key) and the basedpyright type checker (pip install "since-cutoff[run]", or uvx --with basedpyright since-cutoff run); scan, sync and status make no model calls, and status works offline.
03 SKILL.md 正文
since-cutoff
since-cutoff is a command-line tool that finds the APIs a project's dependencies changed after
the model's training cutoff and writes notes about them, each with its source. Run the tool and
report what it prints.
Overview
since-cutoff scancompares, for each dependency, the public API of the latest release published at least 30 days before the model's training cutoff (--cutoff-margin; 0 compares from the cutoff itself) with the version the project pins (a static diff, no model calls), and lists the changed APIs the code uses.since-cutoff syncwrites notes from that diff into a marked block in AGENTS.md or CLAUDE.md and keeps them in step with the lockfile;since-cutoff statuschecks the block offline, andsince-cutoff unapplyremoves it.since-cutoff runis optional. It does the measuring itself by calling a fresh copy of the model with no tools and no project context, so do not answer the probe tasks yourself and do not guess results.
Prerequisites
- A Python project with its dependencies in
pyproject.toml,requirements.txtor a lockfile. - since-cutoff on PATH, or uv or pipx to run it (
uvx since-cutoff,pipx run since-cutoff). - Network access to PyPI;
statusworks offline. - For
runonly: model access, through theclaudeCLI for Claude Code models or the provider's API key for other models, and the basedpyright type checker:pip install "since-cutoff[run]", or run it asuvx --with basedpyright since-cutoff run. Without it,runstops at once and says so.scan,syncandstatusmake no model calls.
Instructions
- Work from the project root (the directory with
pyproject.toml,requirements.txtor a lockfile). - Pick the command. Use the arguments the user gave, if any (
scan,sync,statusorrun, with flags such as--apply,--quickor--model); otherwise:- quick look, no model calls:
since-cutoff scan(then offersync, step 7) - full measurement, with notes a model writes and keeps only when their example
type-checks:
since-cutoff run --quick
- quick look, no model calls:
- Name the model. Without
--model, the tool tests the model your coding agent is set up with (SINCE_CUTOFF_MODEL; inside Claude Code, only Claude Code's settings; elsewhere the Claude Code, Codex, Gemini CLI, OpenCode and Aider settings, the project's before the user's) and says where it read it ("model from ..."). If that is not the model you are, or it warns that no model setting was found, add--model PROVIDER:MODELfor the model you are (for example--model openai:gpt-5.4, or--model claude-code:MODELfor a model picked with/model).runcalls Claude Code models through theclaudeCLI and other models through their provider's API key. - Before
run, tell the user that it sends prompts (package names, versions, public API signatures and generated tasks, never their source code) to the model provider they choose, uses their API credits or Claude Code usage, and can take 5-20 minutes. Wait for a yes.scanneeds no confirmation. - Run it. If
since-cutoffis not installed, useuvx since-cutoffwith the same arguments (or usepipx run since-cutoff); forrun,uvx --with basedpyright since-cutoff run, which adds the type checker. Startrunin the background or with a long timeout, not a 2-minute foreground call. Everything is cached, so re-running after an interruption resumes quickly. - Summarise the result card: the model and its training cutoff, how many dependencies changed
after the cutoff, what was stale, and the held-out before/after numbers. Quote each interval
with the number it belongs to: the bootstrap CI goes with the difference of the task-level
rates, the other CI with "changes fixed: X of Y". If the run compared baseline notes
(
--compare), give each block's "changes fixed" with its CI, and call one block better only when report.md's head-to-head sign test for it has a small p-value. - Only write notes into the user's files if they asked for it. After
scan, runsince-cutoff sync --dry-run, show the user the diff it prints, and only when they agree runsince-cutoff sync --yes: it writes the notes from the API diff into a marked block in AGENTS.md (or CLAUDE.md if that is the file the project uses) and, run again later, keeps them in step with the lockfile. Whenscansuggestssync --scope imported(the code uses none of the changed APIs yet), add it to both commands. When the user asks for the configuration the benchmark measured, use--profile evaluated; for the shortest block (the changed packages and their versions, no change lines),--header-only. Afterrun,--applywrites that run's notes in place of the whole block;since-cutoff syncthen adds the notes from the diff for the other changed APIs the code uses, keeping the run's[type-checked]ones.since-cutoff unapplyremoves the block again.since-cutoff statussays, without the network, whether the notes still match the lockfile; from since-cutoff 0.4.0 the Claude Code plugin runs it when a session starts (the first time, uvx downloads since-cutoff).
Output
- The terminal: the model, its training cutoff and where the tool read the model, the
dependencies that changed since the release each is compared from, and the changed APIs the
code uses, each with its note.
runadds the result card of step 6. .since-cutoff/report.md: the full report. Read it when the user wants details.- The notes block in AGENTS.md or CLAUDE.md, written only by
sync --yesorrun --apply. The notes in it are API reference facts about the versions the project pins: what was removed, renamed or deprecated after the release they compare from (the latest one published at least 30 days before your training cutoff), each tagged with what was checked, for example[diff](a static comparison of the two releases' public APIs),[diff + library](the library's own deprecation text names the replacement),[diff + move checked],[diff + metadata](the two releases' Requires-Dist: the package requires another library, such ashttpx2instead ofhttpx, and its API takes that library's types),[diff; probable rename](a guess, labelled as one) or[type-checked](written duringrun; its example passed the type check). Check them when you write code that uses those libraries, and claim no more than the tag says.
Error Handling
- No lockfile or pinned versions found: tell the user which file the tool asked for; do not invent versions.
- A model or login error during
run: report the message and offerscan, which makes no model calls. syncexits with code 4: the block was edited by hand. Tell the user, and pass--forceonly if they say so.syncwithout--yesasks on the terminal; with no terminal to ask in it writes nothing and exits with code 3. Show the--dry-rundiff and use--yesonce the user agrees.statusexits with code 3 when the notes are out of date: offersyncas in step 7.runwas interrupted: run the same command again; it resumes from the cache.
Examples
Example 1: "Does the model know the library versions this project pins?"
since-cutoff scan
Summarise what it lists. If the user wants notes, show the diff and write once they agree:
since-cutoff sync --dry-run
since-cutoff sync --yes
Example 2: "Measure which of these changes you get wrong, and add the notes."
Tell the user what run sends, what it costs and how long it takes (step 4). After a yes, start
it in the background (with --model if step 3 calls for it), then show the notes sync would
add for the other changed APIs:
since-cutoff run --quick --apply
since-cutoff sync --dry-run
Example 3: "I upgraded openai. Are the notes in AGENTS.md still right?"
since-cutoff status
If the notes are out of date, show the diff of since-cutoff sync --dry-run and run
since-cutoff sync --yes once the user agrees; to remove them, since-cutoff unapply.
Quick lookups without a run
If the since-cutoff MCP server is connected (the Claude Code plugin starts it), its tools answer
from a static diff, with no model calls: api_changes (one package, optionally one symbol),
project_changes (every dependency of the project) and model_cutoff. Pass your own model id
as model. Use them before writing code against a dependency that may be newer than your
training data; use the CLI above when the user wants the model measured or notes written.
Resources
- since-cutoff on GitHub: the README, with every command, flag and exit code, and what each note tag means.
- How it works: the diff, the notes and the measurement in detail.
- Privacy: what
runsends to the model provider, and what stays on the machine. - Changelog: what changed in each release.
04 同仓库更多 Skills
worktable-review-with-annotations
Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.
worktable-find-and-synthesize-context
Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.
worktable-create-or-update-docs
Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.
worktable-create-or-manage-records
Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.
worktable-create-or-manage-interactive-html
Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.
worktable-collaborate-threads
Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.