challenge-sciml-model
Challenge SciML evidence for leakage, physical constraints, extrapolation, uncertainty, baselines, and cost.
- License 许可
- MIT
- 所属仓库
- dasobral/skills
- 最近更新
- 2026-08-15
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- challenge-sciml-model
- license
- MIT
- metadata
- { "plugin": "scientific-claim-ledger", "version": "0.1.0" }
- description
- Challenge SciML evidence for leakage, physical constraints, extrapolation, uncertainty, baselines, and cost.
02 资源文件 2
03 SKILL.md 正文
Challenge SciML Model
Use references/sciml-challenge-result.schema.json to record every check.
Required challenges
- Grouped, temporal, spatial, and regime-aware splits
- Preprocessing and calibration leakage
- Extrapolation beyond observed regimes
- Conservation, invariance, positivity, and boundary conditions
- Residual error versus solution and quantity-of-interest error
- Uncertainty coverage and sharpness
- Seed and initialization sensitivity
- Strong baselines compared at matched error
- End-to-end training, inference, hardware, memory, and energy cost
Workflow
- Pin datasets, split identities, preprocessing state, model revision, seeds, hardware, and evaluation commands.
- Record each evidence check as
pass,fail,unknown,not-applicable, orevidence-gap, with evidence and rationale. Keep these evidence states separate fromblock,review-required, and other workflow decisions. - Keep objective contract failures distinct from model-adequacy findings.
- Run
python3 scripts/check_sciml_challenge.py <challenge-result.json>. - Block only forbidden calibration/validation overlap or declared objective
constraint failures. Escalate model adequacy as
review-required.
Rules
- Never hide failed seeds or regimes in aggregate metrics.
- Never compare cost or accuracy at unmatched error targets.
- Agents may challenge and summarize evidence but cannot make the final pass decision.
04 同仓库更多 Skills
fixture-skill
Fixture skill used to prove trust inventory coverage.
write-conformant-code
Write code in any language that matches project conventions exactly. Reads CODING_REQUIREMENTS.md, prior review output, or built-in defaults before writing. Part of Codecraft pipeline: analyze → write → review → audit. Use when implementing features, functions, or any code that should fit seamlessly into an existing codebase.
test-crypto-interoperability
Record reproducible cryptographic interoperability cases, including negotiation, fallback, sizes, latency, fragmentation, and negative tests.
scientific-platform-architect
Design scientific computing platforms combining HPC kernels with modern infrastructure. Enforces compute-layer separation (Rust/PyTorch/Python), emulator-first thinking, and right-sized infrastructure. Use for MCMC pipelines, neural emulators, platform design, or scientific code review.
rust-systems
Rust systems engineering for daemons, HTTP APIs, and AOS components. Enforces Tokio/Axum async-first, LiteLLM-compatible OpenAI APIs, workspace layouts, tracing, and systemd-compatible daemon design. Part of AOS Stack plugin. Use for .rs files, Cargo.toml, daemons, or Rust orchestrator services.
run-agent-attack-replay
Aggregate isolated agent attack trials with deterministic side-effect assertions, Wilson intervals, and preserved trace evidence.