[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6M7BXMKSR2554WHW7P8JHDF":3,"skill-same:01M6M7BXMKSR2554WHW7P8JHDF":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":42,"linkOnly":43,"markdownContent":44,"markdownBody":45,"frontMatter":46,"refName":47,"commitSha":48,"skillDirectory":49,"forksCount":50,"repositoryHtmlUrl":51,"repositoryHomepageUrl":52,"ownerHtmlUrl":53,"ownerBlogUrl":54,"ownerDescription":55,"updatedAt":56,"install":57,"resources":70},"01M6M7BXMKSR2554WHW7P8JHDF","testing",null,"Select, run, and write CCF unit, end-to-end, partition, compatibility, coverage, and Python SDK tests. Covers prerequisites, the tests.sh wrapper, test labels, and e2e registration.","microsoft\u002FCCF","microsoft","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6154722?v=4","01M698VYDXZ6JW5WH3CRV650R8","Microsoft","01M6M7BPRHNJVQCKMZJK6GEEB5",879,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FCCF\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Ftesting\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",[5,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41],"coverage","labels","unit","false","end","tests","registration","compatibility","Wrapper","partition","write","python sdk","select","e2e","run","test","sdk","python","2026-11-10 10:12:30",false,"---\nname: testing\nuser-invocable: false\ndescription: \"Select, run, and write CCF unit, end-to-end, partition, compatibility, coverage, and Python SDK tests. Covers prerequisites, the tests.sh wrapper, test labels, and e2e registration.\"\n---\n\n# Testing\n\n## Running tests\n\nFollow the global validation policy for choosing relevant tests and reporting blockers. Before running tests, configure and build the affected targets using the documented development environment. The Copilot formatting\u002Flint setup alone is not a full test environment.\n\nFrom the repository root, enter the configured build directory (shown here as `build`). Use its generated `tests.sh` wrapper for e2e tests: it creates\u002Factivates a Python venv, installs the SDK and test dependencies, then invokes `ctest`. First-time setup requires Python venv support and network access; subsequent runs still invoke dependency installation.\n\n```bash\ncd build\n.\u002Ftests.sh -N                       # Discover registered tests; does not run them\n.\u002Ftests.sh -L '^unit$' --no-tests=error\n.\u002Ftests.sh -L '^e2e$' --no-tests=error\n.\u002Ftests.sh -L '^partitions$' --no-tests=error  # Requires NET_ADMIN\n.\u002Ftests.sh -VV --timeout 360 -R '^recovery_test$' --no-tests=error\n```\n\nUse `-R` for name regexes and `-L` for labels. `.\u002Ftests.sh` without a filter runs all registered tests; do not make this the default for a small change. Pure C++ unit tests and test discovery may use `ctest` directly from the build directory without Python setup. Use `--no-tests=error` for execution so an empty selection is not mistaken for passing tests.\n\nLabels include `unit`, `e2e`, `partitions`, `perf`, `benchmark`, `cft_scenario`, `suite`, `lts_compatibility`, `snp`, and CI routing labels `bucket_a`, `bucket_b`, `bucket_c`. Registration depends on build options; inspect the configured inventory rather than assuming a named test exists.\n\nThe release workflow sets `CCF_USE_RELEASED_PYTHON_PACKAGE=1` so `tests.sh`, installed-package tests, and sandboxes install the latest released `ccf` package from PyPI with `uv pip install --upgrade --reinstall-package ccf ccf`, replacing any previously installed editable SDK.\nWithout this flag, development and normal CI tests retain their editable local SDK installation.\nInstalled sandboxes still default to the SDK version matching the installed CCF release.\nAn existing caller-supplied `VENV_DIR` is reused without reinstalling packages; `tests.sh` selects the SDK before exporting its environment to child sandboxes.\n\n### Compatibility\n\nFor changes affecting older releases, configure the intended build directory with `-DLONG_TESTS=ON` (a CMake option, not just a shell variable), rebuild affected targets, then run from that directory:\n\n```bash\n.\u002Ftests.sh -R '^lts_compatibility$' --no-tests=error\n```\n\n`LONG_TESTS` enables additional ledger compatibility coverage. This test is not registered with `SAN=ON`; use a suitable separate build rather than silently skipping it. It also needs access to older releases; report unavailable downloads as blockers.\n\n### Python SDK\n\nSDK tests are separate from e2e. From the repository root, in an activated Python venv meeting `python\u002Fpyproject.toml`'s Python requirement, install the SDK and pytest as the SDK CI job does:\n\n```bash\nuv pip install -e .\u002Fpython pytest\ncd python\npytest\n```\n\nExit status 0 indicates success; report the selected tests and their actual result, not merely a successful setup step.\n\n## Code coverage\n\nConfigure with `-DCOVERAGE=ON`, build instrumented targets, and run the selected tests. From that build directory, with `llvm-profdata` and `llvm-cov` available:\n\n```bash\n..\u002Fscripts\u002Fcoverage.sh                  # Print summary\n..\u002Fscripts\u002Fcoverage.sh --html report\u002F   # Generate HTML report\n```\n\nThese paths assume `build` is immediately under the repository root. For another layout, use the actual path to `scripts\u002Fcoverage.sh` while retaining the build working directory. The script consumes `.profraw` files and the generated `coverage_binaries.txt`; building alone does not produce coverage.\n\nTo combine coverage from another build directory of the same revision, compiler and options, such as one on a different platform, run `..\u002Fscripts\u002Fcoverage.sh --merge-only -o \u003Cname>.profdata` there, then pass the resulting file to the reporting run with `--add-profile \u003Cname>.profdata`. llvm-cov excludes functions compiled differently from reports and warns that they have mismatched data.\n\nText and HTML reports include only framework sources under `src\u002F` and `include\u002F`, excluding test and performance code. First-party C++ static libraries are instrumented, but only linked binaries are report inputs; archives are not counted a second time. Their coverage runtime link requirement also applies to consumers of installed instrumented libraries. Rust is not instrumented by the C++ coverage flags.\n\nFor a baseline, use a fresh coverage build\u002Fprofile directory and record the source revision, build options and exact test selection. Targeted tests do not establish a full-suite baseline. Reports predating library instrumentation have a different denominator and are not directly comparable; including previously invisible code can lower the headline percentage.\n\n## End-to-end test infrastructure\n\nE2e tests use the infrastructure in `tests\u002Finfra\u002F`. The key classes are:\n\n- `infra.network.Network`: manages a multi-node network; use the existing `infra.network.network(...)` context-manager pattern for lifecycle cleanup.\n- `infra.node.Node`: represents one node process.\n- `infra.consortium.Consortium`: member governance operations.\n- `infra.runner.ConcurrentRunner`: schedules `run_*(args)` functions, each owning its network.\n\n## Writing e2e tests\n\nFollow a nearby test for the same application. In `tests\u002Fe2e_logging.py`, individual test cases accept `(network, args)`, while `run(args)` creates\u002Fopens the network and calls those cases. Requirement decorators from `suite.test_requirements` express the case's prerequisites.\n\n- Register the network-owning `run_*(args)` function with `ConcurrentRunner.add`, not a `(network, args)` case. The runner invokes its target with one argument.\n- `ConcurrentRunner.add(prefix, target, **args_overrides)` already deep-copies arguments and assigns a distinct label. Prefer its overrides for separate configurations rather than duplicating that work. Deep-copy arguments and set a distinct label yourself only when manually creating an additional independent configuration outside that mechanism.\n- Reuse network\u002Fclient\u002Fgovernance helpers; assert observable behaviour and clean up through existing context managers.\n- Ensure the case is called by a runner. For a new e2e executable entry point, follow existing CMake `add_e2e_test` registration, including its CI bucket. Run `scripts\u002Ftest-buckets-checks.sh` from the repository root when changing inventory, and update `tests\u002Fci-buckets.txt` only for intentional registration changes.\n","# Testing\n\n## Running tests\n\nFollow the global validation policy for choosing relevant tests and reporting blockers. Before running tests, configure and build the affected targets using the documented development environment. The Copilot formatting\u002Flint setup alone is not a full test environment.\n\nFrom the repository root, enter the configured build directory (shown here as `build`). Use its generated `tests.sh` wrapper for e2e tests: it creates\u002Factivates a Python venv, installs the SDK and test dependencies, then invokes `ctest`. First-time setup requires Python venv support and network access; subsequent runs still invoke dependency installation.\n\n```bash\ncd build\n.\u002Ftests.sh -N                       # Discover registered tests; does not run them\n.\u002Ftests.sh -L '^unit$' --no-tests=error\n.\u002Ftests.sh -L '^e2e$' --no-tests=error\n.\u002Ftests.sh -L '^partitions$' --no-tests=error  # Requires NET_ADMIN\n.\u002Ftests.sh -VV --timeout 360 -R '^recovery_test$' --no-tests=error\n```\n\nUse `-R` for name regexes and `-L` for labels. `.\u002Ftests.sh` without a filter runs all registered tests; do not make this the default for a small change. Pure C++ unit tests and test discovery may use `ctest` directly from the build directory without Python setup. Use `--no-tests=error` for execution so an empty selection is not mistaken for passing tests.\n\nLabels include `unit`, `e2e`, `partitions`, `perf`, `benchmark`, `cft_scenario`, `suite`, `lts_compatibility`, `snp`, and CI routing labels `bucket_a`, `bucket_b`, `bucket_c`. Registration depends on build options; inspect the configured inventory rather than assuming a named test exists.\n\nThe release workflow sets `CCF_USE_RELEASED_PYTHON_PACKAGE=1` so `tests.sh`, installed-package tests, and sandboxes install the latest released `ccf` package from PyPI with `uv pip install --upgrade --reinstall-package ccf ccf`, replacing any previously installed editable SDK.\nWithout this flag, development and normal CI tests retain their editable local SDK installation.\nInstalled sandboxes still default to the SDK version matching the installed CCF release.\nAn existing caller-supplied `VENV_DIR` is reused without reinstalling packages; `tests.sh` selects the SDK before exporting its environment to child sandboxes.\n\n### Compatibility\n\nFor changes affecting older releases, configure the intended build directory with `-DLONG_TESTS=ON` (a CMake option, not just a shell variable), rebuild affected targets, then run from that directory:\n\n```bash\n.\u002Ftests.sh -R '^lts_compatibility$' --no-tests=error\n```\n\n`LONG_TESTS` enables additional ledger compatibility coverage. This test is not registered with `SAN=ON`; use a suitable separate build rather than silently skipping it. It also needs access to older releases; report unavailable downloads as blockers.\n\n### Python SDK\n\nSDK tests are separate from e2e. From the repository root, in an activated Python venv meeting `python\u002Fpyproject.toml`'s Python requirement, install the SDK and pytest as the SDK CI job does:\n\n```bash\nuv pip install -e .\u002Fpython pytest\ncd python\npytest\n```\n\nExit status 0 indicates success; report the selected tests and their actual result, not merely a successful setup step.\n\n## Code coverage\n\nConfigure with `-DCOVERAGE=ON`, build instrumented targets, and run the selected tests. From that build directory, with `llvm-profdata` and `llvm-cov` available:\n\n```bash\n..\u002Fscripts\u002Fcoverage.sh                  # Print summary\n..\u002Fscripts\u002Fcoverage.sh --html report\u002F   # Generate HTML report\n```\n\nThese paths assume `build` is immediately under the repository root. For another layout, use the actual path to `scripts\u002Fcoverage.sh` while retaining the build working directory. The script consumes `.profraw` files and the generated `coverage_binaries.txt`; building alone does not produce coverage.\n\nTo combine coverage from another build directory of the same revision, compiler and options, such as one on a different platform, run `..\u002Fscripts\u002Fcoverage.sh --merge-only -o \u003Cname>.profdata` there, then pass the resulting file to the reporting run with `--add-profile \u003Cname>.profdata`. llvm-cov excludes functions compiled differently from reports and warns that they have mismatched data.\n\nText and HTML reports include only framework sources under `src\u002F` and `include\u002F`, excluding test and performance code. First-party C++ static libraries are instrumented, but only linked binaries are report inputs; archives are not counted a second time. Their coverage runtime link requirement also applies to consumers of installed instrumented libraries. Rust is not instrumented by the C++ coverage flags.\n\nFor a baseline, use a fresh coverage build\u002Fprofile directory and record the source revision, build options and exact test selection. Targeted tests do not establish a full-suite baseline. Reports predating library instrumentation have a different denominator and are not directly comparable; including previously invisible code can lower the headline percentage.\n\n## End-to-end test infrastructure\n\nE2e tests use the infrastructure in `tests\u002Finfra\u002F`. The key classes are:\n\n- `infra.network.Network`: manages a multi-node network; use the existing `infra.network.network(...)` context-manager pattern for lifecycle cleanup.\n- `infra.node.Node`: represents one node process.\n- `infra.consortium.Consortium`: member governance operations.\n- `infra.runner.ConcurrentRunner`: schedules `run_*(args)` functions, each owning its network.\n\n## Writing e2e tests\n\nFollow a nearby test for the same application. In `tests\u002Fe2e_logging.py`, individual test cases accept `(network, args)`, while `run(args)` creates\u002Fopens the network and calls those cases. Requirement decorators from `suite.test_requirements` express the case's prerequisites.\n\n- Register the network-owning `run_*(args)` function with `ConcurrentRunner.add`, not a `(network, args)` case. The runner invokes its target with one argument.\n- `ConcurrentRunner.add(prefix, target, **args_overrides)` already deep-copies arguments and assigns a distinct label. Prefer its overrides for separate configurations rather than duplicating that work. Deep-copy arguments and set a distinct label yourself only when manually creating an additional independent configuration outside that mechanism.\n- Reuse network\u002Fclient\u002Fgovernance helpers; assert observable behaviour and clean up through existing context managers.\n- Ensure the case is called by a runner. For a new e2e executable entry point, follow existing CMake `add_e2e_test` registration, including its CI bucket. Run `scripts\u002Ftest-buckets-checks.sh` from the repository root when changing inventory, and update `tests\u002Fci-buckets.txt` only for intentional registration changes.\n",{"name":5,"description":7,"user-invocable":43},"main","00d0976e71afc0613328225e6f4074a78f52a592",".github\u002Fskills\u002Ftesting",265,"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FCCF","http:\u002F\u002Fccf.dev\u002F","https:\u002F\u002Fgithub.com\u002Fmicrosoft","https:\u002F\u002Fopensource.microsoft.com","Open source projects and samples from Microsoft","2026-11-14 21:02:39",{"source":8,"skillName":5,"commands":58,"prompt":68,"zipAvailable":69},[59,62,65],{"tool":60,"command":61},"npx","npx skills add microsoft\u002FCCF --skill testing",{"tool":63,"command":64},"bunx","bunx skills add microsoft\u002FCCF --skill testing",{"tool":66,"command":67},"pnpm","pnpm dlx skills add microsoft\u002FCCF --skill testing","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FCCF\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Ftesting\u002FSKILL.md",true,[],[72],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":42},"01M6M7BT2JSJC9KDNZNST62164","formatting-and-linting","Format and lint CCF changes. Use when choosing or running checks for C++, Python, TypeScript, JavaScript, Markdown, YAML, JSON, CMake, shell scripts, or release notes, and when deciding which checks support automatic fixes.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FCCF\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fformatting-and-linting\u002FSKILL.md",[],[79,80,81,82,83,84,85,86,87,88,89,27,90,91,92,41,93,94,95,96],"formatting and linting","formatting","linting","when","running","markdown","shell scripts","changes","notes","javascript","lint","use","release notes","cmake","for","fixes","c++","which"]