[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M699BDD5RBFRHD2AKZ0QYQ72":3,"skill-same:01M699BDD5RBFRHD2AKZ0QYQ72":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":46,"repositoryHtmlUrl":47,"repositoryHomepageUrl":48,"ownerHtmlUrl":49,"ownerBlogUrl":50,"ownerDescription":51,"updatedAt":52,"install":53,"resources":65},"01M699BDD5RBFRHD2AKZ0QYQ72","smoke-tests",null,"Use when running VS Code smoke tests or working on smoke-test CI steps. Covers pnpm run smoketest \u002F smoketest-no-compile, grep filtering tests, and a temporary repeat-loop technique for tracking down flaky smoke tests in CI.","microsoft\u002Fvscode","microsoft","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6154722?v=4","01M698VYDXZ6JW5WH3CRV650R8","Microsoft","01M6998NK2WED5M549DX06VBCH",195270,"MIT","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fsmoke-tests\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"smoke","tests","smoke tests","loop","running","when","for","down","pnpm","tracking","grep","run","code","use","filtering","ci","compile","flaky smoke tests","test","2026-11-10 10:12:30",false,"---\nname: smoke-tests\ndescription: Use when running VS Code smoke tests or working on smoke-test CI steps. Covers pnpm run smoketest \u002F smoketest-no-compile, grep filtering tests, and a temporary repeat-loop technique for tracking down flaky smoke tests in CI.\n---\n\n# Running Smoke Tests\n\nSmoke tests live in `test\u002Fsmoke\u002F` and drive a full VS Code instance (Electron, web, or remote) through end-to-end user flows.\n\n## Scripts\n\n- `pnpm run smoketest` — compiles the smoke tests first (`test\u002Fsmoke`), then runs them.\n- `pnpm run smoketest-no-compile` — runs the already-compiled smoke tests. CI uses this after an explicit compile step.\n\nBoth forward extra arguments after `--` to the runner (`test\u002Fsmoke\u002Ftest\u002Findex.js`).\n\n## Common options\n\n| Option | Description |\n|--------|-------------|\n| `-g \u003Cpattern>` (alias `-f`) | Grep filter on test\u002Fsuite titles (mocha `grep`). |\n| `--build \u003Cpath>` | Run against a packaged build instead of the compiled-from-source dev build. |\n| `--tracing` | Capture Playwright traces (and screenshots on failure). |\n| `--web` | Run the browser smoke tests instead of Electron. |\n| `--headless` | Headless browser (used with `--web`). |\n| `--remote` | Run the remote smoke tests. |\n| `--shard \u003Cindex>\u002F\u003Ctotal>` | Run only one part of the top-level suites, for example `--shard 1\u002F2`. Suites are balanced across shards by estimated duration (see `test\u002Fsmoke\u002Fsrc\u002Fshard.ts`); each suite runs in exactly one shard. |\n\n```bash\n# Run everything (Electron, from source)\npnpm run smoketest\n\n# Run only a subset of suites by name, with tracing (replace \u003Csuite name> with your suite, e.g. \"Agents Window\")\npnpm run smoketest -g \"\u003Csuite name>\" --tracing\n\n# Run against a packaged build (CI style)\npnpm run smoketest-no-compile --tracing --build \"\u002Fpath\u002Fto\u002FVSCode-darwin-arm64\u002FCode - OSS.app\"\n```\n\nThe `-g` pattern matches against test\u002Fsuite titles. For example, `-g \"Agents Window\"` matches all three Agents Window suites (`Agents Window`, `Agents Window (local AgentHost)`, and `Agents Window (local AgentHost, SDK sandbox)`); use whatever substring identifies the suite(s) you care about.\n\nThe runner exits non-zero if any test fails, so a `0` exit code means every selected test passed.\n\n### Sharding in PR CI\n\nThe GitHub PR workflows split the Electron smoke tests into shards: the `electron-smoke-1` through `electron-smoke-3` jobs of `.github\u002Fworkflows\u002Fpr-linux-test.yml` and `pr-win32-test.yml` pass their `SMOKE_SHARD_INDEX`\u002F`SMOKE_SHARD_TOTAL` env to the runner as `--shard`. They report as `\u003COS> \u002F Electron-Smoke (shard 1\u002F3)` through `\u003COS> \u002F Electron-Smoke (shard 3\u002F3)`. On Windows, the WSL Dev Container suite runs in its own `electron-smoke-wsl` job instead of a shard, which reports as `Windows \u002F Electron-Smoke (WSL)`. The `electron-smoke-1` and `electron-smoke-2` jobs of `pr-darwin-test.yml` split them into two shards on macOS, but `pr.yml` does not run the macOS jobs for now (see its commented-out `macos-tests` job). On Linux, the remote smoke tests also run in their own `Linux \u002F Remote (smoke)` job, and on Windows in two shards, `Windows \u002F Remote (smoke, shard 1\u002F2)` and `Windows \u002F Remote (smoke, shard 2\u002F2)`, next to `\u003COS> \u002F Remote (integration)`. The `PR Gate` job (`pr-gate`) reports a single check for every job of the workflow and fails if any shard did not succeed. The shards are skipped, and the gate passes, when the pull request only changes paths the smoke tests cannot read (see `.github\u002Factions\u002Fdetect-changes\u002Fdetect.ts`). To change the shard count of a platform, add or remove `electron-smoke-\u003Cindex>` jobs in its workflow, and update the shard numbers in the names and env of all of them. Near the top of each shard's `smoke-test-runner.log`, a `Running smoke test shard \u003Cindex>\u002F\u003Ctotal>` line lists the suites that shard ran.\n\nSome placement is coupled to the workflows:\n\n- **WSL:** The `electron-smoke-wsl` job of `pr-win32-test.yml` (env `WSL_SMOKE_TESTS`) provisions WSL, runs only the WSL Dev Container suite (`-g`), and then the WSL Kerberos proxy run. The provisioning runs in the background while the job installs and builds, and a step before the smoke tests waits for it. The shards don't set the `VSCODE_SMOKE_TEST_WSL_*` variables, so every shard skips the WSL suite and computes the same split. The WSL job runs on the 1ES pool instead of the `vscode-large-runners-windows-8-core` larger runners, so it sets up without a workspace image: GitHub-hosted Windows runners intermittently freeze while WSL2 provisions a distribution ([actions\u002Frunner-images#14938](https:\u002F\u002Fgithub.com\u002Factions\u002Frunner-images\u002Fissues\u002F14938)), which leaves the check pending for about 45 minutes until GitHub reports that it lost communication with the runner.\n- **WSL version:** the agents' Windows Server 2022 image only has the inbox WSL, so provisioning installs the WSL release pinned in `build\u002Fchecksums\u002Fwsl.txt` (see `build\u002Fazure-pipelines\u002Fcommon\u002Fwsl.psm1`). To update it, put the SHA256 and file name of a newer `wsl.\u003Cversion>.x64.msi` from the [microsoft\u002FWSL releases](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FWSL\u002Freleases) there. The installer caches key on that file; `pr-pnpm-store.yml` fills the GitHub cache on `main`.\n- **Prepared WSL distribution:** the `wsl-distro` job of `pr-workspace-images.yml` (1ES pool, daily schedule and manual runs only) runs `wslDevContainer.ps1 -Operation Prepare` and saves the exported VHD in a cache keyed on the hash of `wslDevContainer.ps1` and `build\u002Fchecksums\u002Fwsl.txt`. The WSL job restores the newest entry for its hash and imports it with `wsl --import-in-place` instead of building the distribution. It builds the distribution when there is no entry for its hash (for example after a change to either file, until the next daily run or a manual run of the workflow on `main`), or when the import fails. On Azure Pipelines, the Windows x64 smoke job prepares the distribution itself: on a miss of its daily cache key, it runs Prepare before Setup and saves the VHD when the job succeeds (see `test\u002Fsmoke\u002FREADME.md`).\n- **macOS proxy runs:** In `pr-darwin-test.yml`, the PAC, Kerberos and Basic proxy Agents Window runs happen only in the last shard, so `test\u002Fsmoke\u002Fsrc\u002Fshard.ts` reserves their time there.\n\nEvery shard must register the same suites; otherwise suites can be skipped or run twice. Don't make suite registration depend on anything that differs between shards.\n\nWhen a run finishes, the runner prints each top-level suite's duration, longest first. When you add a smoke suite that takes much longer than launching the app (about 20 seconds), or the printed durations drift from `suiteDurations` in `test\u002Fsmoke\u002Fsrc\u002Fshard.ts`, update that table so the shards stay balanced. Use the durations of the PR workflows, and a condition on `process.platform` for a suite whose duration differs between platforms, such as `Terminal`, which takes seconds on Linux and over two minutes on macOS and Windows.\n\n## Temporarily looping a suite to hunt flaky CI tests\n\nWhen a smoke test fails intermittently only in CI, a useful technique is to **temporarily** run the suspect suite many times in a row and fail on the first failure. This reproduces the flake under the real CI environment and captures its traces\u002Fscreenshots, instead of waiting for it to recur naturally across unrelated PRs.\n\nThis is a debugging aid, **not a permanent CI fixture**:\n\n- Add it on a throwaway branch, push, and let CI run it. Iterate until you reproduce (and then fix) the flake.\n- **Remove the loop before merging** — leaving it in would add ~an hour per platform to every run.\n- It is **not specific to any one suite**. Point the `-g` filter at whichever suite you are investigating (the examples below use `\"Agents Window\"`, but substitute your own).\n\n### Where to add it\n\nDrop the loop next to the existing Electron smoke step, gated on the same condition, in the test step(s) for the platform(s) where the flake reproduces. In the sharded GitHub PR workflows, also gate it on a single shard (as in the example below) so it does not run in every shard:\n\n**GitHub PR workflows** (run from source, no `--build`):\n- `.github\u002Fworkflows\u002Fpr-linux-test.yml` (bash; sets `DISPLAY: \":10\"`)\n- `.github\u002Fworkflows\u002Fpr-win32-test.yml` (PowerShell)\n\n**Azure DevOps test steps** (run against the packaged build via `--build`):\n- `build\u002Fazure-pipelines\u002Flinux\u002Fsteps\u002Fproduct-build-linux-test.yml`\n- `build\u002Fazure-pipelines\u002Fdarwin\u002Fsteps\u002Fproduct-build-darwin-test.yml`\n- `build\u002Fazure-pipelines\u002Fwin32\u002Fsteps\u002Fproduct-build-win32-test.yml`\n\n### Shape\n\nLoop N iterations (e.g. 20) and abort on the first failing run. Give it a generous timeout — N sequential runs of a ~3-minute suite can take roughly an hour.\n\nBash (Linux\u002FmacOS):\n\n```yaml\n# TEMPORARY: loop the suite to reproduce a flaky failure. Remove before merge.\n# Replace \u003Csuite name> with the suite you're investigating (e.g. \"Agents Window\").\n- name: 🧪 Smoke test flakiness probe (TEMPORARY)\n  if: ${{ env.ELECTRON_TESTS == 'true' && env.SMOKE_TESTS == 'true' && env.SMOKE_SHARD_INDEX == '1' }}\n  timeout-minutes: 60\n  run: |\n    for i in $(seq 1 20); do\n      echo \"::group::Smoke probe run $i\u002F20\"\n      pnpm run smoketest-no-compile --tracing -g \"\u003Csuite name>\" || { echo \"::error::Smoke test failed on run $i\u002F20\"; exit 1; }\n      echo \"::endgroup::\"\n    done\n```\n\nPowerShell (Windows) checks `$LASTEXITCODE` after each run and `exit 1` on failure. The AzDO variants use `set -e` (bash) \u002F `$LASTEXITCODE` (pwsh) for fail-fast and append `--build \"\u003Cpackaged app path>\"`.\n\n### Why fail-fast\n\nThe loop is a probe: the first failure is the signal. Stopping immediately preserves the failing run's traces\u002Fscreenshots (under the logs artifact) and avoids burning ~an hour of agent time finishing a run that has already proven flaky.\n\n## Debugging CI smoke failures\n\nBoth CI systems publish the smoke runner's per-platform logs (the `.build\u002Flogs` directory) as a downloadable artifact. The artifact's internal layout is identical on both — only the artifact name and the download tool differ.\n\n### Downloading the logs artifact\n\n#### GitHub Actions\n\nThe GitHub PR workflows upload the artifact as `logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt>`, where `\u003Cos>` is `linux` \u002F `windows`, `\u003Csuite>` identifies the job (for example `electron`, `browser`, `remote-smoke`, or `electron-smoke-shard1` for the first Electron smoke shard), and `\u003Cattempt>` is the run attempt (e.g. `logs-linux-x64-electron-smoke-shard1-1`).\n\nThe run id is the number in the run\u002Fjob URL — for `…\u002Factions\u002Fruns\u002F\u003Crun-id>\u002Fjob\u002F\u003Cjob-id>` use `\u003Crun-id>`. Download with the `gh` CLI:\n\n```bash\n# A specific artifact into .\u002Flogs\ngh run download \u003Crun-id> -n logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt> -D .\u002Flogs\n\n# Or every artifact from the run\ngh run download \u003Crun-id>\n```\n\n`gh run view \u003Crun-id>` lists the run's jobs\u002Fartifacts; the run summary page in the browser also has an **Artifacts** section at the bottom.\n\n#### Azure DevOps\n\nThe artifact name depends on which pipeline produced it:\n\n- **Product build** (`product-build-\u003Cos>.yml`): `logs-\u003Cos>-\u003Carch>-\u003Cattempt>` — no suite segment, e.g. `logs-macos-arm64-1`.\n- **Suite-split CI build** (`product-build-\u003Cos>-ci.yml`): `logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt>` — the `\u003Csuite>` segment is `lower(VSCODE_TEST_SUITE)` (e.g. `electron`), so e.g. `logs-macos-arm64-electron-1` (same shape as GitHub). Only published when the job fails (or, on Windows, is cancelled).\n\n`\u003Cos>` is `linux` \u002F `macos` \u002F `windows`, `\u003Carch>` is `x64` \u002F `arm64`, and `\u003Cattempt>` is `$(System.JobAttempt)`. Download with the Azure CLI:\n\n```bash\naz pipelines runs artifact download \\\n  --org \u003CORG_URL> --project \u003CPROJECT_NAME> \\\n  --run-id \u003CBUILD_ID> --artifact-name \u003Cartifact-name> \\\n  --path .\u002Flogs\n```\n\nFor the VS Code build that is `--org https:\u002F\u002Fdev.azure.com\u002Fmonacotools --project Monaco`; see the `azure-pipelines` skill for finding the `\u003CBUILD_ID>`.\n\n### Inside the artifact\n\nUnder `smoke-tests-\u003Csuite>\u002F` (`smoke-tests-electron\u002F`, `smoke-tests-browser\u002F`, or `smoke-tests-remote\u002F`, matching the suite that ran):\n\n- `smoke-test-runner.log` — the mocha driver output plus, for suites that use the mock LLM server, its verbose request\u002Fresponse bodies (look for `request body:`). On a Copilot CLI \u002F Copilot session failure it also carries a tail of the captured Copilot runtime logs (see below).\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fcopilot-runtime-logs\u002Fprocess-*.log` — the Copilot runtime (`copilot-runtime` from `@github\u002Fcopilot-sdk-\u003Cplatform>`) process logs, captured by `dumpFailureDiagnostics` when a Copilot-runtime session fails. **Check these first for a hang or \"Timed out waiting for response\"**: they are the SDK\u002FCLI's own account of what it did (startup, auth, model request, turn lifecycle, and any panic \u002F out-of-order event \u002F protocol error) and explain a timeout that the test error alone does not. **Agent Host** sessions (Agents Window \u002F local AgentHost) write a full log run at `trace` (`chat.agentHost.copilotSdk.logLevel`). Chat Sessions editor (Copilot CLI \u002F Claude) and Local sessions run the SDK in-process and write only a minimal startup log here (`Server started, waiting for requests`) — enough to tell whether the runtime came up; their detailed model\u002Fturn diagnostics are in the `GitHub Copilot Chat.log` below. (Claude \u002F Codex sessions use a different runtime and are not captured here.)\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fwindow2\u002Fexthost\u002F\u003Cextension>\u002F…log` — per-suite extension-host logs (e.g. `GitHub.copilot-chat\u002FGitHub Copilot Chat.log`). Many diagnostics are gated behind a setting the suite enables in its `before` hook, so check the suite's setup if an expected log line is missing.\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fplaywright-screenshot-*.png` — last-frame screenshot captured when a test fails (only when the suite ran with `--tracing`).\n\n`\u003CSuite_Name>` is the mocha suite title with non-word characters replaced by `_`. See also the `code-oss-logs` skill.\n\n\n\n## Distinction from other test types\n\n- **Unit tests** (`.test.ts`) → `scripts\u002Ftest.sh` \u002F `runTests` tool (see the `unit-tests` skill).\n- **Integration tests** (`.integrationTest.ts` + extension tests) → `scripts\u002Ftest-integration.sh` (see the `integration-tests` skill).\n- **Smoke tests** (`test\u002Fsmoke\u002F`) → `pnpm run smoketest` — full end-to-end UI flows.\n","# Running Smoke Tests\n\nSmoke tests live in `test\u002Fsmoke\u002F` and drive a full VS Code instance (Electron, web, or remote) through end-to-end user flows.\n\n## Scripts\n\n- `pnpm run smoketest` — compiles the smoke tests first (`test\u002Fsmoke`), then runs them.\n- `pnpm run smoketest-no-compile` — runs the already-compiled smoke tests. CI uses this after an explicit compile step.\n\nBoth forward extra arguments after `--` to the runner (`test\u002Fsmoke\u002Ftest\u002Findex.js`).\n\n## Common options\n\n| Option | Description |\n|--------|-------------|\n| `-g \u003Cpattern>` (alias `-f`) | Grep filter on test\u002Fsuite titles (mocha `grep`). |\n| `--build \u003Cpath>` | Run against a packaged build instead of the compiled-from-source dev build. |\n| `--tracing` | Capture Playwright traces (and screenshots on failure). |\n| `--web` | Run the browser smoke tests instead of Electron. |\n| `--headless` | Headless browser (used with `--web`). |\n| `--remote` | Run the remote smoke tests. |\n| `--shard \u003Cindex>\u002F\u003Ctotal>` | Run only one part of the top-level suites, for example `--shard 1\u002F2`. Suites are balanced across shards by estimated duration (see `test\u002Fsmoke\u002Fsrc\u002Fshard.ts`); each suite runs in exactly one shard. |\n\n```bash\n# Run everything (Electron, from source)\npnpm run smoketest\n\n# Run only a subset of suites by name, with tracing (replace \u003Csuite name> with your suite, e.g. \"Agents Window\")\npnpm run smoketest -g \"\u003Csuite name>\" --tracing\n\n# Run against a packaged build (CI style)\npnpm run smoketest-no-compile --tracing --build \"\u002Fpath\u002Fto\u002FVSCode-darwin-arm64\u002FCode - OSS.app\"\n```\n\nThe `-g` pattern matches against test\u002Fsuite titles. For example, `-g \"Agents Window\"` matches all three Agents Window suites (`Agents Window`, `Agents Window (local AgentHost)`, and `Agents Window (local AgentHost, SDK sandbox)`); use whatever substring identifies the suite(s) you care about.\n\nThe runner exits non-zero if any test fails, so a `0` exit code means every selected test passed.\n\n### Sharding in PR CI\n\nThe GitHub PR workflows split the Electron smoke tests into shards: the `electron-smoke-1` through `electron-smoke-3` jobs of `.github\u002Fworkflows\u002Fpr-linux-test.yml` and `pr-win32-test.yml` pass their `SMOKE_SHARD_INDEX`\u002F`SMOKE_SHARD_TOTAL` env to the runner as `--shard`. They report as `\u003COS> \u002F Electron-Smoke (shard 1\u002F3)` through `\u003COS> \u002F Electron-Smoke (shard 3\u002F3)`. On Windows, the WSL Dev Container suite runs in its own `electron-smoke-wsl` job instead of a shard, which reports as `Windows \u002F Electron-Smoke (WSL)`. The `electron-smoke-1` and `electron-smoke-2` jobs of `pr-darwin-test.yml` split them into two shards on macOS, but `pr.yml` does not run the macOS jobs for now (see its commented-out `macos-tests` job). On Linux, the remote smoke tests also run in their own `Linux \u002F Remote (smoke)` job, and on Windows in two shards, `Windows \u002F Remote (smoke, shard 1\u002F2)` and `Windows \u002F Remote (smoke, shard 2\u002F2)`, next to `\u003COS> \u002F Remote (integration)`. The `PR Gate` job (`pr-gate`) reports a single check for every job of the workflow and fails if any shard did not succeed. The shards are skipped, and the gate passes, when the pull request only changes paths the smoke tests cannot read (see `.github\u002Factions\u002Fdetect-changes\u002Fdetect.ts`). To change the shard count of a platform, add or remove `electron-smoke-\u003Cindex>` jobs in its workflow, and update the shard numbers in the names and env of all of them. Near the top of each shard's `smoke-test-runner.log`, a `Running smoke test shard \u003Cindex>\u002F\u003Ctotal>` line lists the suites that shard ran.\n\nSome placement is coupled to the workflows:\n\n- **WSL:** The `electron-smoke-wsl` job of `pr-win32-test.yml` (env `WSL_SMOKE_TESTS`) provisions WSL, runs only the WSL Dev Container suite (`-g`), and then the WSL Kerberos proxy run. The provisioning runs in the background while the job installs and builds, and a step before the smoke tests waits for it. The shards don't set the `VSCODE_SMOKE_TEST_WSL_*` variables, so every shard skips the WSL suite and computes the same split. The WSL job runs on the 1ES pool instead of the `vscode-large-runners-windows-8-core` larger runners, so it sets up without a workspace image: GitHub-hosted Windows runners intermittently freeze while WSL2 provisions a distribution ([actions\u002Frunner-images#14938](https:\u002F\u002Fgithub.com\u002Factions\u002Frunner-images\u002Fissues\u002F14938)), which leaves the check pending for about 45 minutes until GitHub reports that it lost communication with the runner.\n- **WSL version:** the agents' Windows Server 2022 image only has the inbox WSL, so provisioning installs the WSL release pinned in `build\u002Fchecksums\u002Fwsl.txt` (see `build\u002Fazure-pipelines\u002Fcommon\u002Fwsl.psm1`). To update it, put the SHA256 and file name of a newer `wsl.\u003Cversion>.x64.msi` from the [microsoft\u002FWSL releases](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FWSL\u002Freleases) there. The installer caches key on that file; `pr-pnpm-store.yml` fills the GitHub cache on `main`.\n- **Prepared WSL distribution:** the `wsl-distro` job of `pr-workspace-images.yml` (1ES pool, daily schedule and manual runs only) runs `wslDevContainer.ps1 -Operation Prepare` and saves the exported VHD in a cache keyed on the hash of `wslDevContainer.ps1` and `build\u002Fchecksums\u002Fwsl.txt`. The WSL job restores the newest entry for its hash and imports it with `wsl --import-in-place` instead of building the distribution. It builds the distribution when there is no entry for its hash (for example after a change to either file, until the next daily run or a manual run of the workflow on `main`), or when the import fails. On Azure Pipelines, the Windows x64 smoke job prepares the distribution itself: on a miss of its daily cache key, it runs Prepare before Setup and saves the VHD when the job succeeds (see `test\u002Fsmoke\u002FREADME.md`).\n- **macOS proxy runs:** In `pr-darwin-test.yml`, the PAC, Kerberos and Basic proxy Agents Window runs happen only in the last shard, so `test\u002Fsmoke\u002Fsrc\u002Fshard.ts` reserves their time there.\n\nEvery shard must register the same suites; otherwise suites can be skipped or run twice. Don't make suite registration depend on anything that differs between shards.\n\nWhen a run finishes, the runner prints each top-level suite's duration, longest first. When you add a smoke suite that takes much longer than launching the app (about 20 seconds), or the printed durations drift from `suiteDurations` in `test\u002Fsmoke\u002Fsrc\u002Fshard.ts`, update that table so the shards stay balanced. Use the durations of the PR workflows, and a condition on `process.platform` for a suite whose duration differs between platforms, such as `Terminal`, which takes seconds on Linux and over two minutes on macOS and Windows.\n\n## Temporarily looping a suite to hunt flaky CI tests\n\nWhen a smoke test fails intermittently only in CI, a useful technique is to **temporarily** run the suspect suite many times in a row and fail on the first failure. This reproduces the flake under the real CI environment and captures its traces\u002Fscreenshots, instead of waiting for it to recur naturally across unrelated PRs.\n\nThis is a debugging aid, **not a permanent CI fixture**:\n\n- Add it on a throwaway branch, push, and let CI run it. Iterate until you reproduce (and then fix) the flake.\n- **Remove the loop before merging** — leaving it in would add ~an hour per platform to every run.\n- It is **not specific to any one suite**. Point the `-g` filter at whichever suite you are investigating (the examples below use `\"Agents Window\"`, but substitute your own).\n\n### Where to add it\n\nDrop the loop next to the existing Electron smoke step, gated on the same condition, in the test step(s) for the platform(s) where the flake reproduces. In the sharded GitHub PR workflows, also gate it on a single shard (as in the example below) so it does not run in every shard:\n\n**GitHub PR workflows** (run from source, no `--build`):\n- `.github\u002Fworkflows\u002Fpr-linux-test.yml` (bash; sets `DISPLAY: \":10\"`)\n- `.github\u002Fworkflows\u002Fpr-win32-test.yml` (PowerShell)\n\n**Azure DevOps test steps** (run against the packaged build via `--build`):\n- `build\u002Fazure-pipelines\u002Flinux\u002Fsteps\u002Fproduct-build-linux-test.yml`\n- `build\u002Fazure-pipelines\u002Fdarwin\u002Fsteps\u002Fproduct-build-darwin-test.yml`\n- `build\u002Fazure-pipelines\u002Fwin32\u002Fsteps\u002Fproduct-build-win32-test.yml`\n\n### Shape\n\nLoop N iterations (e.g. 20) and abort on the first failing run. Give it a generous timeout — N sequential runs of a ~3-minute suite can take roughly an hour.\n\nBash (Linux\u002FmacOS):\n\n```yaml\n# TEMPORARY: loop the suite to reproduce a flaky failure. Remove before merge.\n# Replace \u003Csuite name> with the suite you're investigating (e.g. \"Agents Window\").\n- name: 🧪 Smoke test flakiness probe (TEMPORARY)\n  if: ${{ env.ELECTRON_TESTS == 'true' && env.SMOKE_TESTS == 'true' && env.SMOKE_SHARD_INDEX == '1' }}\n  timeout-minutes: 60\n  run: |\n    for i in $(seq 1 20); do\n      echo \"::group::Smoke probe run $i\u002F20\"\n      pnpm run smoketest-no-compile --tracing -g \"\u003Csuite name>\" || { echo \"::error::Smoke test failed on run $i\u002F20\"; exit 1; }\n      echo \"::endgroup::\"\n    done\n```\n\nPowerShell (Windows) checks `$LASTEXITCODE` after each run and `exit 1` on failure. The AzDO variants use `set -e` (bash) \u002F `$LASTEXITCODE` (pwsh) for fail-fast and append `--build \"\u003Cpackaged app path>\"`.\n\n### Why fail-fast\n\nThe loop is a probe: the first failure is the signal. Stopping immediately preserves the failing run's traces\u002Fscreenshots (under the logs artifact) and avoids burning ~an hour of agent time finishing a run that has already proven flaky.\n\n## Debugging CI smoke failures\n\nBoth CI systems publish the smoke runner's per-platform logs (the `.build\u002Flogs` directory) as a downloadable artifact. The artifact's internal layout is identical on both — only the artifact name and the download tool differ.\n\n### Downloading the logs artifact\n\n#### GitHub Actions\n\nThe GitHub PR workflows upload the artifact as `logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt>`, where `\u003Cos>` is `linux` \u002F `windows`, `\u003Csuite>` identifies the job (for example `electron`, `browser`, `remote-smoke`, or `electron-smoke-shard1` for the first Electron smoke shard), and `\u003Cattempt>` is the run attempt (e.g. `logs-linux-x64-electron-smoke-shard1-1`).\n\nThe run id is the number in the run\u002Fjob URL — for `…\u002Factions\u002Fruns\u002F\u003Crun-id>\u002Fjob\u002F\u003Cjob-id>` use `\u003Crun-id>`. Download with the `gh` CLI:\n\n```bash\n# A specific artifact into .\u002Flogs\ngh run download \u003Crun-id> -n logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt> -D .\u002Flogs\n\n# Or every artifact from the run\ngh run download \u003Crun-id>\n```\n\n`gh run view \u003Crun-id>` lists the run's jobs\u002Fartifacts; the run summary page in the browser also has an **Artifacts** section at the bottom.\n\n#### Azure DevOps\n\nThe artifact name depends on which pipeline produced it:\n\n- **Product build** (`product-build-\u003Cos>.yml`): `logs-\u003Cos>-\u003Carch>-\u003Cattempt>` — no suite segment, e.g. `logs-macos-arm64-1`.\n- **Suite-split CI build** (`product-build-\u003Cos>-ci.yml`): `logs-\u003Cos>-\u003Carch>-\u003Csuite>-\u003Cattempt>` — the `\u003Csuite>` segment is `lower(VSCODE_TEST_SUITE)` (e.g. `electron`), so e.g. `logs-macos-arm64-electron-1` (same shape as GitHub). Only published when the job fails (or, on Windows, is cancelled).\n\n`\u003Cos>` is `linux` \u002F `macos` \u002F `windows`, `\u003Carch>` is `x64` \u002F `arm64`, and `\u003Cattempt>` is `$(System.JobAttempt)`. Download with the Azure CLI:\n\n```bash\naz pipelines runs artifact download \\\n  --org \u003CORG_URL> --project \u003CPROJECT_NAME> \\\n  --run-id \u003CBUILD_ID> --artifact-name \u003Cartifact-name> \\\n  --path .\u002Flogs\n```\n\nFor the VS Code build that is `--org https:\u002F\u002Fdev.azure.com\u002Fmonacotools --project Monaco`; see the `azure-pipelines` skill for finding the `\u003CBUILD_ID>`.\n\n### Inside the artifact\n\nUnder `smoke-tests-\u003Csuite>\u002F` (`smoke-tests-electron\u002F`, `smoke-tests-browser\u002F`, or `smoke-tests-remote\u002F`, matching the suite that ran):\n\n- `smoke-test-runner.log` — the mocha driver output plus, for suites that use the mock LLM server, its verbose request\u002Fresponse bodies (look for `request body:`). On a Copilot CLI \u002F Copilot session failure it also carries a tail of the captured Copilot runtime logs (see below).\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fcopilot-runtime-logs\u002Fprocess-*.log` — the Copilot runtime (`copilot-runtime` from `@github\u002Fcopilot-sdk-\u003Cplatform>`) process logs, captured by `dumpFailureDiagnostics` when a Copilot-runtime session fails. **Check these first for a hang or \"Timed out waiting for response\"**: they are the SDK\u002FCLI's own account of what it did (startup, auth, model request, turn lifecycle, and any panic \u002F out-of-order event \u002F protocol error) and explain a timeout that the test error alone does not. **Agent Host** sessions (Agents Window \u002F local AgentHost) write a full log run at `trace` (`chat.agentHost.copilotSdk.logLevel`). Chat Sessions editor (Copilot CLI \u002F Claude) and Local sessions run the SDK in-process and write only a minimal startup log here (`Server started, waiting for requests`) — enough to tell whether the runtime came up; their detailed model\u002Fturn diagnostics are in the `GitHub Copilot Chat.log` below. (Claude \u002F Codex sessions use a different runtime and are not captured here.)\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fwindow2\u002Fexthost\u002F\u003Cextension>\u002F…log` — per-suite extension-host logs (e.g. `GitHub.copilot-chat\u002FGitHub Copilot Chat.log`). Many diagnostics are gated behind a setting the suite enables in its `before` hook, so check the suite's setup if an expected log line is missing.\n- `\u003CN>_suite_\u003CSuite_Name>\u002Fplaywright-screenshot-*.png` — last-frame screenshot captured when a test fails (only when the suite ran with `--tracing`).\n\n`\u003CSuite_Name>` is the mocha suite title with non-word characters replaced by `_`. See also the `code-oss-logs` skill.\n\n\n\n## Distinction from other test types\n\n- **Unit tests** (`.test.ts`) → `scripts\u002Ftest.sh` \u002F `runTests` tool (see the `unit-tests` skill).\n- **Integration tests** (`.integrationTest.ts` + extension tests) → `scripts\u002Ftest-integration.sh` (see the `integration-tests` skill).\n- **Smoke tests** (`test\u002Fsmoke\u002F`) → `pnpm run smoketest` — full end-to-end UI flows.\n",{"name":5,"description":7},"main","6fef772036848fdba226014ed798eb1654b32c39",".github\u002Fskills\u002Fsmoke-tests",43791,"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode","https:\u002F\u002Fcode.visualstudio.com","https:\u002F\u002Fgithub.com\u002Fmicrosoft","https:\u002F\u002Fopensource.microsoft.com","Open source projects and samples from Microsoft","2026-11-12 03:13:10",{"source":8,"skillName":5,"commands":54,"prompt":63,"zipAvailable":64},[55,58,61],{"tool":56,"command":57},"npx","npx skills add microsoft\u002Fvscode --skill smoke-tests",{"tool":59,"command":60},"bunx","bunx skills add microsoft\u002Fvscode --skill smoke-tests",{"tool":27,"command":62},"pnpm dlx skills add microsoft\u002Fvscode --skill smoke-tests","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fsmoke-tests\u002FSKILL.md",true,[],[67,92,113,127,142,158],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":38},"01M70H0ANBKZAVEP1NVV4K28M3","sweeper-query","Answer natural-language questions about the VS Code Sweeper's published issue index (index-\u003Cname>.json for microsoft\u002Fvscode and the enrolled family repos) — which issues are agent-ready to implement or plan, what the sweeper proposed for an assignee or area, whether the sweeper knows a report as a duplicate, the highest-value open bugs in a lane — and, for one issue, the review's full record (evidence with file citations, fix brief, proposed comment). Use ONLY when the request explicitly names the sweeper — \"sweeper\", \"vscodesweeper\", \"sweeper-query\", a sweeper verdict, record, brief or index, an agent-ready issue — or asks what \"the sweeper\" found, proposed or knows. Do NOT use for general questions about issues, duplicates, priorities or ranking that do not mention the sweeper. Read-only; discovery only.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fsweeper-query\u002FSKILL.md",[],[74,75,76,77,78,24,79,80,81,82,83,84,85,86,87,88,89,90,91,31],"sweeper query","query","sweeper","review","fix","request","which","natural-language","ranking","duplicates","ready","only","read","plan","open","Citations","brief","issues",{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":38},"01M699DRJS4YJCV600Q72SVVWE","update-skills","Create or update repository skills and instructions when major learnings are discovered during a session. Use when the user says \"learn!\", when a significant pattern or pitfall is identified, or when reusable domain knowledge should be captured for future sessions.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fupdate-skills\u002FSKILL.md",[],[99,100,25,101,102,103,104,32,105,106,107,108,109,110,111,112,24],"update","update skills","pattern","domain","learnings","reusable","session","user","instructions","knowledge","repository","learn","create","sessions",{"skillNo":114,"name":115,"nameZh":6,"description":116,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":117,"categories":118,"keywords":119,"publishedAt":38},"01M699DPWHZ72B66P82EW31RNE","update-pr","Update the pull request for the current session. Use when the user wants to push new changes to an existing PR.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fupdate-pr\u002FSKILL.md",[],[99,120,121,24,79,122,25,32,105,106,123,124,125,126],"pr","update pr","Existing","push","pull","pull request","changes",{"skillNo":128,"name":129,"nameZh":6,"description":130,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":131,"categories":132,"keywords":133,"publishedAt":38},"01M699DJMR6CE1MH5XCJZ0GFQ0","sync","Sync the current session branch with its upstream branch, or publish the current session branch to a remote. Use when the user asks to sync a branch, pull latest changes, rebase onto upstream, push current branch, publish branch, or set upstream.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fsync\u002FSKILL.md",[],[129,134,135,123,124,136,24,126,137,138,139,140,32,105,106,141],"SET","publish","rebase","latest","with","remote","upstream","branch",{"skillNo":143,"name":144,"nameZh":6,"description":145,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":146,"categories":147,"keywords":148,"publishedAt":38},"01M699DGXHX6A2A72X57R6GGHJ","sync-upstream","Update a stale session branch by rebasing onto the latest origin. Use when the upstream has moved significantly and the session needs to catch up, resolving conflicts by preserving upstream changes and adapting session work to fit.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fsync-upstream\u002FSKILL.md",[],[140,129,149,150,24,151,152,126,137,153,154,155,156,32,105,141,157,99],"sync upstream","fit","catch","catch up","origin","conflicts","stale","up","work",{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":164,"publishedAt":38},"01M699DEXYFWDGW0BAXJ5QQ377","merge","Merge changes from the topic branch to the merge base branch. Use when the user wants to merge their session's work back to the base branch.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fmerge\u002FSKILL.md",[],[160,105,32,157,141,106,24,165,166,126],"back","base"]