[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M699AP3N652AXKCVX2N33DXY":3,"skill-same:01M699AP3N652AXKCVX2N33DXY":70},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":41,"linkOnly":42,"markdownContent":43,"markdownBody":44,"frontMatter":45,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":51,"ownerHtmlUrl":52,"ownerBlogUrl":53,"ownerDescription":54,"updatedAt":55,"install":56,"resources":69},"01M699AP3N652AXKCVX2N33DXY","flaky-smoke-tests",null,"Diagnose intermittent VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline (definition 700). Covers finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs.","microsoft\u002Fvscode","microsoft","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6154722?v=4","01M698VYDXZ6JW5WH3CRV650R8","Microsoft","01M6998NK2WED5M549DX06VBCH",195271,"MIT","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fflaky-smoke-tests\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],"flaky smoke tests","smoke tests","tests","smoke","tracing","definition","with","platform","runner","task","azure cli","pipeline","logs","cli","artifacts","code","azure","diagnose","azure devops","commit","test","electron","2026-11-10 10:12:30",false,"---\nname: flaky-smoke-tests\ndescription: Diagnose intermittent VS Code Electron smoke-test failures from the Azure DevOps Flaky Smoke Tests pipeline (definition 700). Covers finding failed iterations, downloading task logs and platform artifacts with Azure CLI, correlating cumulative runner logs, tracing the introducing commit, and queueing focused validation runs.\n---\n\n# Diagnosing Flaky Smoke Tests\n\nUse this skill for failures from the Azure DevOps **Flaky Smoke Tests** pipeline:\n\n- Organization: `https:\u002F\u002Fdev.azure.com\u002Fmonacotools`\n- Project: `Monaco`\n- Definition ID: `700`\n- Pipeline YAML: `build\u002Fazure-pipelines\u002Fproduct-smoke-flaky.yml`\n- Scheduled runs: twice daily on `main`\n\nThe pipeline builds VS Code from source and runs the complete Electron smoke suite once per entry in the `iterations` parameter. Each platform runs its iterations sequentially in one job. The exception is Windows unit and integration tests, which are each split into two parallel jobs: `WindowsSmokeFlakyUnit_odd` \u002F `WindowsSmokeFlakyIntegration_odd` run the odd iterations and `WindowsSmokeFlakyUnit_even` \u002F `WindowsSmokeFlakyIntegration_even` run the even ones. Each shard has its own artifacts (`logs-windows-x64-\u003Cunit|integration>-\u003Codd|even>-\u003Cattempt>`) and its own Slack tally line.\n\n## Important Pipeline Behavior\n\n- Each iteration is a separate timeline task named `Smoke test iteration \u003CN>\u002F\u003Ctotal> (Electron)`.\n- Iteration tasks use `continueOnError: true`, so a failed iteration normally has result `succeededWithIssues` and later iterations still run.\n- Do not infer smoke-test health from the overall build or job result. Inspect every iteration task.\n- Logs are published once per platform job from the shared `.build\u002Flogs` directory. The artifact is job-scoped, not iteration-scoped: `smoke-test-runner.log` can contain multiple iterations, and suite directories are not separated by iteration.\n- Do not cancel a platform job while investigating if you still need its logs artifact. Cancellation can skip the publish steps entirely, even when one or more failed iteration task logs are already available. If a validation run has become predictably red, either let the current job finish or queue a single diagnostic iteration and let that job publish before iterating.\n- Later iterations can overwrite or truncate files in the shared logs directory. The final artifact may therefore contain the last iteration's suite directory or runner log but not the failed iteration's detailed diagnostics. Preserve the exact task log, and use a one-iteration diagnostic run when extension-host, renderer, mock-server, screenshot, or trace evidence from the failed attempt is required.\n- The exact iteration task log is authoritative for its failure summary and time range. Correlate that range with the job-level artifact.\n- Iterations can see persisted smoke-test state from earlier iterations, such as historical session rows. Never assume the newest-looking row or a matching prompt belongs to the active conversation without verifying the active view.\n\nPlatform definitions and artifacts:\n\n| Platform | Job | Logs artifact | Crash artifact |\n|---|---|---|---|\n| macOS arm64 | `macOSSmokeFlaky` | `logs-macos-arm64-smoke-\u003Cattempt>` | `crash-dump-macos-arm64-smoke-\u003Cattempt>` |\n| Windows x64 | `WindowsSmokeFlaky` | `logs-windows-x64-smoke-\u003Cattempt>` | `crash-dump-windows-x64-smoke-\u003Cattempt>` |\n| Linux x64 | `LinuxSmokeFlaky` | `logs-linux-x64-smoke-\u003Cattempt>` | `crash-dump-linux-x64-smoke-\u003Cattempt>` |\n\n## Prerequisites\n\nUse Azure CLI rather than the browser:\n\n```bash\naz --version\naz extension show --name azure-devops\naz devops configure --defaults \\\n  organization=https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  project=Monaco\n```\n\nIf authentication fails, run `az login`.\n\n## 1. Confirm the Build and Source Revision\n\n```bash\naz pipelines build show \\\n  --id \u003CBUILD_ID> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --query \"{id:id,status:status,result:result,sourceBranch:sourceBranch,sourceVersion:sourceVersion,templateParameters:templateParameters}\" \\\n  --output json\n```\n\nRecord `sourceVersion`. Investigate the exact code built by the pipeline, not the current working tree.\n\n## 2. Find Failed Iterations\n\nDownload the build timeline:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource timeline \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> \\\n  --output json\n```\n\nFilter task records whose names contain `Smoke test iteration`. Treat `succeededWithIssues`, `failed`, and tasks with error issues as failed iterations. Record:\n\n- task `id`\n- parent job `id`\n- task `name`\n- `startTime` and `finishTime`\n- `log.id`\n- `issues`\n\nIf the user supplied an Azure log URL, its `j=` value is the job ID and its `t=` value is the task ID. Query those records directly:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource timeline \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> \\\n  --query \"records[?id=='\u003CJOB_OR_TASK_ID>'].{id:id,parentId:parentId,name:name,type:type,state:state,result:result,logId:log.id,startTime:startTime,finishTime:finishTime,issues:issues}\" \\\n  --output json\n```\n\n## 3. Download the Exact Iteration Task Log\n\nUse timeline record `log.id`. The correct REST resource is `logs`, not `buildLog`:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource logs \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> logId=\u003CLOG_ID> \\\n  --out-file task-log.json\n```\n\nThe downloaded response is JSON with a `value` array containing one string per log line. Convert it to plain text.\n\nPowerShell:\n\n```powershell\n(Get-Content -Raw task-log.json | ConvertFrom-Json).value |\n  Set-Content task-log.txt\n```\n\nBash:\n\n```bash\njq -r '.value[]' task-log.json > task-log.txt\n```\n\nThe task log provides the concise Mocha failure, stack, and the iteration's exact time range even before platform artifacts are published. Diagnostics written through the smoke runner's `Logger` (including `dumpFailureDiagnostics`) may exist only in `smoke-test-runner.log`, so do not assume they will be present in the Azure task log.\n\n## 4. Download the Platform Logs Artifact\n\nList artifacts first; names include the job attempt:\n\n```bash\naz pipelines runs artifact list \\\n  --run-id \u003CBUILD_ID> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --output table\n```\n\nDownload the relevant platform:\n\n```bash\naz pipelines runs artifact download \\\n  --run-id \u003CBUILD_ID> \\\n  --artifact-name \u003CLOGS_ARTIFACT> \\\n  --path \u003CDESTINATION> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco\n```\n\nArtifacts are available after the platform job publishes its outputs. An absent platform artifact after a canceled run is expected; it is not evidence that the artifact name was wrong.\n\n## 5. Correlate the Failure\n\nStart with `\u003CDESTINATION>\u002Fsmoke-tests-electron\u002Fsmoke-test-runner.log`. Use the failed task's timestamps and test title to isolate the matching `Test start` \u002F `Test end` interval. Do not use the first occurrence of a test title because the runner log can contain several iterations.\n\nWithin that interval, establish this chain:\n\n1. **Gesture**: Was the expected editor clicked and prompt typed?\n2. **Dispatch**: Did the send action fire and did the new-session view close?\n3. **Request**: Did the mock server receive the expected scenario tag?\n4. **Tool loop**: For shell tests, did the second model request contain the expected tool result?\n5. **Rendering**: Did an assistant response render, and in which session view?\n6. **Routing**: Did the active session auto-swap to a new composer while the completed response remained in another session?\n\nFor mock-LLM suites, search for:\n\n- the scenario ID\n- `request body:`\n- the expected response marker\n- `model turn 1\u002F2` and `model turn 2\u002F2`\n\nThe request log distinguishes:\n\n- no dispatch\n- wrong provider or scenario\n- tool call never executed\n- tool result returned but UI rendering\u002Frouting lost it\n\nThen inspect the suite directory:\n\n```text\nsmoke-tests-electron\u002F\u003CN>_suite_\u003CSuite_Name>\u002F\n```\n\nUseful files include:\n\n- `window*\u002Fexthost\u002F\u003Cextension>\u002F\u003Cextension>.log`\n- `main.log`, `renderer.log`, and `agenthost.log`\n- `copilot-runtime-logs\u002Fprocess-*.log` — the Copilot runtime (`copilot-runtime` from `@github\u002Fcopilot-sdk-\u003Cplatform>`) process logs, captured by `dumpFailureDiagnostics` when a Copilot-runtime session fails. Check these first for a **hang or \"Timed out waiting for response\"**: they are the SDK\u002FCLI's own record (startup, auth, model request, turn lifecycle, panics, out-of-order or protocol errors) and explain a timeout the test error alone does not. A tail is also mirrored into `smoke-test-runner.log`. **Agent Host** sessions (Agents Window \u002F local AgentHost) write a full log run at `trace`; Chat Sessions editor (Copilot CLI \u002F Claude) and Local sessions write only a minimal startup log here (whether the runtime came up), with their detailed diagnostics in `GitHub Copilot Chat.log`. (Claude \u002F Codex sessions use a different runtime and are not captured here.)\n- `playwright-screenshot-*.png`\n- Playwright trace archives\n\nFor native exits or renderer crashes, also download the platform crash artifact.\n\n## 6. Interpret UI Diagnostics Carefully\n\nSession-list text and active-view text answer different questions:\n\n- A list row can contain a prompt from the current test but a command or title inherited from a prior request.\n- The active view can already be a fresh untitled composer while the response belongs to a completed, inactive session.\n- Broad response selectors can match stale DOM from an earlier session or iteration.\n- Reusing a fixed warm-up scenario marker can let a later warm-up wait match an earlier response and abandon the actually-running warm-up.\n- Standard smoke suites created through `installAllHandlers` use distinct randomized user-data directories. Do not attribute a setting from the preceding suite to the failing suite without comparing the actual `vscode-userdata:` or `--user-data-dir` paths in their logs. Iterations share broader job state, but sibling suites normally do not share the same profile.\n- A setup change can alter rather than fix the symptom. Re-check the exact stack and the gesture\u002Fdispatch\u002Frequest\u002Frendering chain on every validation run. For example, moving extension enablement before startup can eliminate an editor-open timeout while activating the extension before test settings are written, producing a later request or response timeout instead.\n\nPrefer assertions that establish identity:\n\n- unique scenario IDs or response markers per attempt\n- active chat\u002Fsession resource attributes\n- expected response in the active session before follow-up input\n- mock-server request count or captured request content after the gesture\n\nDo not \"fix\" these races by only increasing a timeout. Wait for the actual state transition or remove duplicate\u002Fin-flight work.\n\n## 7. Find the Introducing Commit\n\nUse the build's `sourceVersion` and identify the source file from the stack:\n\n```bash\ngit show \u003CSOURCE_VERSION>:test\u002Fsmoke\u002Fsrc\u002Fareas\u002F\u003Carea>\u002F\u003Ctest>.test.ts\ngit log --oneline \u003CKNOWN_GOOD>..\u003CSOURCE_VERSION> -- \u003Crelevant paths>\ngit blame -L \u003Cstart>,\u003Cend> \u003Cfile>\ngit show \u003CSUSPECT_COMMIT> -- \u003Crelevant paths>\n```\n\nTrace the whole causal sequence, not only the failing assertion. For example, inspect setup hooks, warm-ups, session selection, response waits, and teardown.\n\nDistinguish:\n\n- the commit that introduced the racy behavior\n- a later unrelated commit after which timing happened to expose it\n- a commit that only added diagnostics or made the flake visible\n\nState the introducing commit only when the diff contains the causal behavior and the pipeline evidence matches it.\n\n## 8. Queue a Focused Validation Run\n\nThe branch and commit must already be pushed to `microsoft\u002Fvscode`. Check for and cancel obsolete definition-700 runs on the same branch before queueing.\n\nUse `az pipelines run` directly because `iterations` is an object parameter:\n\n```bash\naz pipelines run \\\n  --id 700 \\\n  --branch \u003CBRANCH> \\\n  --commit-id \u003CCOMMIT_SHA> \\\n  --parameters \"iterations=[1,2,3,4,5,6]\" \\\n    VSCODE_BUILD_MACOS=true \\\n    VSCODE_BUILD_LINUX=false \\\n    VSCODE_BUILD_WIN32=true \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --output json\n```\n\nEnable only platforms relevant to the failure. Confirm the queued run's `sourceVersion` and `templateParameters` with `az pipelines build show`.\n\nSix iterations are a useful quick validation sample. Use the default 20 when the failure is rare or when validating before declaring a recurring flake resolved.\n\n## 9. Validate the Local Change\n\nFor smoke-test TypeScript changes:\n\n```bash\npnpm -C test\u002Fsmoke run compile\nnode --experimental-strip-types build\u002Fhygiene.ts \u003Cchanged-file>\n```\n\nIf local dependencies are missing or stale, report that explicitly and rely on the focused definition-700 run for full compiled validation; do not silently skip validation or install unrelated tooling.\n\n## Expected Report\n\nSummarize:\n\n- build ID and source revision\n- failing platform and iteration count\n- exact failing test and symptom\n- dispatch\u002Frequest\u002Frendering evidence\n- root cause\n- introducing commit, with why it is causal\n- fix commit\n- focused validation run ID, platforms, iterations, and current result\n","# Diagnosing Flaky Smoke Tests\n\nUse this skill for failures from the Azure DevOps **Flaky Smoke Tests** pipeline:\n\n- Organization: `https:\u002F\u002Fdev.azure.com\u002Fmonacotools`\n- Project: `Monaco`\n- Definition ID: `700`\n- Pipeline YAML: `build\u002Fazure-pipelines\u002Fproduct-smoke-flaky.yml`\n- Scheduled runs: twice daily on `main`\n\nThe pipeline builds VS Code from source and runs the complete Electron smoke suite once per entry in the `iterations` parameter. Each platform runs its iterations sequentially in one job. The exception is Windows unit and integration tests, which are each split into two parallel jobs: `WindowsSmokeFlakyUnit_odd` \u002F `WindowsSmokeFlakyIntegration_odd` run the odd iterations and `WindowsSmokeFlakyUnit_even` \u002F `WindowsSmokeFlakyIntegration_even` run the even ones. Each shard has its own artifacts (`logs-windows-x64-\u003Cunit|integration>-\u003Codd|even>-\u003Cattempt>`) and its own Slack tally line.\n\n## Important Pipeline Behavior\n\n- Each iteration is a separate timeline task named `Smoke test iteration \u003CN>\u002F\u003Ctotal> (Electron)`.\n- Iteration tasks use `continueOnError: true`, so a failed iteration normally has result `succeededWithIssues` and later iterations still run.\n- Do not infer smoke-test health from the overall build or job result. Inspect every iteration task.\n- Logs are published once per platform job from the shared `.build\u002Flogs` directory. The artifact is job-scoped, not iteration-scoped: `smoke-test-runner.log` can contain multiple iterations, and suite directories are not separated by iteration.\n- Do not cancel a platform job while investigating if you still need its logs artifact. Cancellation can skip the publish steps entirely, even when one or more failed iteration task logs are already available. If a validation run has become predictably red, either let the current job finish or queue a single diagnostic iteration and let that job publish before iterating.\n- Later iterations can overwrite or truncate files in the shared logs directory. The final artifact may therefore contain the last iteration's suite directory or runner log but not the failed iteration's detailed diagnostics. Preserve the exact task log, and use a one-iteration diagnostic run when extension-host, renderer, mock-server, screenshot, or trace evidence from the failed attempt is required.\n- The exact iteration task log is authoritative for its failure summary and time range. Correlate that range with the job-level artifact.\n- Iterations can see persisted smoke-test state from earlier iterations, such as historical session rows. Never assume the newest-looking row or a matching prompt belongs to the active conversation without verifying the active view.\n\nPlatform definitions and artifacts:\n\n| Platform | Job | Logs artifact | Crash artifact |\n|---|---|---|---|\n| macOS arm64 | `macOSSmokeFlaky` | `logs-macos-arm64-smoke-\u003Cattempt>` | `crash-dump-macos-arm64-smoke-\u003Cattempt>` |\n| Windows x64 | `WindowsSmokeFlaky` | `logs-windows-x64-smoke-\u003Cattempt>` | `crash-dump-windows-x64-smoke-\u003Cattempt>` |\n| Linux x64 | `LinuxSmokeFlaky` | `logs-linux-x64-smoke-\u003Cattempt>` | `crash-dump-linux-x64-smoke-\u003Cattempt>` |\n\n## Prerequisites\n\nUse Azure CLI rather than the browser:\n\n```bash\naz --version\naz extension show --name azure-devops\naz devops configure --defaults \\\n  organization=https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  project=Monaco\n```\n\nIf authentication fails, run `az login`.\n\n## 1. Confirm the Build and Source Revision\n\n```bash\naz pipelines build show \\\n  --id \u003CBUILD_ID> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --query \"{id:id,status:status,result:result,sourceBranch:sourceBranch,sourceVersion:sourceVersion,templateParameters:templateParameters}\" \\\n  --output json\n```\n\nRecord `sourceVersion`. Investigate the exact code built by the pipeline, not the current working tree.\n\n## 2. Find Failed Iterations\n\nDownload the build timeline:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource timeline \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> \\\n  --output json\n```\n\nFilter task records whose names contain `Smoke test iteration`. Treat `succeededWithIssues`, `failed`, and tasks with error issues as failed iterations. Record:\n\n- task `id`\n- parent job `id`\n- task `name`\n- `startTime` and `finishTime`\n- `log.id`\n- `issues`\n\nIf the user supplied an Azure log URL, its `j=` value is the job ID and its `t=` value is the task ID. Query those records directly:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource timeline \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> \\\n  --query \"records[?id=='\u003CJOB_OR_TASK_ID>'].{id:id,parentId:parentId,name:name,type:type,state:state,result:result,logId:log.id,startTime:startTime,finishTime:finishTime,issues:issues}\" \\\n  --output json\n```\n\n## 3. Download the Exact Iteration Task Log\n\nUse timeline record `log.id`. The correct REST resource is `logs`, not `buildLog`:\n\n```bash\naz devops invoke \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --area build \\\n  --resource logs \\\n  --route-parameters project=Monaco buildId=\u003CBUILD_ID> logId=\u003CLOG_ID> \\\n  --out-file task-log.json\n```\n\nThe downloaded response is JSON with a `value` array containing one string per log line. Convert it to plain text.\n\nPowerShell:\n\n```powershell\n(Get-Content -Raw task-log.json | ConvertFrom-Json).value |\n  Set-Content task-log.txt\n```\n\nBash:\n\n```bash\njq -r '.value[]' task-log.json > task-log.txt\n```\n\nThe task log provides the concise Mocha failure, stack, and the iteration's exact time range even before platform artifacts are published. Diagnostics written through the smoke runner's `Logger` (including `dumpFailureDiagnostics`) may exist only in `smoke-test-runner.log`, so do not assume they will be present in the Azure task log.\n\n## 4. Download the Platform Logs Artifact\n\nList artifacts first; names include the job attempt:\n\n```bash\naz pipelines runs artifact list \\\n  --run-id \u003CBUILD_ID> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --output table\n```\n\nDownload the relevant platform:\n\n```bash\naz pipelines runs artifact download \\\n  --run-id \u003CBUILD_ID> \\\n  --artifact-name \u003CLOGS_ARTIFACT> \\\n  --path \u003CDESTINATION> \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco\n```\n\nArtifacts are available after the platform job publishes its outputs. An absent platform artifact after a canceled run is expected; it is not evidence that the artifact name was wrong.\n\n## 5. Correlate the Failure\n\nStart with `\u003CDESTINATION>\u002Fsmoke-tests-electron\u002Fsmoke-test-runner.log`. Use the failed task's timestamps and test title to isolate the matching `Test start` \u002F `Test end` interval. Do not use the first occurrence of a test title because the runner log can contain several iterations.\n\nWithin that interval, establish this chain:\n\n1. **Gesture**: Was the expected editor clicked and prompt typed?\n2. **Dispatch**: Did the send action fire and did the new-session view close?\n3. **Request**: Did the mock server receive the expected scenario tag?\n4. **Tool loop**: For shell tests, did the second model request contain the expected tool result?\n5. **Rendering**: Did an assistant response render, and in which session view?\n6. **Routing**: Did the active session auto-swap to a new composer while the completed response remained in another session?\n\nFor mock-LLM suites, search for:\n\n- the scenario ID\n- `request body:`\n- the expected response marker\n- `model turn 1\u002F2` and `model turn 2\u002F2`\n\nThe request log distinguishes:\n\n- no dispatch\n- wrong provider or scenario\n- tool call never executed\n- tool result returned but UI rendering\u002Frouting lost it\n\nThen inspect the suite directory:\n\n```text\nsmoke-tests-electron\u002F\u003CN>_suite_\u003CSuite_Name>\u002F\n```\n\nUseful files include:\n\n- `window*\u002Fexthost\u002F\u003Cextension>\u002F\u003Cextension>.log`\n- `main.log`, `renderer.log`, and `agenthost.log`\n- `copilot-runtime-logs\u002Fprocess-*.log` — the Copilot runtime (`copilot-runtime` from `@github\u002Fcopilot-sdk-\u003Cplatform>`) process logs, captured by `dumpFailureDiagnostics` when a Copilot-runtime session fails. Check these first for a **hang or \"Timed out waiting for response\"**: they are the SDK\u002FCLI's own record (startup, auth, model request, turn lifecycle, panics, out-of-order or protocol errors) and explain a timeout the test error alone does not. A tail is also mirrored into `smoke-test-runner.log`. **Agent Host** sessions (Agents Window \u002F local AgentHost) write a full log run at `trace`; Chat Sessions editor (Copilot CLI \u002F Claude) and Local sessions write only a minimal startup log here (whether the runtime came up), with their detailed diagnostics in `GitHub Copilot Chat.log`. (Claude \u002F Codex sessions use a different runtime and are not captured here.)\n- `playwright-screenshot-*.png`\n- Playwright trace archives\n\nFor native exits or renderer crashes, also download the platform crash artifact.\n\n## 6. Interpret UI Diagnostics Carefully\n\nSession-list text and active-view text answer different questions:\n\n- A list row can contain a prompt from the current test but a command or title inherited from a prior request.\n- The active view can already be a fresh untitled composer while the response belongs to a completed, inactive session.\n- Broad response selectors can match stale DOM from an earlier session or iteration.\n- Reusing a fixed warm-up scenario marker can let a later warm-up wait match an earlier response and abandon the actually-running warm-up.\n- Standard smoke suites created through `installAllHandlers` use distinct randomized user-data directories. Do not attribute a setting from the preceding suite to the failing suite without comparing the actual `vscode-userdata:` or `--user-data-dir` paths in their logs. Iterations share broader job state, but sibling suites normally do not share the same profile.\n- A setup change can alter rather than fix the symptom. Re-check the exact stack and the gesture\u002Fdispatch\u002Frequest\u002Frendering chain on every validation run. For example, moving extension enablement before startup can eliminate an editor-open timeout while activating the extension before test settings are written, producing a later request or response timeout instead.\n\nPrefer assertions that establish identity:\n\n- unique scenario IDs or response markers per attempt\n- active chat\u002Fsession resource attributes\n- expected response in the active session before follow-up input\n- mock-server request count or captured request content after the gesture\n\nDo not \"fix\" these races by only increasing a timeout. Wait for the actual state transition or remove duplicate\u002Fin-flight work.\n\n## 7. Find the Introducing Commit\n\nUse the build's `sourceVersion` and identify the source file from the stack:\n\n```bash\ngit show \u003CSOURCE_VERSION>:test\u002Fsmoke\u002Fsrc\u002Fareas\u002F\u003Carea>\u002F\u003Ctest>.test.ts\ngit log --oneline \u003CKNOWN_GOOD>..\u003CSOURCE_VERSION> -- \u003Crelevant paths>\ngit blame -L \u003Cstart>,\u003Cend> \u003Cfile>\ngit show \u003CSUSPECT_COMMIT> -- \u003Crelevant paths>\n```\n\nTrace the whole causal sequence, not only the failing assertion. For example, inspect setup hooks, warm-ups, session selection, response waits, and teardown.\n\nDistinguish:\n\n- the commit that introduced the racy behavior\n- a later unrelated commit after which timing happened to expose it\n- a commit that only added diagnostics or made the flake visible\n\nState the introducing commit only when the diff contains the causal behavior and the pipeline evidence matches it.\n\n## 8. Queue a Focused Validation Run\n\nThe branch and commit must already be pushed to `microsoft\u002Fvscode`. Check for and cancel obsolete definition-700 runs on the same branch before queueing.\n\nUse `az pipelines run` directly because `iterations` is an object parameter:\n\n```bash\naz pipelines run \\\n  --id 700 \\\n  --branch \u003CBRANCH> \\\n  --commit-id \u003CCOMMIT_SHA> \\\n  --parameters \"iterations=[1,2,3,4,5,6]\" \\\n    VSCODE_BUILD_MACOS=true \\\n    VSCODE_BUILD_LINUX=false \\\n    VSCODE_BUILD_WIN32=true \\\n  --org https:\u002F\u002Fdev.azure.com\u002Fmonacotools \\\n  --project Monaco \\\n  --output json\n```\n\nEnable only platforms relevant to the failure. Confirm the queued run's `sourceVersion` and `templateParameters` with `az pipelines build show`.\n\nSix iterations are a useful quick validation sample. Use the default 20 when the failure is rare or when validating before declaring a recurring flake resolved.\n\n## 9. Validate the Local Change\n\nFor smoke-test TypeScript changes:\n\n```bash\npnpm -C test\u002Fsmoke run compile\nnode --experimental-strip-types build\u002Fhygiene.ts \u003Cchanged-file>\n```\n\nIf local dependencies are missing or stale, report that explicitly and rely on the focused definition-700 run for full compiled validation; do not silently skip validation or install unrelated tooling.\n\n## Expected Report\n\nSummarize:\n\n- build ID and source revision\n- failing platform and iteration count\n- exact failing test and symptom\n- dispatch\u002Frequest\u002Frendering evidence\n- root cause\n- introducing commit, with why it is causal\n- fix commit\n- focused validation run ID, platforms, iterations, and current result\n",{"name":5,"description":7},"main","d2b1bf2230b505648e972e81d854743de79b65af",".github\u002Fskills\u002Fflaky-smoke-tests",43773,"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode","https:\u002F\u002Fcode.visualstudio.com","https:\u002F\u002Fgithub.com\u002Fmicrosoft","https:\u002F\u002Fopensource.microsoft.com","Open source projects and samples from Microsoft","2026-11-11 08:47:49",{"source":8,"skillName":5,"commands":57,"prompt":67,"zipAvailable":68},[58,61,64],{"tool":59,"command":60},"npx","npx skills add microsoft\u002Fvscode --skill flaky-smoke-tests",{"tool":62,"command":63},"bunx","bunx skills add microsoft\u002Fvscode --skill flaky-smoke-tests",{"tool":65,"command":66},"pnpm","pnpm dlx skills add microsoft\u002Fvscode --skill flaky-smoke-tests","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fflaky-smoke-tests\u002FSKILL.md",true,[],[71,97,120,134,148,164],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":41},"01M70H0ANBKZAVEP1NVV4K28M3","sweeper-query","Answer natural-language questions about the VS Code Sweeper's published issue index (index-\u003Cname>.json for microsoft\u002Fvscode and the enrolled family repos) — which issues are agent-ready to implement or plan, what the sweeper proposed for an assignee or area, whether the sweeper knows a report as a duplicate, the highest-value open bugs in a lane — and, for one issue, the review's full record (evidence with file citations, fix brief, proposed comment). Use ONLY when the request explicitly names the sweeper — \"sweeper\", \"vscodesweeper\", \"sweeper-query\", a sweeper verdict, record, brief or index, an agent-ready issue — or asks what \"the sweeper\" found, proposed or knows. Do NOT use for general questions about issues, duplicates, priorities or ranking that do not mention the sweeper. Read-only; discovery only.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002F.github\u002Fskills\u002Fsweeper-query\u002FSKILL.md",[],[78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,34],"sweeper query","query","sweeper","review","fix","when","request","which","natural-language","ranking","duplicates","ready","only","read","plan","open","Citations","brief","issues",{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":41},"01M699DRJS4YJCV600Q72SVVWE","update-skills","Create or update repository skills and instructions when major learnings are discovered during a session. Use when the user says \"learn!\", when a significant pattern or pitfall is identified, or when reusable domain knowledge should be captured for future sessions.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fupdate-skills\u002FSKILL.md",[],[104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,83],"update","update skills","for","pattern","domain","learnings","reusable","use","session","user","instructions","knowledge","repository","learn","create","sessions",{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":41},"01M699DPWHZ72B66P82EW31RNE","update-pr","Update the pull request for the current session. Use when the user wants to push new changes to an existing PR.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fupdate-pr\u002FSKILL.md",[],[104,127,128,83,84,129,106,111,112,113,130,131,132,133],"pr","update pr","Existing","push","pull","pull request","changes",{"skillNo":135,"name":136,"nameZh":6,"description":137,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":138,"categories":139,"keywords":140,"publishedAt":41},"01M699DJMR6CE1MH5XCJZ0GFQ0","sync","Sync the current session branch with its upstream branch, or publish the current session branch to a remote. Use when the user asks to sync a branch, pull latest changes, rebase onto upstream, push current branch, publish branch, or set upstream.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fsync\u002FSKILL.md",[],[136,141,142,130,131,143,83,133,144,25,145,146,111,112,113,147],"SET","publish","rebase","latest","remote","upstream","branch",{"skillNo":149,"name":150,"nameZh":6,"description":151,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":152,"categories":153,"keywords":154,"publishedAt":41},"01M699DGXHX6A2A72X57R6GGHJ","sync-upstream","Update a stale session branch by rebasing onto the latest origin. Use when the upstream has moved significantly and the session needs to catch up, resolving conflicts by preserving upstream changes and adapting session work to fit.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fsync-upstream\u002FSKILL.md",[],[146,136,155,156,83,157,158,133,144,159,160,161,162,111,112,147,163,104],"sync upstream","fit","catch","catch up","origin","conflicts","stale","up","work",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":170,"publishedAt":41},"01M699DEXYFWDGW0BAXJ5QQ377","merge","Merge changes from the topic branch to the merge base branch. Use when the user wants to merge their session's work back to the base branch.","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fvscode\u002Fblob\u002Fmain\u002Fsrc\u002Fvs\u002Fsessions\u002Fskills\u002Fmerge\u002FSKILL.md",[],[166,112,111,163,147,113,83,171,172,133],"back","base"]