[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M698ZDRR8GH0HXYHZ26Q0QEC":3,"skill-same:01M698ZDRR8GH0HXYHZ26Q0QEC":95},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":52,"ownerDescription":53,"updatedAt":54,"install":55,"resources":68},"01M698ZDRR8GH0HXYHZ26Q0QEC","application-integration-troubleshooting",null,"Diagnoses failed, stuck, or silent Google Cloud Application Integration runs and Integration Connectors tasks using the REST API, Cloud Logging, and Cloud Monitoring. Use when an integration fails, returns HTTP 200 without doing the expected work, hangs in a suspended or retry state, or hits IAM, run-as service account, connector health, or quota errors. Do not use for initial region provisioning (use application-integration-basics) or building and deploying versions (use application-integration-authoring).","google\u002Fskills","google","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F1342004?v=4","01M698VYEFDM0VJBZDYYJ2T83S","Google","01M698WR0H76ENHE587T9EJT89",24122,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,9,33,34,35,36,37],"application integration troubleshooting","application","troubleshooting","integration","run","logging","tasks","monitoring","iam","api","google cloud","returns","work","Connector","authoring","as","retry","errors","when","2026-11-10 10:12:30",false,"---\nname: application-integration-troubleshooting\nmetadata:\n  version: \"1.0.2\"\n  category: ApplicationDevelopment\ndescription: >-\n  Diagnoses failed, stuck, or silent Google Cloud Application Integration runs and\n  Integration Connectors tasks using the REST API, Cloud Logging, and Cloud Monitoring.\n  Use when an integration fails, returns HTTP 200 without doing the expected work, hangs\n  in a suspended or retry state, or hits IAM, run-as service account, connector health,\n  or quota errors. Do not use for initial region provisioning (use\n  application-integration-basics) or building and deploying versions (use application-integration-authoring).\n---\n\n# Application Integration Troubleshooting\n\nDiagnose and recover Google Cloud Application Integration executions that fail, stall, or appear to do nothing. This skill walks through execution state verification, per-task failure inspection, Cloud Logging and Cloud Monitoring checks, Integration Connectors health probes, suspension resolution, and IAM or quota checks.\n\n## Attribution\n\n**MUST READ**: Before executing any `gcloud` command, REST API call, or Python\ndiscovery-client call, you MUST read\n[Usage Attribution](references\u002Fattribution.md). For commands executed directly\nin the terminal (never in generated scripts, Terraform, ADK agents, or user\ncode), prefix `gcloud` with\n`CLOUDSDK_METRICS_REQUEST_ATTRIBUTION=\"agent-skills\u002F1.0 (skill:application-integration-troubleshooting)\"`\nand add `agent-skills\u002F1.0 (skill:application-integration-troubleshooting)` to\nthe `User-Agent` header of `curl` requests.\n\n## Tooling and Scope Boundaries\n\n- **REST is the interface.** There is no `gcloud integrations` command group and no Google-managed MCP server for Application Integration. Use `curl` with `gcloud auth print-access-token`, or the discovery-based Python client (`googleapiclient.discovery.build(\"integrations\", \"v1\")` and `build(\"connectors\", \"v1\")`; install with `pip install \"google-api-python-client>=2.100.0,\u003C3\" \"google-auth>=2.20.0,\u003C3\"`) (changes host or credential state; get the user's approval first).\n- **Generic `gcloud` commands apply** for authentication (`gcloud auth`), enabling APIs (`gcloud services`), IAM checks (`gcloud projects get-iam-policy`, `gcloud iam service-accounts get-iam-policy`), and log queries (`gcloud logging read`).\n- **Regional scope.** Integrations, executions, connections, and execution logs live in a single Google Cloud region (`LOCATION`), which is the `LOCATION` in every REST path; no `gcloud` default-region setting applies. Ask the user which region the integration is deployed in. If they don't know, start with `us-central1` and, if `executions.list` is empty there, check the other [Application Integration locations](https:\u002F\u002Fdocs.cloud.google.com\u002Fapplication-integration\u002Fdocs\u002Flocations.md.txt) before concluding nothing ran (Step 2).\n- **Sibling skills:**\n  - Use `application-integration-basics` for product concepts, enabling APIs, provisioning a region, initial IAM roles, and service limits.\n  - Use `application-integration-authoring` for creating drafts, editing tasks and triggers, config variables, publishing, `:execute` and `:schedule` payloads, and CI\u002FCD export\u002Fimport.\n\n## Safety and Confirmation Rules\n\n- **Read-only checks run freely:** `executions.list`, `suspensions.list`, `connections.get`, `connectionSchemaMetadata` get calls, runtime v2 `actions.list` \u002F `entityTypes.list`, `gcloud logging read`, and `v2` `executions:troubleshoot`.\n- **State-changing actions require explicit user confirmation.** Before canceling or replaying an execution, lifting or resolving a suspension, refreshing connector schema metadata, repairing connector eventing, publishing or unpublishing a version, or changing IAM bindings, show the exact command and wait for user approval.\n- **Host changes require approval.** Before installing packages (`pip install`) or changing global gcloud or credential state (`gcloud config set ...` such as `auth\u002Fimpersonate_service_account`, `gcloud auth application-default login` \u002F `set-quota-project`), show the exact command and wait for the user's approval.\n- **Never bundle destructive scripts** or delete integrations, versions, connections, or auth profiles during troubleshooting unless the user specifically asks.\n\n## Prerequisites and Setup Check\n\nBefore querying APIs, confirm the environment has credentials, active APIs, and the right target region:\n\n```bash\n# Verify active identity and project\ngcloud config get-value account\ngcloud config get-value project\n\n# Check that required APIs are enabled on PROJECT_ID\ngcloud services list --enabled --project=\"PROJECT_ID\" \\\n  --filter=\"config.name:(integrations.googleapis.com OR connectors.googleapis.com OR secretmanager.googleapis.com)\"\n```\n\n- If an API is missing, ask the user before running `gcloud services enable integrations.googleapis.com connectors.googleapis.com secretmanager.googleapis.com --project=\"PROJECT_ID\"`.\n- Ensure an active billing account is attached to `PROJECT_ID` (required even when usage stays within the monthly free tier; see the [Application Integration pricing page](https:\u002F\u002Fcloud.google.com\u002Fapplication-integration\u002Fpricing)).\n- For Python discovery clients (install with `pip install \"google-api-python-client>=2.100.0,\u003C3\" \"google-auth>=2.20.0,\u003C3\"`) or ADK (`ApplicationIntegrationToolset`), authenticate with `gcloud auth application-default login` and `gcloud auth application-default set-quota-project PROJECT_ID` (changes host or credential state; get the user's approval first).\n\n## Diagnostic Decision Flow\n\nFollow these steps in order whenever a user reports that an integration failed, did nothing, or is stuck.\n\n### Step 1: Confirm Whether an Execution Ran and Check Its True State\n\nDo not rely on an HTTP 200 status from `:execute` as proof that every task succeeded (note: `integrations:test` also exists in the v1 REST API, but its required `TestIntegrationsRequest.clientId` value is not publicly documented, so do not invent one; test by publishing to a non-production project and calling `:execute`). As a diagnostic rule, always check the response payload and execution record because:\n- The required verification is the execution record from `executions.list?filter=execution_id=\"EXECUTION_ID\"`—inspect `executionDetails.state`, each task's `taskExecutionState`, and `ErrorInfo` in `responseParameters` (do not rely on the legacy\u002Fdeprecated `executionFailed` field, which is not a supported public signal).\n- If the caller passed `\"doNotPropagateError\": true` on `:execute`, the API returns HTTP 200 with an `executionId` and places error details inside event parameters instead of returning an HTTP error.\n- A duplicate `requestId` on `:execute` or `:schedule` returns the previous execution's response without running a new execution.\n- Task failure policies (`IGNORE` and `NONE`) allow an integration to continue or finish as `SUCCEEDED` down an alternate branch even when an individual task failed.\n\nList executions for the integration (use `-` as `INTEGRATION_NAME` to search across all integrations in the region; the discovery doc does not specify whether `orderBy=create_time` sorts ascending or descending, so add a `CreateTimestamp` filter or paginate with `pageToken` if needed):\n\n```bash\ncurl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions?pageSize=10&orderBy=create_time&snapshotMetadataWithoutParams=true\"\n```\n\nOnce you have the `EXECUTION_ID`, fetch its full `ExecutionDetails` and per-task states (use `filter`, `snapshotMetadataWithoutParams=true`, and `readMask` on `executions.list`—never the deprecated `truncateParams` [UI-only] or `filterParams.*` [internal-only] parameters):\n\n```bash\ncurl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions?filter=execution_id%3D%22EXECUTION_ID%22\"\n```\n\nTo filter `executions.list` by state, use the `state` field (`=` or `!=`) with unquoted `ExecutionDetails.state` values (for example, `filter=state=FAILED`, `filter=state!=SUCCEEDED`, `filter=state=CANCELLED`, or `filter=state=FAILED OR state=CANCELLED`, URL-encoded as `filter=state%3DFAILED` or `filter=state%3DFAILED%20OR%20state%3DCANCELLED`); use `state` (not `eventExecutionState`, as shown on the REST reference page, or `event_execution_state` in the API reference's filter description), and use the supported filter keys (`execution_id`, `state`, `update_time`, `workflow_name`, `trigger_id`, `CreateTimestamp`, `parameter_type`)—camelCase variants such as `executionId` are not supported even though some docs examples show them.\n\n### Step 2: What If No Execution Record Exists?\n\nIf `executions.list` returns no matching run, check these five causes before assuming the trigger never fired:\n\n| Cause | Symptom | What to verify |\n|---|---|---|\n| **Wrong region** | Empty list in `LOCATION` | Check `https:\u002F\u002Fconsole.cloud.google.com\u002Fintegrations` or list integrations in other regions. |\n| **API trigger 403** | Caller got HTTP 403 `PERMISSION_DENIED` | API trigger auth denials fail before execution creation; nothing is written to execution logs. Check caller IAM and `roles\u002Fiam.serviceAccountUser` on the run-as service account. |\n| **API trigger payload > 8 MB** | Caller got an error | Cumulative API trigger input variables exceed the 8 MB limit, so the integration does not start. |\n| **Local logging disabled** | Integration ran, but logs are empty | Check `IntegrationVersion.databasePersistencePolicy` (`DATABASE_PERSISTENCE_DISABLED` = `Disable`; `DATABASE_PERSISTENCE_POLICY_UNSPECIFIED` = `Enable (sync)`, the API default; `DATABASE_PERSISTENCE_ASYNC` = `Enable (async)`, the console default). Also, if total integration data exceeds 10 MB (up to the 30 MB limit), local logging is disabled automatically. |\n| **Version not `ACTIVE`** | Trigger never fires | Only an `ACTIVE` (published) version runs from triggers. After publishing, an older version can keep serving for up to 10 minutes due to eventual consistency. |\n\n### Step 3: Branch by Execution State (`ExecutionDetails.state`)\n\nSee [Execution States](references\u002Fexecution-states.md) for the full state reference, task-level states (`TaskExecutionDetails.taskExecutionState`), filter syntax, and cancel\u002Freplay rules.\n\n| `ExecutionDetails.state` | Console status | Meaning and next step |\n|---|---|---|\n| `FAILED` | Failed | Check `ErrorInfo` (`message`, `code`) in `responseParameters`, inspect the failing task's `taskExecutionState` (`FAILED` or `FATAL`), and run Step 4 (`:troubleshoot` and Cloud Logging). |\n| `SUCCEEDED` | Succeeded | If expected work did not happen, inspect `taskExecutionDetails` for tasks in `SKIPPED` (edge condition false) or `FAILED` (masked by `IGNORE` or `NONE` retry strategy). |\n| `CANCELLED` | Canceled | User canceled the run, a new integration version was published while the run was suspended, a SYNC run hit the 2-minute timeout, or a replayed run hit the 10-minute deadline. |\n| `SUSPENDED` | Suspended | Waiting on an Approval task (resolvable in the Cloud console UI or via `suspensions:resolve`) or a Suspend task (resumable via `suspensions:lift`). See [Suspensions](references\u002Fsuspensions.md). |\n| `RETRY_ON_HOLD` | Retry on hold | A task or integration failed with a temporary error and is waiting in the backoff retry queue. |\n| `PROCESSING` | In process | Currently running. Check whether an ASYNC run is approaching the 10-minute timeout. |\n| `PENDING` | On hold | Scheduled or temporarily paused and waiting to be triggered. |\n\n### Step 4: Inspect Logs, AI Root Cause, and Subsystem Health\n\n1. **Run AI Troubleshoot (v2 REST API)** on a failed execution to get a structured root-cause summary (requires `integrations.executions.get`):\n   ```bash\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv2\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:troubleshoot\"\n   ```\n2. **Query Cloud Logging** if Cloud Logging was enabled on the version. Note that Cloud Logging is **disabled by default** and writes nothing if local logging is disabled. See [Logging and Monitoring](references\u002Flogging-and-monitoring.md) for how to enable it, severity behavior, and exact queries.\n   ```bash\n   gcloud logging read \\\n     'resource.type=\"integrations.googleapis.com\u002FIntegrationVersion\" AND jsonPayload.executionId=\"EXECUTION_ID\"' \\\n     --project=\"PROJECT_ID\" --limit=20 --format=json\n   ```\n3. **If a Connectors task failed**, do not guess whether the connector or IAM is at fault. Inspect the connection resource (`connections.get`) and its schema metadata (`GET ...\u002Fconnections\u002FCONNECTION_NAME\u002FconnectionSchemaMetadata` for `state` and `errorMessage`, and runtime v2 `GET ...\u002Fv2\u002F...\u002Fconnections\u002FCONNECTION_NAME\u002Factions` and `...\u002FentityTypes` for listing—v1 `connectionSchemaMetadata:listActions` and `:listEntityTypes` require a `filter` and reject unfiltered calls). See [Connectors Health](references\u002Fconnectors-health.md) for connection states, schema metadata checks, and node throttling (HTTP 429).\n   ```bash\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fconnectors.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fconnections\u002FCONNECTION_NAME?view=FULL\"\n   ```\n4. **If an IAM (403), run-as service account, timeout, or quota error occurred**, check both the caller's role and the integration's run-as service account. Integrations created on or after **October 15, 2026** with any trigger other than an API or Private trigger (for example Schedule, Cloud Scheduler, Cloud Pub\u002FSub, Salesforce, or Connector Event; Private triggers are exempt like API triggers, so a sub-integration whose only trigger is Private can be published without `runAsServiceAccount`) cannot be tested or published unless `runAsServiceAccount` is set; whenever `runAsServiceAccount` is set, the user publishing or testing the version needs `roles\u002Fiam.serviceAccountUser` on it. See [Common Errors](references\u002Fcommon-errors.md) for the two-sided 403 checklist, error codes, task limits, and ADK `ApplicationIntegrationToolset` errors.\n\n## Common Diagnostic Scenarios\n\n### Scenario A: Failed Connectors Task (Connection Health vs Caller IAM)\n\n1. Run `connections.get` with `view=FULL` (command above) and inspect `status.state` and `status.status`.\n2. **If `status.state` is `ERROR` or `AUTHORIZATION_REQUIRED`:** the connection itself cannot reach or authenticate to the target backend. Fix the connection's credentials in Secret Manager or the Cloud console (`https:\u002F\u002Fconsole.cloud.google.com\u002Fconnectors\u002Fconnections`). If the connection uses a service account in another project, verify `service-PROJECT_NUMBER@gcp-sa-connectors.iam.gserviceaccount.com` has `roles\u002Fiam.serviceAccountTokenCreator` in that project.\n3. **If `status.state` is `ACTIVE`:** the connection is healthy. Check whether the identity running the task (the integration's `runAsServiceAccount`, or if unset, `service-PROJECT_NUMBER@gcp-sa-integrations.iam.gserviceaccount.com`) has a required Integration Connectors role (such as `roles\u002Fconnectors.invoker` or `roles\u002Fconnectors.admin`) in the connection's project.\n4. **If the task failed with HTTP 429:** the connection exceeded its provisioned nodes. Raise min\u002Fmax nodes on the connection and add a task retry strategy (`EXPONENTIAL_BACKOFF`).\n5. **Trace the exact connector attempt in Cloud Logging** (when connector logging is enabled on the connection):\n   ```bash\n   gcloud logging read \\\n     'resource.type=\"connectors.googleapis.com\u002FConnection\" AND jsonPayload.integration_connectors_trace_id=\"EXECUTIONID_TASKID_ATTEMPTNUMBER\"' \\\n     --project=\"PROJECT_ID\" --limit=20 --format=json\n   ```\n\n### Scenario B: Stuck in `SUSPENDED` (Approval vs Suspend Task)\n\n1. Identify which task paused the execution from `taskExecutionDetails` and list pending suspensions (read-only; requires `integrations.suspensions.list`, included in `roles\u002Fintegrations.suspensionResolver`) to find `SUSPENSION_ID` and `taskId`. Note that `roles\u002Fintegrations.suspensionResolver` alone cannot read execution records (no `integrations.executions.list` or `integrations.executions.get`, returning HTTP 403 on `executions.list` and v2 `:troubleshoot`); to inspect `taskExecutionDetails` (or run `:troubleshoot`), also grant `roles\u002Fintegrations.integrationViewer` (or use the approval email link, which carries the execution and task IDs). See [Suspensions](references\u002Fsuspensions.md) for full details.\n   ```bash\n   # Read-only: list suspensions on EXECUTION_ID\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\"\n   ```\n2. **Approval task:** can be approved or rejected in the Cloud console UI or via the REST API (`POST https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\u002FSUSPENSION_ID:resolve` with body `{\"suspension\": {\"eventExecutionInfoId\": \"EXECUTION_ID\", \"integration\": \"INTEGRATION_NAME\", \"taskId\": \"TASK_ID\", \"state\": \"LIFTED\"}}` or `\"REJECTED\"`—the same call the console Approve\u002FReject buttons make). Always confirm with the user before resolving. Verify the caller holds `integrations.suspensions.resolve` (included in `roles\u002Fintegrations.suspensionResolver`, which is not included in `roles\u002Fintegrations.integrationEditor`) **and** `roles\u002Fiam.serviceAccountUser` on the integration's `runAsServiceAccount`. Without `roles\u002Fiam.serviceAccountUser`, approving leaves the execution paused until it expires.\n3. **Suspend task:** `:lift` is the Suspend-task variant and requires `integrations.suspensions.lift` (included in `roles\u002Fintegrations.suspensionResolver`) plus `roles\u002Fiam.serviceAccountUser` on the integration's `runAsServiceAccount`. After user confirmation, call `POST ...\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\u002FSUSPENSION_ID:lift` with an optional `suspensionResult`.\n4. **Watch out for version publish cancellation:** publishing a new version of an integration automatically cancels all suspended executions of the previous version.\n\n### Scenario C: HTTP 403 `PERMISSION_DENIED` on Trigger, Test, or Publish\n\nApplication Integration enforces two independent permission checks:\n1. **Caller IAM check:** the caller needs permission on the integration (`roles\u002Fintegrations.integrationInvoker` to invoke, `roles\u002Fintegrations.integrationEditor` to edit\u002Ftest, `roles\u002Fintegrations.integrationDeployer` or `roles\u002Fintegrations.integrationEditor` to publish).\n2. **Run-as service account check:** whenever `runAsServiceAccount` is set on the version, the user (or principal) publishing or testing the version needs `roles\u002Fiam.serviceAccountUser` on `runAsServiceAccount` at publish\u002Ftest time, and at runtime the invoking caller or trigger service account needs `roles\u002Fiam.serviceAccountUser` on `runAsServiceAccount` (granted in the project that owns the service account). In addition, `runAsServiceAccount` needs roles on downstream Google Cloud resources.\n\nCheck both bindings before retrying. For Pub\u002FSub, Eventarc, and Connector Event triggers, a run-as service account (`roles\u002Fiam.serviceAccountUser`) denial records a failed execution and the event is **not** redelivered.\n\n## Recovery Actions (Require User Confirmation)\n\nAlways display the exact command and obtain user confirmation before running `:cancel`, `:replay`, `:lift`, or `:resolve`. `:cancel` requires `integrations.executions.cancel` and `:replay` requires `integrations.executions.replay` (both are held by `roles\u002Fintegrations.integrationInvoker`, `roles\u002Fintegrations.integrationEditor`, and `roles\u002Fintegrations.integrationAdmin`—**not** by `roles\u002Fintegrations.suspensionResolver` or `roles\u002Fintegrations.integrationViewer`, so approvers with only `roles\u002Fintegrations.suspensionResolver` cannot cancel a stuck `SUSPENDED` run). `:replay` also enforces two preconditions that return `FAILED_PRECONDITION`: (a) test-case runs cannot be replayed in any mode (`\"Test runs cannot be replayed.\"`), and (b) `REPLAY_MODE_POINT_OF_FAILURE` is refused for executions that ran with database persistence (local logging \u002F `databasePersistencePolicy: DATABASE_PERSISTENCE_DISABLED`) disabled because it needs execution snapshots—use `REPLAY_MODE_FROM_BEGINNING` in that case. When overriding inputs on `:replay` with `modifiedParameters` and `updateMask`, list every overridden key in `updateMask`; without `updateMask`, all original inputs are discarded and only `modifiedParameters` are used, and a key in `updateMask` missing from `modifiedParameters` is an error.\n\n```bash\n# Cancel an execution in PENDING (console \"On hold\"), RETRY_ON_HOLD, or SUSPENDED state (requires integrations.executions.cancel)\n# (Cannot cancel executions in PROCESSING state [console \"In process\"], completed runs [SUCCEEDED, FAILED, CANCELLED], or Schedule-trigger runs; approvers with only roles\u002Fintegrations.suspensionResolver cannot cancel a stuck SUSPENDED run)\ncurl -X POST \\\n  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"cancelReason\": \"Canceling stuck execution after investigation\"}' \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:cancel\"\n\n# Replay a SUCCEEDED, FAILED, or CANCELLED execution on a published integration (Preview; requires integrations.executions.replay; test-case runs and Schedule-trigger runs cannot be replayed)\n# replayMode: REPLAY_MODE_FROM_BEGINNING or REPLAY_MODE_POINT_OF_FAILURE (failed runs only; refused with FAILED_PRECONDITION if databasePersistencePolicy was DATABASE_PERSISTENCE_DISABLED—use REPLAY_MODE_FROM_BEGINNING in that case)\n# Optional modifiedParameters\u002FupdateMask: list every overridden key in updateMask; without updateMask, all original inputs are discarded and only modifiedParameters are used; a key in updateMask missing from modifiedParameters is an error\ncurl -X POST \\\n  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"replayReason\": \"Replay after fixing downstream permission\", \"replayMode\": \"REPLAY_MODE_POINT_OF_FAILURE\"}' \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:replay\"\n```\n\n## Reference Files\n\nLoad the matching reference file when you need deeper details, exact field names, or edge-case checks:\n\n- [Usage Attribution](references\u002Fattribution.md): Usage attribution for `gcloud` CLI commands, direct `curl` REST calls, and Python discovery-client snippets run in the terminal.\n- [Execution States](references\u002Fexecution-states.md): `ExecutionDetails.state` and `TaskExecutionDetails.taskExecutionState` enums, `executions.list` filter operators, large-log workarounds (> 300 MB), and `:cancel` \u002F `:replay` constraints.\n- [Logging and Monitoring](references\u002Flogging-and-monitoring.md): Local logging vs Cloud Logging interactions, enabling Cloud Logging on a draft version, Logs Explorer queries, `integrations.googleapis.com\u002F*` and `connectors.googleapis.com\u002F*` metric schemas, and alerting setup.\n- [Common Errors](references\u002Fcommon-errors.md): HTTP\u002FgRPC error codes, the two-sided 403 checklist, the October 15, 2026 run-as service account requirement, task-level limits and gotchas, and ADK `ApplicationIntegrationToolset` errors.\n- [Connectors Health](references\u002Fconnectors-health.md): `ConnectionStatus.state`, `EventingStatus.state`, `connectionSchemaMetadata` probes, runtime v2 readiness calls, node throttling (HTTP 429), and separating connection auth from caller IAM.\n- [Suspensions](references\u002Fsuspensions.md): Suspend vs Approval vs Timer behavior, `suspensions.list` \u002F `:lift` \u002F `:resolve` payloads, expiration settings, and why approvals stay stuck without `roles\u002Fiam.serviceAccountUser`.\n","# Application Integration Troubleshooting\n\nDiagnose and recover Google Cloud Application Integration executions that fail, stall, or appear to do nothing. This skill walks through execution state verification, per-task failure inspection, Cloud Logging and Cloud Monitoring checks, Integration Connectors health probes, suspension resolution, and IAM or quota checks.\n\n## Attribution\n\n**MUST READ**: Before executing any `gcloud` command, REST API call, or Python\ndiscovery-client call, you MUST read\n[Usage Attribution](references\u002Fattribution.md). For commands executed directly\nin the terminal (never in generated scripts, Terraform, ADK agents, or user\ncode), prefix `gcloud` with\n`CLOUDSDK_METRICS_REQUEST_ATTRIBUTION=\"agent-skills\u002F1.0 (skill:application-integration-troubleshooting)\"`\nand add `agent-skills\u002F1.0 (skill:application-integration-troubleshooting)` to\nthe `User-Agent` header of `curl` requests.\n\n## Tooling and Scope Boundaries\n\n- **REST is the interface.** There is no `gcloud integrations` command group and no Google-managed MCP server for Application Integration. Use `curl` with `gcloud auth print-access-token`, or the discovery-based Python client (`googleapiclient.discovery.build(\"integrations\", \"v1\")` and `build(\"connectors\", \"v1\")`; install with `pip install \"google-api-python-client>=2.100.0,\u003C3\" \"google-auth>=2.20.0,\u003C3\"`) (changes host or credential state; get the user's approval first).\n- **Generic `gcloud` commands apply** for authentication (`gcloud auth`), enabling APIs (`gcloud services`), IAM checks (`gcloud projects get-iam-policy`, `gcloud iam service-accounts get-iam-policy`), and log queries (`gcloud logging read`).\n- **Regional scope.** Integrations, executions, connections, and execution logs live in a single Google Cloud region (`LOCATION`), which is the `LOCATION` in every REST path; no `gcloud` default-region setting applies. Ask the user which region the integration is deployed in. If they don't know, start with `us-central1` and, if `executions.list` is empty there, check the other [Application Integration locations](https:\u002F\u002Fdocs.cloud.google.com\u002Fapplication-integration\u002Fdocs\u002Flocations.md.txt) before concluding nothing ran (Step 2).\n- **Sibling skills:**\n  - Use `application-integration-basics` for product concepts, enabling APIs, provisioning a region, initial IAM roles, and service limits.\n  - Use `application-integration-authoring` for creating drafts, editing tasks and triggers, config variables, publishing, `:execute` and `:schedule` payloads, and CI\u002FCD export\u002Fimport.\n\n## Safety and Confirmation Rules\n\n- **Read-only checks run freely:** `executions.list`, `suspensions.list`, `connections.get`, `connectionSchemaMetadata` get calls, runtime v2 `actions.list` \u002F `entityTypes.list`, `gcloud logging read`, and `v2` `executions:troubleshoot`.\n- **State-changing actions require explicit user confirmation.** Before canceling or replaying an execution, lifting or resolving a suspension, refreshing connector schema metadata, repairing connector eventing, publishing or unpublishing a version, or changing IAM bindings, show the exact command and wait for user approval.\n- **Host changes require approval.** Before installing packages (`pip install`) or changing global gcloud or credential state (`gcloud config set ...` such as `auth\u002Fimpersonate_service_account`, `gcloud auth application-default login` \u002F `set-quota-project`), show the exact command and wait for the user's approval.\n- **Never bundle destructive scripts** or delete integrations, versions, connections, or auth profiles during troubleshooting unless the user specifically asks.\n\n## Prerequisites and Setup Check\n\nBefore querying APIs, confirm the environment has credentials, active APIs, and the right target region:\n\n```bash\n# Verify active identity and project\ngcloud config get-value account\ngcloud config get-value project\n\n# Check that required APIs are enabled on PROJECT_ID\ngcloud services list --enabled --project=\"PROJECT_ID\" \\\n  --filter=\"config.name:(integrations.googleapis.com OR connectors.googleapis.com OR secretmanager.googleapis.com)\"\n```\n\n- If an API is missing, ask the user before running `gcloud services enable integrations.googleapis.com connectors.googleapis.com secretmanager.googleapis.com --project=\"PROJECT_ID\"`.\n- Ensure an active billing account is attached to `PROJECT_ID` (required even when usage stays within the monthly free tier; see the [Application Integration pricing page](https:\u002F\u002Fcloud.google.com\u002Fapplication-integration\u002Fpricing)).\n- For Python discovery clients (install with `pip install \"google-api-python-client>=2.100.0,\u003C3\" \"google-auth>=2.20.0,\u003C3\"`) or ADK (`ApplicationIntegrationToolset`), authenticate with `gcloud auth application-default login` and `gcloud auth application-default set-quota-project PROJECT_ID` (changes host or credential state; get the user's approval first).\n\n## Diagnostic Decision Flow\n\nFollow these steps in order whenever a user reports that an integration failed, did nothing, or is stuck.\n\n### Step 1: Confirm Whether an Execution Ran and Check Its True State\n\nDo not rely on an HTTP 200 status from `:execute` as proof that every task succeeded (note: `integrations:test` also exists in the v1 REST API, but its required `TestIntegrationsRequest.clientId` value is not publicly documented, so do not invent one; test by publishing to a non-production project and calling `:execute`). As a diagnostic rule, always check the response payload and execution record because:\n- The required verification is the execution record from `executions.list?filter=execution_id=\"EXECUTION_ID\"`—inspect `executionDetails.state`, each task's `taskExecutionState`, and `ErrorInfo` in `responseParameters` (do not rely on the legacy\u002Fdeprecated `executionFailed` field, which is not a supported public signal).\n- If the caller passed `\"doNotPropagateError\": true` on `:execute`, the API returns HTTP 200 with an `executionId` and places error details inside event parameters instead of returning an HTTP error.\n- A duplicate `requestId` on `:execute` or `:schedule` returns the previous execution's response without running a new execution.\n- Task failure policies (`IGNORE` and `NONE`) allow an integration to continue or finish as `SUCCEEDED` down an alternate branch even when an individual task failed.\n\nList executions for the integration (use `-` as `INTEGRATION_NAME` to search across all integrations in the region; the discovery doc does not specify whether `orderBy=create_time` sorts ascending or descending, so add a `CreateTimestamp` filter or paginate with `pageToken` if needed):\n\n```bash\ncurl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions?pageSize=10&orderBy=create_time&snapshotMetadataWithoutParams=true\"\n```\n\nOnce you have the `EXECUTION_ID`, fetch its full `ExecutionDetails` and per-task states (use `filter`, `snapshotMetadataWithoutParams=true`, and `readMask` on `executions.list`—never the deprecated `truncateParams` [UI-only] or `filterParams.*` [internal-only] parameters):\n\n```bash\ncurl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions?filter=execution_id%3D%22EXECUTION_ID%22\"\n```\n\nTo filter `executions.list` by state, use the `state` field (`=` or `!=`) with unquoted `ExecutionDetails.state` values (for example, `filter=state=FAILED`, `filter=state!=SUCCEEDED`, `filter=state=CANCELLED`, or `filter=state=FAILED OR state=CANCELLED`, URL-encoded as `filter=state%3DFAILED` or `filter=state%3DFAILED%20OR%20state%3DCANCELLED`); use `state` (not `eventExecutionState`, as shown on the REST reference page, or `event_execution_state` in the API reference's filter description), and use the supported filter keys (`execution_id`, `state`, `update_time`, `workflow_name`, `trigger_id`, `CreateTimestamp`, `parameter_type`)—camelCase variants such as `executionId` are not supported even though some docs examples show them.\n\n### Step 2: What If No Execution Record Exists?\n\nIf `executions.list` returns no matching run, check these five causes before assuming the trigger never fired:\n\n| Cause | Symptom | What to verify |\n|---|---|---|\n| **Wrong region** | Empty list in `LOCATION` | Check `https:\u002F\u002Fconsole.cloud.google.com\u002Fintegrations` or list integrations in other regions. |\n| **API trigger 403** | Caller got HTTP 403 `PERMISSION_DENIED` | API trigger auth denials fail before execution creation; nothing is written to execution logs. Check caller IAM and `roles\u002Fiam.serviceAccountUser` on the run-as service account. |\n| **API trigger payload > 8 MB** | Caller got an error | Cumulative API trigger input variables exceed the 8 MB limit, so the integration does not start. |\n| **Local logging disabled** | Integration ran, but logs are empty | Check `IntegrationVersion.databasePersistencePolicy` (`DATABASE_PERSISTENCE_DISABLED` = `Disable`; `DATABASE_PERSISTENCE_POLICY_UNSPECIFIED` = `Enable (sync)`, the API default; `DATABASE_PERSISTENCE_ASYNC` = `Enable (async)`, the console default). Also, if total integration data exceeds 10 MB (up to the 30 MB limit), local logging is disabled automatically. |\n| **Version not `ACTIVE`** | Trigger never fires | Only an `ACTIVE` (published) version runs from triggers. After publishing, an older version can keep serving for up to 10 minutes due to eventual consistency. |\n\n### Step 3: Branch by Execution State (`ExecutionDetails.state`)\n\nSee [Execution States](references\u002Fexecution-states.md) for the full state reference, task-level states (`TaskExecutionDetails.taskExecutionState`), filter syntax, and cancel\u002Freplay rules.\n\n| `ExecutionDetails.state` | Console status | Meaning and next step |\n|---|---|---|\n| `FAILED` | Failed | Check `ErrorInfo` (`message`, `code`) in `responseParameters`, inspect the failing task's `taskExecutionState` (`FAILED` or `FATAL`), and run Step 4 (`:troubleshoot` and Cloud Logging). |\n| `SUCCEEDED` | Succeeded | If expected work did not happen, inspect `taskExecutionDetails` for tasks in `SKIPPED` (edge condition false) or `FAILED` (masked by `IGNORE` or `NONE` retry strategy). |\n| `CANCELLED` | Canceled | User canceled the run, a new integration version was published while the run was suspended, a SYNC run hit the 2-minute timeout, or a replayed run hit the 10-minute deadline. |\n| `SUSPENDED` | Suspended | Waiting on an Approval task (resolvable in the Cloud console UI or via `suspensions:resolve`) or a Suspend task (resumable via `suspensions:lift`). See [Suspensions](references\u002Fsuspensions.md). |\n| `RETRY_ON_HOLD` | Retry on hold | A task or integration failed with a temporary error and is waiting in the backoff retry queue. |\n| `PROCESSING` | In process | Currently running. Check whether an ASYNC run is approaching the 10-minute timeout. |\n| `PENDING` | On hold | Scheduled or temporarily paused and waiting to be triggered. |\n\n### Step 4: Inspect Logs, AI Root Cause, and Subsystem Health\n\n1. **Run AI Troubleshoot (v2 REST API)** on a failed execution to get a structured root-cause summary (requires `integrations.executions.get`):\n   ```bash\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv2\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:troubleshoot\"\n   ```\n2. **Query Cloud Logging** if Cloud Logging was enabled on the version. Note that Cloud Logging is **disabled by default** and writes nothing if local logging is disabled. See [Logging and Monitoring](references\u002Flogging-and-monitoring.md) for how to enable it, severity behavior, and exact queries.\n   ```bash\n   gcloud logging read \\\n     'resource.type=\"integrations.googleapis.com\u002FIntegrationVersion\" AND jsonPayload.executionId=\"EXECUTION_ID\"' \\\n     --project=\"PROJECT_ID\" --limit=20 --format=json\n   ```\n3. **If a Connectors task failed**, do not guess whether the connector or IAM is at fault. Inspect the connection resource (`connections.get`) and its schema metadata (`GET ...\u002Fconnections\u002FCONNECTION_NAME\u002FconnectionSchemaMetadata` for `state` and `errorMessage`, and runtime v2 `GET ...\u002Fv2\u002F...\u002Fconnections\u002FCONNECTION_NAME\u002Factions` and `...\u002FentityTypes` for listing—v1 `connectionSchemaMetadata:listActions` and `:listEntityTypes` require a `filter` and reject unfiltered calls). See [Connectors Health](references\u002Fconnectors-health.md) for connection states, schema metadata checks, and node throttling (HTTP 429).\n   ```bash\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fconnectors.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fconnections\u002FCONNECTION_NAME?view=FULL\"\n   ```\n4. **If an IAM (403), run-as service account, timeout, or quota error occurred**, check both the caller's role and the integration's run-as service account. Integrations created on or after **October 15, 2026** with any trigger other than an API or Private trigger (for example Schedule, Cloud Scheduler, Cloud Pub\u002FSub, Salesforce, or Connector Event; Private triggers are exempt like API triggers, so a sub-integration whose only trigger is Private can be published without `runAsServiceAccount`) cannot be tested or published unless `runAsServiceAccount` is set; whenever `runAsServiceAccount` is set, the user publishing or testing the version needs `roles\u002Fiam.serviceAccountUser` on it. See [Common Errors](references\u002Fcommon-errors.md) for the two-sided 403 checklist, error codes, task limits, and ADK `ApplicationIntegrationToolset` errors.\n\n## Common Diagnostic Scenarios\n\n### Scenario A: Failed Connectors Task (Connection Health vs Caller IAM)\n\n1. Run `connections.get` with `view=FULL` (command above) and inspect `status.state` and `status.status`.\n2. **If `status.state` is `ERROR` or `AUTHORIZATION_REQUIRED`:** the connection itself cannot reach or authenticate to the target backend. Fix the connection's credentials in Secret Manager or the Cloud console (`https:\u002F\u002Fconsole.cloud.google.com\u002Fconnectors\u002Fconnections`). If the connection uses a service account in another project, verify `service-PROJECT_NUMBER@gcp-sa-connectors.iam.gserviceaccount.com` has `roles\u002Fiam.serviceAccountTokenCreator` in that project.\n3. **If `status.state` is `ACTIVE`:** the connection is healthy. Check whether the identity running the task (the integration's `runAsServiceAccount`, or if unset, `service-PROJECT_NUMBER@gcp-sa-integrations.iam.gserviceaccount.com`) has a required Integration Connectors role (such as `roles\u002Fconnectors.invoker` or `roles\u002Fconnectors.admin`) in the connection's project.\n4. **If the task failed with HTTP 429:** the connection exceeded its provisioned nodes. Raise min\u002Fmax nodes on the connection and add a task retry strategy (`EXPONENTIAL_BACKOFF`).\n5. **Trace the exact connector attempt in Cloud Logging** (when connector logging is enabled on the connection):\n   ```bash\n   gcloud logging read \\\n     'resource.type=\"connectors.googleapis.com\u002FConnection\" AND jsonPayload.integration_connectors_trace_id=\"EXECUTIONID_TASKID_ATTEMPTNUMBER\"' \\\n     --project=\"PROJECT_ID\" --limit=20 --format=json\n   ```\n\n### Scenario B: Stuck in `SUSPENDED` (Approval vs Suspend Task)\n\n1. Identify which task paused the execution from `taskExecutionDetails` and list pending suspensions (read-only; requires `integrations.suspensions.list`, included in `roles\u002Fintegrations.suspensionResolver`) to find `SUSPENSION_ID` and `taskId`. Note that `roles\u002Fintegrations.suspensionResolver` alone cannot read execution records (no `integrations.executions.list` or `integrations.executions.get`, returning HTTP 403 on `executions.list` and v2 `:troubleshoot`); to inspect `taskExecutionDetails` (or run `:troubleshoot`), also grant `roles\u002Fintegrations.integrationViewer` (or use the approval email link, which carries the execution and task IDs). See [Suspensions](references\u002Fsuspensions.md) for full details.\n   ```bash\n   # Read-only: list suspensions on EXECUTION_ID\n   curl -sS -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n     \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\"\n   ```\n2. **Approval task:** can be approved or rejected in the Cloud console UI or via the REST API (`POST https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\u002FSUSPENSION_ID:resolve` with body `{\"suspension\": {\"eventExecutionInfoId\": \"EXECUTION_ID\", \"integration\": \"INTEGRATION_NAME\", \"taskId\": \"TASK_ID\", \"state\": \"LIFTED\"}}` or `\"REJECTED\"`—the same call the console Approve\u002FReject buttons make). Always confirm with the user before resolving. Verify the caller holds `integrations.suspensions.resolve` (included in `roles\u002Fintegrations.suspensionResolver`, which is not included in `roles\u002Fintegrations.integrationEditor`) **and** `roles\u002Fiam.serviceAccountUser` on the integration's `runAsServiceAccount`. Without `roles\u002Fiam.serviceAccountUser`, approving leaves the execution paused until it expires.\n3. **Suspend task:** `:lift` is the Suspend-task variant and requires `integrations.suspensions.lift` (included in `roles\u002Fintegrations.suspensionResolver`) plus `roles\u002Fiam.serviceAccountUser` on the integration's `runAsServiceAccount`. After user confirmation, call `POST ...\u002Fexecutions\u002FEXECUTION_ID\u002Fsuspensions\u002FSUSPENSION_ID:lift` with an optional `suspensionResult`.\n4. **Watch out for version publish cancellation:** publishing a new version of an integration automatically cancels all suspended executions of the previous version.\n\n### Scenario C: HTTP 403 `PERMISSION_DENIED` on Trigger, Test, or Publish\n\nApplication Integration enforces two independent permission checks:\n1. **Caller IAM check:** the caller needs permission on the integration (`roles\u002Fintegrations.integrationInvoker` to invoke, `roles\u002Fintegrations.integrationEditor` to edit\u002Ftest, `roles\u002Fintegrations.integrationDeployer` or `roles\u002Fintegrations.integrationEditor` to publish).\n2. **Run-as service account check:** whenever `runAsServiceAccount` is set on the version, the user (or principal) publishing or testing the version needs `roles\u002Fiam.serviceAccountUser` on `runAsServiceAccount` at publish\u002Ftest time, and at runtime the invoking caller or trigger service account needs `roles\u002Fiam.serviceAccountUser` on `runAsServiceAccount` (granted in the project that owns the service account). In addition, `runAsServiceAccount` needs roles on downstream Google Cloud resources.\n\nCheck both bindings before retrying. For Pub\u002FSub, Eventarc, and Connector Event triggers, a run-as service account (`roles\u002Fiam.serviceAccountUser`) denial records a failed execution and the event is **not** redelivered.\n\n## Recovery Actions (Require User Confirmation)\n\nAlways display the exact command and obtain user confirmation before running `:cancel`, `:replay`, `:lift`, or `:resolve`. `:cancel` requires `integrations.executions.cancel` and `:replay` requires `integrations.executions.replay` (both are held by `roles\u002Fintegrations.integrationInvoker`, `roles\u002Fintegrations.integrationEditor`, and `roles\u002Fintegrations.integrationAdmin`—**not** by `roles\u002Fintegrations.suspensionResolver` or `roles\u002Fintegrations.integrationViewer`, so approvers with only `roles\u002Fintegrations.suspensionResolver` cannot cancel a stuck `SUSPENDED` run). `:replay` also enforces two preconditions that return `FAILED_PRECONDITION`: (a) test-case runs cannot be replayed in any mode (`\"Test runs cannot be replayed.\"`), and (b) `REPLAY_MODE_POINT_OF_FAILURE` is refused for executions that ran with database persistence (local logging \u002F `databasePersistencePolicy: DATABASE_PERSISTENCE_DISABLED`) disabled because it needs execution snapshots—use `REPLAY_MODE_FROM_BEGINNING` in that case. When overriding inputs on `:replay` with `modifiedParameters` and `updateMask`, list every overridden key in `updateMask`; without `updateMask`, all original inputs are discarded and only `modifiedParameters` are used, and a key in `updateMask` missing from `modifiedParameters` is an error.\n\n```bash\n# Cancel an execution in PENDING (console \"On hold\"), RETRY_ON_HOLD, or SUSPENDED state (requires integrations.executions.cancel)\n# (Cannot cancel executions in PROCESSING state [console \"In process\"], completed runs [SUCCEEDED, FAILED, CANCELLED], or Schedule-trigger runs; approvers with only roles\u002Fintegrations.suspensionResolver cannot cancel a stuck SUSPENDED run)\ncurl -X POST \\\n  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"cancelReason\": \"Canceling stuck execution after investigation\"}' \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:cancel\"\n\n# Replay a SUCCEEDED, FAILED, or CANCELLED execution on a published integration (Preview; requires integrations.executions.replay; test-case runs and Schedule-trigger runs cannot be replayed)\n# replayMode: REPLAY_MODE_FROM_BEGINNING or REPLAY_MODE_POINT_OF_FAILURE (failed runs only; refused with FAILED_PRECONDITION if databasePersistencePolicy was DATABASE_PERSISTENCE_DISABLED—use REPLAY_MODE_FROM_BEGINNING in that case)\n# Optional modifiedParameters\u002FupdateMask: list every overridden key in updateMask; without updateMask, all original inputs are discarded and only modifiedParameters are used; a key in updateMask missing from modifiedParameters is an error\ncurl -X POST \\\n  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"replayReason\": \"Replay after fixing downstream permission\", \"replayMode\": \"REPLAY_MODE_POINT_OF_FAILURE\"}' \\\n  \"https:\u002F\u002Fintegrations.googleapis.com\u002Fv1\u002Fprojects\u002FPROJECT_ID\u002Flocations\u002FLOCATION\u002Fintegrations\u002FINTEGRATION_NAME\u002Fexecutions\u002FEXECUTION_ID:replay\"\n```\n\n## Reference Files\n\nLoad the matching reference file when you need deeper details, exact field names, or edge-case checks:\n\n- [Usage Attribution](references\u002Fattribution.md): Usage attribution for `gcloud` CLI commands, direct `curl` REST calls, and Python discovery-client snippets run in the terminal.\n- [Execution States](references\u002Fexecution-states.md): `ExecutionDetails.state` and `TaskExecutionDetails.taskExecutionState` enums, `executions.list` filter operators, large-log workarounds (> 300 MB), and `:cancel` \u002F `:replay` constraints.\n- [Logging and Monitoring](references\u002Flogging-and-monitoring.md): Local logging vs Cloud Logging interactions, enabling Cloud Logging on a draft version, Logs Explorer queries, `integrations.googleapis.com\u002F*` and `connectors.googleapis.com\u002F*` metric schemas, and alerting setup.\n- [Common Errors](references\u002Fcommon-errors.md): HTTP\u002FgRPC error codes, the two-sided 403 checklist, the October 15, 2026 run-as service account requirement, task-level limits and gotchas, and ADK `ApplicationIntegrationToolset` errors.\n- [Connectors Health](references\u002Fconnectors-health.md): `ConnectionStatus.state`, `EventingStatus.state`, `connectionSchemaMetadata` probes, runtime v2 readiness calls, node throttling (HTTP 429), and separating connection auth from caller IAM.\n- [Suspensions](references\u002Fsuspensions.md): Suspend vs Approval vs Timer behavior, `suspensions.list` \u002F `:lift` \u002F `:resolve` payloads, expiration settings, and why approvals stay stuck without `roles\u002Fiam.serviceAccountUser`.\n",{"name":5,"metadata":43,"description":7},{"version":44,"category":45},"1.0.2","ApplicationDevelopment","main","bb9b3f464aae37af360d079c1d863524955b0363","skills\u002Fcloud\u002Fapplication-integration-troubleshooting",2005,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills","https:\u002F\u002Fgithub.com\u002Fgoogle","https:\u002F\u002Fopensource.google\u002F","Google ❤️ Open Source","2026-11-14 01:51:35",{"source":8,"skillName":5,"commands":56,"prompt":66,"zipAvailable":67},[57,60,63],{"tool":58,"command":59},"npx","npx skills add google\u002Fskills --skill application-integration-troubleshooting",{"tool":61,"command":62},"bunx","bunx skills add google\u002Fskills --skill application-integration-troubleshooting",{"tool":64,"command":65},"pnpm","pnpm dlx skills add google\u002Fskills --skill application-integration-troubleshooting","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002FSKILL.md",true,[69,75,79,83,87,91],{"path":70,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":73,"sourceHtmlUrl":74},"references\u002Fattribution.md","REFERENCE","参考文档",5115,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Fattribution.md",{"path":76,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":77,"sourceHtmlUrl":78},"references\u002Fcommon-errors.md",17106,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Fcommon-errors.md",{"path":80,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":81,"sourceHtmlUrl":82},"references\u002Fconnectors-health.md",10619,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Fconnectors-health.md",{"path":84,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":85,"sourceHtmlUrl":86},"references\u002Fexecution-states.md",14724,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Fexecution-states.md",{"path":88,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":89,"sourceHtmlUrl":90},"references\u002Flogging-and-monitoring.md",10098,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Flogging-and-monitoring.md",{"path":92,"resourceType":71,"resourceTypeDesc":72,"sizeBytes":93,"sourceHtmlUrl":94},"references\u002Fsuspensions.md",10815,"https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fapplication-integration-troubleshooting\u002Freferences\u002Fsuspensions.md",[96,132,162,190,217,241],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":110,"publishedAt":38},"01M6998S2EZKD3GEJ7F943RZTE","sign-in-with-google-web","Implement, configure, and secure Sign In With Google (SiwG) using Google Identity Services (GIS \u002F `https:\u002F\u002Faccounts.google.com\u002Fgsi\u002Fclient`) across web architectures. Use when creating Google sign-in buttons, implementing Google One Tap with FedCM, integrating GIS in React\u002FNext.js\u002FAngular\u002FHTML, verifying ID tokens on backend runtimes (Python\u002FNode.js\u002FGo\u002FJava), enforcing Google Workspace domain restrictions (`hd`), securing client-side ID tokens via WebCrypto nonces or encrypted IndexedDB, embedding in cross-origin iframes via the Intermediate Iframe API (`https:\u002F\u002Faccounts.google.com\u002Fgsi\u002Fintermediate`), configuring Content Security Policy (CSP), COOP, and Permissions Policy (`identity-credentials-get`) headers, or handling sign-out and revocation. Don't use for legacy Google Sign-In (`gapi.auth2`), Google Cloud IAM service accounts, or Android\u002FiOS native Credential Manager.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fidentity\u002Fsign-in-with-google-web\u002FSKILL.md",[102,107],{"categoryNo":103,"categoryName":104,"taxonomyType":105,"taxonomyTypeDesc":106},"01JBSEEDCAT000000000000009","前端开发","SCENARIO","使用场景",{"categoryNo":108,"categoryName":109,"taxonomyType":105,"taxonomyTypeDesc":106},"01JBSEEDCAT000000000000010","后端与接口",[9,111,112,113,114,115,116,117,118,29,119,120,121,122,123,27,124,125,126,127,128,129,130,131],"sign in with google web","sign","with","web","android","go","security","google workspace","implement","policy","python","implementing","react","permissions","buttons","secure","manager","identity","architectures","domain","runtimes",{"skillNo":133,"name":134,"nameZh":6,"description":135,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":136,"categories":137,"keywords":147,"publishedAt":38},"01M6998PHPDCAD684VW5WRG83D","dpop-adoption","Implement and debug OAuth 2.0 DPoP (RFC 9449) refresh token sender-constraining for WebCrypto, Node.js ES6, and browser runtimes integrating with Google's OAuth platform. Use when configuring non-extractable asymmetric key pairs (P-256), generating DPoP Proof JWTs for authorization code exchange and token refresh, or handling 400 use_dpop_nonce challenge retry loops at oauth2.googleapis.com\u002Ftoken. Don't use for unconstrained OAuth 2.0 flows (where refresh tokens are not bound to a client key pair), or for Google Cloud IAM \u002F service account authentication.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fidentity\u002Fdpop-adoption\u002FSKILL.md",[138,139,142],{"categoryNo":108,"categoryName":109,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":140,"categoryName":141,"taxonomyType":105,"taxonomyTypeDesc":106},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":143,"categoryName":144,"taxonomyType":145,"taxonomyTypeDesc":146},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[148,149,9,29,150,35,151,152,153,154,27,155,156,157,158,159,160,128,131,161,37],"dpop adoption","adoption","es6","authentication","flows","challenge","code","browser","loops","at","node","token","pair","platform",{"skillNo":163,"name":164,"nameZh":6,"description":165,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":166,"categories":167,"keywords":174,"publishedAt":38},"01M6998M72TC73EEAQH4XC7EXW","retrieving-developer-knowledge","Searches, retrieves, and synthesizes official Google developer documentation across Google Cloud, AI\u002FGemini, Android, Chrome, Web, Flutter, Go, Firebase, and other Google developer platforms. Integrates with the Developer Knowledge MCP server (search_documents, get_documents, answer_query) or the Developer Knowledge REST API fallback. Use when searching for gcloud CLI commands, API syntax, IAM permissions, official documentation, architectural comparisons, or product choice overviews. Don't use for local filesystem lookups or non-Google documentation.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fdevelopers\u002Fretrieving-developer-knowledge\u002FSKILL.md",[168,169,172,173],{"categoryNo":108,"categoryName":109,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":170,"categoryName":171,"taxonomyType":105,"taxonomyTypeDesc":106},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":140,"categoryName":141,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":143,"categoryName":144,"taxonomyType":145,"taxonomyTypeDesc":146},[175,176,177,178,29,179,180,181,182,183,184,185,186,37,187,28,188,27,189,124],"retrieving developer knowledge","knowledge","developer","MCP","mcp server","official","developer documentation","get","search","local","choice","documents","query","across","documentation",{"skillNo":191,"name":192,"nameZh":6,"description":193,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":194,"categories":195,"keywords":196,"publishedAt":38},"01M6998J2CZEXS9BES9S3WWFKC","migrate-to-rust","Clones, analyzes, plans, and migrates legacy codebases (C++, Fortran, Java, Python, C, Shell) from a GitHub URL or local directory to a modern, high-performance, deterministic, 100% Safe Rust architecture, tracking live telemetry across every task bucket, test failure\u002Ffix cycle, and Profile-Guided Optimization (PGO) stage, and producing a first-person AI storytelling report and interactive HTML infographic. Use when migrating a GitHub repository or local codebase to Rust, porting legacy C++, Fortran, Python, or Java workloads to memory-safe Rust, designing Struct-of-Arrays (SoA) or SIMD Rust architectures, or optimizing Rust ports with LLVM PGO. Don't use for unrelated non-Rust refactoring or single-line syntax questions.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fdevelopers\u002Fmigrate-to-rust\u002FSKILL.md",[],[197,198,199,200,201,202,203,204,121,205,206,207,208,209,210,184,211,212,213,37,214,215,188,216],"migrate to rust","rust","migrate","github","memory","performance","directory","safe","ports","llvm","soa","cycle","porting","fix","arrays","infographic","high-performance","URL","architecture","unrelated",{"skillNo":218,"name":219,"nameZh":6,"description":220,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":221,"categories":222,"keywords":226,"publishedAt":38},"01M6998GCC3K29WZ82KXMB9E02","finding-google-skills","Google platform decision and setup guidance, loaded on demand from Google's skill catalog. Use when a developer is choosing or setting up part of their stack, such as where to run a service, a database, storage, messaging, authentication, analytics, ads, or AI model serving, and a Google product is a reasonable candidate - whether or not a vendor is named - or when a request names a Google product or API. Brings in the matching Google skill so the answer can weigh Google options, their trade-offs, and when they are not the right fit. Skip when the stack is already settled on another provider and no Google product is named, or the task involves no platform choice.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fdevelopers\u002Ffinding-google-skills\u002FSKILL.md",[223,224,225],{"categoryNo":108,"categoryName":109,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":140,"categoryName":141,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":143,"categoryName":144,"taxonomyType":145,"taxonomyTypeDesc":146},[9,227,228,34,229,161,230,231,232,233,234,151,235,236,237,185,238,239,37,240,28],"finding google skills","storage","messaging","model","trade-offs","provider","answer","fit","catalog","up","decision","use","database","candidate",{"skillNo":242,"name":243,"nameZh":6,"description":244,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":245,"categories":246,"keywords":249,"publishedAt":38},"01M6998DWNHTH0BS6W1CY8ZXMP","workload-manager-basics","Use this skill to manage Google Cloud Workload Manager evaluations, rules, scanned resources, and validation results by using public client libraries and the REST API. Use when you need to inspect workload best-practice rules, create and run evaluations for Google Cloud general best practices, SAP, SQL Server, or custom organizational rules, review violations, export results to BigQuery, or automate Workload Manager through client libraries because no service-specific public CLI or MCP server is available. Don't use for general Google Compute Engine instance management, VPC configuration, or standard IAM auditing.","https:\u002F\u002Fgithub.com\u002Fgoogle\u002Fskills\u002Fblob\u002Fmain\u002Fskills\u002Fcloud\u002Fworkload-manager-basics\u002FSKILL.md",[247,248],{"categoryNo":108,"categoryName":109,"taxonomyType":105,"taxonomyTypeDesc":106},{"categoryNo":140,"categoryName":141,"taxonomyType":105,"taxonomyTypeDesc":106},[250,251,127,252,9,178,253,29,179,254,255,256,257,37,258,28,27,259,260,261,23],"workload manager basics","workload","basics","cli","review","vpc","resources","inspect","you","create","libraries","server"]