[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M699CSN0BS8MXRR5YKK8DMS0":3,"skill-same:01M699CSN0BS8MXRR5YKK8DMS0":81},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":26,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":46,"repositoryHtmlUrl":47,"repositoryHomepageUrl":48,"ownerHtmlUrl":49,"ownerBlogUrl":50,"ownerDescription":51,"updatedAt":52,"install":53,"resources":66},"01M699CSN0BS8MXRR5YKK8DMS0","deploying-scalable-agents",null,"Take a working agent prototype to a scalable, observable production deployment on Microsoft Foundry. Covers deployment patterns (client-hosted, hosted agents, agent workflows), the agent lifecycle, model routing, response caching, evaluation gates, human-in-the-loop approval, observability with OpenTelemetry, cost optimisation, and smoke-testing deployed agents with the AI Smoke Test action. Based on Lesson 16 of AI Agents for Beginners. USE FOR: deploy an agent to production, scale an agent, Microsoft Foundry hosted agent, Foundry Agent Service, model routing, response caching, evaluation gate, release gate, human approval workflow, agent observability, agent tracing, agent cost optimisation, smoke test a hosted agent, production customer support agent. DO NOT USE FOR: building your first agent (start with Lesson 01), running agents locally on-device (use local-ai-agents \u002F Lesson 17), Azure infrastructure provisioning unrelated to agents, non-Foundry deployment targets.","microsoft\u002Fai-agents-for-beginners","microsoft","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F6154722?v=4","01M698VYDXZ6JW5WH3CRV650R8","Microsoft","01M6998ZK2275KMJE6Y5SEA2X2",79423,"MIT","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fen\u002F.agents\u002Fskills\u002Fdeploying-scalable-agents\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[27,28,29,30,31,32,33,34,9,35,36,37],"deploying scalable agents","testing","ai agents","prototype","workflow","customer","support","test","foundry","microsoft foundry","workflows","2026-11-10 10:12:30",false,"---\nname: deploying-scalable-agents\nlicense: MIT\ndescription: 'Take a working agent prototype to a scalable, observable production\n  deployment on Microsoft Foundry. Covers deployment patterns (client-hosted, hosted\n  agents, agent workflows), the agent lifecycle, model routing, response caching,\n  evaluation gates, human-in-the-loop approval, observability with OpenTelemetry,\n  cost optimisation, and smoke-testing deployed agents with the AI Smoke Test action.\n  Based on Lesson 16 of AI Agents for Beginners. USE FOR: deploy an agent to production,\n  scale an agent, Microsoft Foundry hosted agent, Foundry Agent Service, model routing,\n  response caching, evaluation gate, release gate, human approval workflow, agent\n  observability, agent tracing, agent cost optimisation, smoke test a hosted agent,\n  production customer support agent. DO NOT USE FOR: building your first agent (start\n  with Lesson 01), running agents locally on-device (use local-ai-agents \u002F Lesson\n  17), Azure infrastructure provisioning unrelated to agents, non-Foundry deployment\n  targets.'\n---\n# Deploying Scalable Agents with Microsoft Foundry\n\n> Companion skill for [Lesson 16 – Deploying Scalable Agents](..\u002F..\u002F..\u002F16-deploying-scalable-agents\u002FREADME.md).\n> Use it to help a learner move an agent from prototype to a scalable, observable\n> production deployment. Ground every recommendation in the lesson content and\n> the runnable notebook; do not invent Foundry APIs.\n\n## Triggers\n\nActivate this skill when a learner wants to:\n- Deploy an agent to Microsoft Foundry as a **hosted agent** and make it versioned\u002Fobservable.\n- Choose between **client-hosted, hosted-agent, and agent-workflow** deployment patterns.\n- Add **model routing**, **response caching**, or **bounded concurrency** to control latency and cost.\n- Add an **evaluation gate** so a bad agent version cannot ship.\n- Add a **human-in-the-loop approval** step for high-risk actions.\n- Instrument an agent with **OpenTelemetry** tracing for production observability.\n- **Smoke-test** a deployed agent as a fast post-deploy gate.\n\n## Core mental model\n\nA production agent is mostly the operational skeleton *around* the model (~80%),\nnot the model itself. Map every recommendation to one of these concerns:\n\n| Concern | Prototype → Production |\n|---------|------------------------|\n| Hosting | notebook → versioned hosted service |\n| Identity | your `az login` → managed identity + scoped RBAC |\n| State | in-memory → externalised thread\u002Fmemory store |\n| Failure | traceback → retries, fallbacks, alerts |\n| Cost | \"a few cents\" → tracked, routed, cached, budgeted |\n| Quality | eyeballing → automated evaluation gate |\n| Trust | you approve → policy + human-in-the-loop |\n\n## Deployment patterns (pick one, or combine)\n\n1. **Client-hosted** — the reasoning loop runs in your process. Max control; you own scaling\u002Fstate.\n2. **Hosted agent (Foundry Agent Service)** — Foundry hosts the loop, stores threads, enforces RBAC\u002Fcontent safety, shows the agent in the portal. Less control, far less operational surface.\n3. **Agent workflow** — multiple agents\u002Ftools composed into a graph with branching, approval nodes, and durable checkpoints.\n\n## Lifecycle (the loop that ships an agent)\n\n`create → version → evaluate (gate) → deploy hosted → observe online → collect failures → repeat`.\n**Offline evaluation is a gate, not an afterthought** — a version does not ship\nunless it clears the threshold. Online observability feeds real failures back\ninto the offline test set.\n\n## Scaling and cost levers (in priority order)\n\n1. **Right-size the model** — use the smallest model that passes the evaluation gate.\n2. **Route by complexity** — small\u002Ffast model for simple requests, large model for real reasoning (DIY classifier or Foundry Model Router).\n3. **Cache** — serve near-duplicate requests without a model call.\n4. **Stateless design + bounded concurrency** — externalise state; retry with backoff.\n\n## Key patterns to reproduce\n\nPoint the learner at these from the notebook\n[`16-python-agent-framework.ipynb`](..\u002F..\u002F..\u002F16-deploying-scalable-agents\u002Fcode_samples\u002F16-python-agent-framework.ipynb):\n\n- **Request handler**: cache → route by complexity → trace span → run → cache.\n- **Evaluation gate**: score an offline test set; return `pass_rate >= threshold` and only deploy if true.\n- **Human approval**: `@tool(approval_mode=\"always_require\")` for actions like large refunds.\n- **Tracing**: wrap each request in `tracer.start_as_current_span(...)` and set attributes like `routed.model`, `customer.id`.\n\n## Smoke-testing a deployed agent\n\nAfter deploy, verify the endpoint actually answers (a green deploy can still be\nsilent). Use the [AI Smoke Test](https:\u002F\u002Fgithub.com\u002Fmarketplace\u002Factions\u002Fai-smoke-test)\naction via [`.github\u002Fworkflows\u002Fsmoke-test.yml`](..\u002F..\u002F..\u002F..\u002F..\u002F.github\u002Fworkflows\u002Fsmoke-test.yml)\nwith the catalog in [`tests\u002F`](..\u002F..\u002F..\u002Ftests\u002FREADME.md). The runner POSTs each\nprompt to `POST {project_endpoint}\u002Fagents\u002F{agent_name}\u002Fendpoint\u002Fprotocols\u002Fopenai\u002Fresponses`\nand asserts on the reply text. The identity needs the **Azure AI User** role at\nFoundry project scope; the token audience must be `https:\u002F\u002Fai.azure.com\u002F`.\n\nLayer the gates: **smoke test** (reachable\u002Fresponding, every deploy) → **offline\nevaluation** (good enough to ship, before promotion) → **online evaluation** (how\nis it doing in the wild, continuous).\n\n## Enterprise controls\n\n- **RBAC**: give each hosted agent a managed identity with least privilege.\n- **MCP in production**: treat every MCP server as an untrusted boundary — pin the version, scope its identity, validate outputs, rate-limit, never expose secrets.\n\n## Guardrails for the assistant\n\n- Prefer the canonical `FoundryChatClient(...)` + `provider.as_agent(...)` pattern used across the course.\n- Do not promise live-Azure results you have not verified; recommend the smoke-test workflow to confirm a deployment.\n- Keep evaluation and cost advice tied together: evaluation sets the quality floor, routing\u002Fcaching keep cost near that floor.\n\n---\n\n\u003C!-- CO-OP TRANSLATOR DISCLAIMER START -->\n**Disclaimer**:\nThis document has been translated using AI translation service [Co-op Translator](https:\u002F\u002Fgithub.com\u002FAzure\u002Fco-op-translator). While we strive for accuracy, please be aware that automated translations may contain errors or inaccuracies. The original document in its native language should be considered the authoritative source. For critical information, professional human translation is recommended. We are not liable for any misunderstandings or misinterpretations arising from the use of this translation.\n\u003C!-- CO-OP TRANSLATOR DISCLAIMER END -->","# Deploying Scalable Agents with Microsoft Foundry\n\n> Companion skill for [Lesson 16 – Deploying Scalable Agents](..\u002F..\u002F..\u002F16-deploying-scalable-agents\u002FREADME.md).\n> Use it to help a learner move an agent from prototype to a scalable, observable\n> production deployment. Ground every recommendation in the lesson content and\n> the runnable notebook; do not invent Foundry APIs.\n\n## Triggers\n\nActivate this skill when a learner wants to:\n- Deploy an agent to Microsoft Foundry as a **hosted agent** and make it versioned\u002Fobservable.\n- Choose between **client-hosted, hosted-agent, and agent-workflow** deployment patterns.\n- Add **model routing**, **response caching**, or **bounded concurrency** to control latency and cost.\n- Add an **evaluation gate** so a bad agent version cannot ship.\n- Add a **human-in-the-loop approval** step for high-risk actions.\n- Instrument an agent with **OpenTelemetry** tracing for production observability.\n- **Smoke-test** a deployed agent as a fast post-deploy gate.\n\n## Core mental model\n\nA production agent is mostly the operational skeleton *around* the model (~80%),\nnot the model itself. Map every recommendation to one of these concerns:\n\n| Concern | Prototype → Production |\n|---------|------------------------|\n| Hosting | notebook → versioned hosted service |\n| Identity | your `az login` → managed identity + scoped RBAC |\n| State | in-memory → externalised thread\u002Fmemory store |\n| Failure | traceback → retries, fallbacks, alerts |\n| Cost | \"a few cents\" → tracked, routed, cached, budgeted |\n| Quality | eyeballing → automated evaluation gate |\n| Trust | you approve → policy + human-in-the-loop |\n\n## Deployment patterns (pick one, or combine)\n\n1. **Client-hosted** — the reasoning loop runs in your process. Max control; you own scaling\u002Fstate.\n2. **Hosted agent (Foundry Agent Service)** — Foundry hosts the loop, stores threads, enforces RBAC\u002Fcontent safety, shows the agent in the portal. Less control, far less operational surface.\n3. **Agent workflow** — multiple agents\u002Ftools composed into a graph with branching, approval nodes, and durable checkpoints.\n\n## Lifecycle (the loop that ships an agent)\n\n`create → version → evaluate (gate) → deploy hosted → observe online → collect failures → repeat`.\n**Offline evaluation is a gate, not an afterthought** — a version does not ship\nunless it clears the threshold. Online observability feeds real failures back\ninto the offline test set.\n\n## Scaling and cost levers (in priority order)\n\n1. **Right-size the model** — use the smallest model that passes the evaluation gate.\n2. **Route by complexity** — small\u002Ffast model for simple requests, large model for real reasoning (DIY classifier or Foundry Model Router).\n3. **Cache** — serve near-duplicate requests without a model call.\n4. **Stateless design + bounded concurrency** — externalise state; retry with backoff.\n\n## Key patterns to reproduce\n\nPoint the learner at these from the notebook\n[`16-python-agent-framework.ipynb`](..\u002F..\u002F..\u002F16-deploying-scalable-agents\u002Fcode_samples\u002F16-python-agent-framework.ipynb):\n\n- **Request handler**: cache → route by complexity → trace span → run → cache.\n- **Evaluation gate**: score an offline test set; return `pass_rate >= threshold` and only deploy if true.\n- **Human approval**: `@tool(approval_mode=\"always_require\")` for actions like large refunds.\n- **Tracing**: wrap each request in `tracer.start_as_current_span(...)` and set attributes like `routed.model`, `customer.id`.\n\n## Smoke-testing a deployed agent\n\nAfter deploy, verify the endpoint actually answers (a green deploy can still be\nsilent). Use the [AI Smoke Test](https:\u002F\u002Fgithub.com\u002Fmarketplace\u002Factions\u002Fai-smoke-test)\naction via [`.github\u002Fworkflows\u002Fsmoke-test.yml`](..\u002F..\u002F..\u002F..\u002F..\u002F.github\u002Fworkflows\u002Fsmoke-test.yml)\nwith the catalog in [`tests\u002F`](..\u002F..\u002F..\u002Ftests\u002FREADME.md). The runner POSTs each\nprompt to `POST {project_endpoint}\u002Fagents\u002F{agent_name}\u002Fendpoint\u002Fprotocols\u002Fopenai\u002Fresponses`\nand asserts on the reply text. The identity needs the **Azure AI User** role at\nFoundry project scope; the token audience must be `https:\u002F\u002Fai.azure.com\u002F`.\n\nLayer the gates: **smoke test** (reachable\u002Fresponding, every deploy) → **offline\nevaluation** (good enough to ship, before promotion) → **online evaluation** (how\nis it doing in the wild, continuous).\n\n## Enterprise controls\n\n- **RBAC**: give each hosted agent a managed identity with least privilege.\n- **MCP in production**: treat every MCP server as an untrusted boundary — pin the version, scope its identity, validate outputs, rate-limit, never expose secrets.\n\n## Guardrails for the assistant\n\n- Prefer the canonical `FoundryChatClient(...)` + `provider.as_agent(...)` pattern used across the course.\n- Do not promise live-Azure results you have not verified; recommend the smoke-test workflow to confirm a deployment.\n- Keep evaluation and cost advice tied together: evaluation sets the quality floor, routing\u002Fcaching keep cost near that floor.\n\n---\n\n\u003C!-- CO-OP TRANSLATOR DISCLAIMER START -->\n**Disclaimer**:\nThis document has been translated using AI translation service [Co-op Translator](https:\u002F\u002Fgithub.com\u002FAzure\u002Fco-op-translator). While we strive for accuracy, please be aware that automated translations may contain errors or inaccuracies. The original document in its native language should be considered the authoritative source. For critical information, professional human translation is recommended. We are not liable for any misunderstandings or misinterpretations arising from the use of this translation.\n\u003C!-- CO-OP TRANSLATOR DISCLAIMER END -->",{"name":5,"license":15,"description":7},"main","f09e94ffe170f8af6d3db7c42987bbe4db227723","translations\u002Fen\u002F.agents\u002Fskills\u002Fdeploying-scalable-agents",26019,"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners","https:\u002F\u002Faka.ms\u002Fai-agents-beginners","https:\u002F\u002Fgithub.com\u002Fmicrosoft","https:\u002F\u002Fopensource.microsoft.com","Open source projects and samples from Microsoft","2026-11-07 12:51:12",{"source":8,"skillName":5,"commands":54,"prompt":64,"zipAvailable":65},[55,58,61],{"tool":56,"command":57},"npx","npx skills add microsoft\u002Fai-agents-for-beginners --skill deploying-scalable-agents",{"tool":59,"command":60},"bunx","bunx skills add microsoft\u002Fai-agents-for-beginners --skill deploying-scalable-agents",{"tool":62,"command":63},"pnpm","pnpm dlx skills add microsoft\u002Fai-agents-for-beginners --skill deploying-scalable-agents","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fen\u002F.agents\u002Fskills\u002Fdeploying-scalable-agents\u002FSKILL.md",true,[67,73,77],{"path":68,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002FCALLOUTS.md","REFERENCE","参考文档",1238,"https:\u002F\u002Fgithub.com\u002Faj-gman\u002Fcyberpunk\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fobsidian-markdown\u002Freferences\u002FCALLOUTS.md",{"path":74,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002FEMBEDS.md",861,"https:\u002F\u002Fgithub.com\u002Faj-gman\u002Fcyberpunk\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fobsidian-markdown\u002Freferences\u002FEMBEDS.md",{"path":78,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":79,"sourceHtmlUrl":80},"references\u002FPROPERTIES.md",1149,"https:\u002F\u002Fgithub.com\u002Faj-gman\u002Fcyberpunk\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fobsidian-markdown\u002Freferences\u002FPROPERTIES.md",[82,95,104,111,119,124],{"skillNo":83,"name":84,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":85,"categories":86,"keywords":93,"publishedAt":38},"01M69A206TRZ8KK0CW0HYJF5V5","testing-course-samples","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Ftesting-course-samples\u002FSKILL.md",[87,88],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":89,"categoryName":90,"taxonomyType":91,"taxonomyTypeDesc":92},"01JBSEEDCAT000000000000039","教师与教育","PROFESSION","适用职业",[28,94],"testing course samples",{"skillNo":96,"name":97,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":98,"categories":99,"keywords":101,"publishedAt":38},"01M69A1XRB2SFHYRMEH0K08EKZ","microsoft-docs","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Fmicrosoft-docs\u002FSKILL.md",[100],{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},[102,9,103],"docs","microsoft docs",{"skillNo":105,"name":106,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":38},"01M69A1VPZFJQ1HNGTZQ5P3WSC","local-ai-agents","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Flocal-ai-agents\u002FSKILL.md",[],[29,110],"local ai agents",{"skillNo":112,"name":113,"nameZh":6,"description":114,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":115,"categories":116,"keywords":117,"publishedAt":38},"01M69A1S947N7MM6V9JF85S2E7","jupyter-notebook","當使用者要求建立、搭建或編輯 Jupyter 筆記本（`.ipynb`）用於實驗、探索或教學時使用；優先使用隨附範本並執行輔助腳本 `new_notebook.py` 來產生一個乾淨的起始筆記本。","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Fjupyter-notebook\u002FSKILL.md",[],[118],"jupyter notebook",{"skillNo":120,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":123,"publishedAt":38},"01M69A1Q8EXPWKZSQK69V6WF22","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Fdeploying-scalable-agents\u002FSKILL.md",[],[27],{"skillNo":125,"name":126,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":129,"publishedAt":38},"01M69A1MHZTH3RH9N8VYSYG74X","azure-openai-to-responses","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fai-agents-for-beginners\u002Fblob\u002Fmain\u002Ftranslations\u002Fzh-TW\u002F.agents\u002Fskills\u002Fazure-openai-to-responses\u002FSKILL.md",[],[130],"azure openai to responses"]