[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6KBAHNR9GCV7C7JT6MC9XCP":3,"skill-same:01M6KBAHNR9GCV7C7JT6MC9XCP":102},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":26,"publishedAt":47,"linkOnly":48,"markdownContent":49,"markdownBody":50,"frontMatter":51,"refName":52,"commitSha":53,"skillDirectory":54,"forksCount":55,"repositoryHtmlUrl":56,"repositoryHomepageUrl":6,"ownerHtmlUrl":57,"ownerBlogUrl":58,"ownerDescription":6,"updatedAt":59,"install":60,"resources":73},"01M6KBAHNR9GCV7C7JT6MC9XCP","browser-automation",null,"Use when the user asks to automate browser tasks, scrape websites, fill forms, capture screenshots, extract structured data from web pages, or build web automation workflows. NOT for testing — use playwright-pro for that.","afaizalam2003\u002FMaster-claude-skill","afaizalam2003","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F222019359?v=4","01M6K8F2XNT0HKBPF00JQ78SV3","Faiz Alam","01M6K8F31EBXAPER0EG8R9HTR5",3,"MIT","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000008","测试与质量","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46],"browser automation","automation","browser","data","fill","for","web automation","automation workflows","pages","forms","use","automate","scrape","testing","tasks","workflows","web","playwright","structured data","screenshots","2026-11-10 10:12:30",false,"---\nname: \"browser-automation\"\ndescription: \"Use when the user asks to automate browser tasks, scrape websites, fill forms, capture screenshots, extract structured data from web pages, or build web automation workflows. NOT for testing — use playwright-pro for that.\"\n---\n\n# Browser Automation - POWERFUL\n\n## Overview\n\nThe Browser Automation skill provides comprehensive tools and knowledge for building production-grade web automation workflows using Playwright. This skill covers data extraction, form filling, screenshot capture, session management, and anti-detection patterns for reliable browser automation at scale.\n\n**When to use this skill:**\n- Scraping structured data from websites (tables, listings, search results)\n- Automating multi-step browser workflows (login, fill forms, download files)\n- Capturing screenshots or PDFs of web pages\n- Extracting data from SPAs and JavaScript-heavy sites\n- Building repeatable browser-based data pipelines\n\n**When NOT to use this skill:**\n- Writing browser tests or E2E test suites — use **playwright-pro** instead\n- Testing API endpoints — use **api-test-suite-builder** instead\n- Load testing or performance benchmarking — use **performance-profiler** instead\n\n**Why Playwright over Selenium or Puppeteer:**\n- **Auto-wait built in** — no explicit `sleep()` or `waitForElement()` needed for most actions\n- **Multi-browser from one API** — Chromium, Firefox, WebKit with zero config changes\n- **Network interception** — block ads, mock responses, capture API calls natively\n- **Browser contexts** — isolated sessions without spinning up new browser instances\n- **Codegen** — `playwright codegen` records your actions and generates scripts\n- **Async-first** — Python async\u002Fawait for high-throughput scraping\n\n## Core Competencies\n\n### 1. Web Scraping Patterns\n\n**Selector priority (most to least reliable):**\n1. `data-testid`, `data-id`, or custom data attributes — stable across redesigns\n2. `#id` selectors — unique but may change between deploys\n3. Semantic selectors: `article`, `nav`, `main`, `section` — resilient to CSS changes\n4. Class-based: `.product-card`, `.price` — brittle if classes are generated (e.g., CSS modules)\n5. Positional: `nth-child()`, `nth-of-type()` — last resort, breaks on layout changes\n\nUse XPath only when CSS cannot express the relationship (e.g., ancestor traversal, text-based selection).\n\n**Pagination strategies:** next-button, URL-based (`?page=N`), infinite scroll, load-more button. See [data_extraction_recipes.md](references\u002Fdata_extraction_recipes.md) for complete pagination handlers and scroll patterns.\n\n### 2. Form Filling & Multi-Step Workflows\n\nBreak multi-step forms into discrete functions per step. Each function fills fields, clicks \"Next\"\u002F\"Continue\", and waits for the next step to load (URL change or DOM element).\n\nKey patterns: login flows, multi-page forms, file uploads (including drag-and-drop zones), native and custom dropdown handling. See [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for complete API reference on `fill()`, `select_option()`, `set_input_files()`, and `expect_file_chooser()`.\n\n### 3. Screenshot & PDF Capture\n\n- **Full page:** `await page.screenshot(path=\"full.png\", full_page=True)`\n- **Element:** `await page.locator(\"div.chart\").screenshot(path=\"chart.png\")`\n- **PDF (Chromium only):** `await page.pdf(path=\"out.pdf\", format=\"A4\", print_background=True)`\n- **Visual regression:** Take screenshots at known states, store baselines in version control with naming: `{page}_{viewport}_{state}.png`\n\nSee [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for full screenshot\u002FPDF options.\n\n### 4. Structured Data Extraction\n\nCore extraction patterns:\n- **Tables to JSON** — Extract `\u003Cthead>` headers and `\u003Ctbody>` rows into dictionaries\n- **Listings to arrays** — Map repeating card elements using a field-selector map (supports `::attr()` for attributes)\n- **Nested\u002Fthreaded data** — Recursive extraction for comments with replies, category trees\n\nSee [data_extraction_recipes.md](references\u002Fdata_extraction_recipes.md) for complete extraction functions, price parsing, data cleaning utilities, and output format helpers (JSON, CSV, JSONL).\n\n### 5. Cookie & Session Management\n\n- **Save\u002Frestore cookies:** `context.cookies()` and `context.add_cookies()`\n- **Full storage state** (cookies + localStorage): `context.storage_state(path=\"state.json\")` to save, `browser.new_context(storage_state=\"state.json\")` to restore\n\n**Best practice:** Save state after login, reuse across scraping sessions. Check session validity before starting a long job — make a lightweight request to a protected page and verify you are not redirected to login. See [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for cookie and storage state API details.\n\n### 6. Anti-Detection Patterns\n\nModern websites detect automation through multiple vectors. Apply these in priority order:\n\n1. **WebDriver flag removal** — Remove `navigator.webdriver = true` via init script (critical)\n2. **Custom user agent** — Rotate through real browser UAs; never use the default headless UA\n3. **Realistic viewport** — Set 1920x1080 or similar real-world dimensions (default 800x600 is a red flag)\n4. **Request throttling** — Add `random.uniform()` delays between actions\n5. **Proxy support** — Per-browser or per-context proxy configuration\n\nSee [anti_detection_patterns.md](references\u002Fanti_detection_patterns.md) for the complete stealth stack: navigator property hardening, WebGL\u002Fcanvas fingerprint evasion, behavioral simulation (mouse movement, typing speed, scroll patterns), proxy rotation strategies, and detection self-test URLs.\n\n### 7. Dynamic Content Handling\n\n- **SPA rendering:** Wait for content selectors (`wait_for_selector`), not the page load event\n- **AJAX\u002FFetch waiting:** Use `page.expect_response(\"**\u002Fapi\u002Fdata*\")` to intercept and wait for specific API calls\n- **Shadow DOM:** Playwright pierces open Shadow DOM with `>>` operator: `page.locator(\"custom-element >> .inner-class\")`\n- **Lazy-loaded images:** Scroll elements into view with `scroll_into_view_if_needed()` to trigger loading\n\nSee [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for wait strategies, network interception, and Shadow DOM details.\n\n### 8. Error Handling & Retry Logic\n\n- **Retry with backoff:** Wrap page interactions in retry logic with exponential backoff (e.g., 1s, 2s, 4s)\n- **Fallback selectors:** On `TimeoutError`, try alternative selectors before failing\n- **Error-state screenshots:** Capture `page.screenshot(path=\"error-state.png\")` on unexpected failures for debugging\n- **Rate limit detection:** Check for HTTP 429 responses and respect `Retry-After` headers\n\nSee [anti_detection_patterns.md](references\u002Fanti_detection_patterns.md) for the complete exponential backoff implementation and rate limiter class.\n\n## Workflows\n\n### Workflow 1: Single-Page Data Extraction\n\n**Scenario:** Extract product data from a single page with JavaScript-rendered content.\n\n**Steps:**\n1. Launch browser in headed mode during development (`headless=False`), switch to headless for production\n2. Navigate to URL and wait for content selector\n3. Extract data using `query_selector_all` with field mapping\n4. Validate extracted data (check for nulls, expected types)\n5. Output as JSON\n\n```python\nasync def extract_single_page(url, selectors):\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        context = await browser.new_context(\n            viewport={\"width\": 1920, \"height\": 1080},\n            user_agent=\"Mozilla\u002F5.0 ...\"\n        )\n        page = await context.new_page()\n        await page.goto(url, wait_until=\"networkidle\")\n        data = await extract_listings(page, selectors[\"container\"], selectors[\"fields\"])\n        await browser.close()\n    return data\n```\n\n### Workflow 2: Multi-Page Scraping with Pagination\n\n**Scenario:** Scrape search results across 50+ pages.\n\n**Steps:**\n1. Launch browser with anti-detection settings\n2. Navigate to first page\n3. Extract data from current page\n4. Check if \"Next\" button exists and is enabled\n5. Click next, wait for new content to load (not just navigation)\n6. Repeat until no next page or max pages reached\n7. Deduplicate results by unique key\n8. Write output incrementally (don't hold everything in memory)\n\n```python\nasync def scrape_paginated(base_url, selectors, max_pages=100):\n    all_data = []\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        page = await (await browser.new_context()).new_page()\n        await page.goto(base_url)\n\n        for page_num in range(max_pages):\n            items = await extract_listings(page, selectors[\"container\"], selectors[\"fields\"])\n            all_data.extend(items)\n\n            next_btn = page.locator(selectors[\"next_button\"])\n            if await next_btn.count() == 0 or await next_btn.is_disabled():\n                break\n\n            await next_btn.click()\n            await page.wait_for_selector(selectors[\"container\"])\n            await human_delay(800, 2000)\n\n        await browser.close()\n    return all_data\n```\n\n### Workflow 3: Authenticated Workflow Automation\n\n**Scenario:** Log into a portal, navigate a multi-step form, download a report.\n\n**Steps:**\n1. Check for existing session state file\n2. If no session, perform login and save state\n3. Navigate to target page using saved session\n4. Fill multi-step form with provided data\n5. Wait for download to trigger\n6. Save downloaded file to target directory\n\n```python\nasync def authenticated_workflow(credentials, form_data, download_dir):\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        state_file = \"session_state.json\"\n\n        # Restore or create session\n        if os.path.exists(state_file):\n            context = await browser.new_context(storage_state=state_file)\n        else:\n            context = await browser.new_context()\n            page = await context.new_page()\n            await login(page, credentials[\"url\"], credentials[\"user\"], credentials[\"pass\"])\n            await context.storage_state(path=state_file)\n\n        page = await context.new_page()\n        await page.goto(form_data[\"target_url\"])\n\n        # Fill form steps\n        for step_fn in [fill_step_1, fill_step_2]:\n            await step_fn(page, form_data)\n\n        # Handle download\n        async with page.expect_download() as dl_info:\n            await page.click(\"button:has-text('Download Report')\")\n        download = await dl_info.value\n        await download.save_as(os.path.join(download_dir, download.suggested_filename))\n\n        await browser.close()\n```\n\n## Tools Reference\n\n| Script | Purpose | Key Flags | Output |\n|--------|---------|-----------|--------|\n| `scraping_toolkit.py` | Generate Playwright scraping script skeleton | `--url`, `--selectors`, `--paginate`, `--output` | Python script or JSON config |\n| `form_automation_builder.py` | Generate form-fill automation script from field spec | `--fields`, `--url`, `--output` | Python automation script |\n| `anti_detection_checker.py` | Audit a Playwright script for detection vectors | `--file`, `--verbose` | Risk report with score |\n\nAll scripts are stdlib-only. Run `python3 \u003Cscript> --help` for full usage.\n\n## Anti-Patterns\n\n### Hardcoded Waits\n**Bad:** `await page.wait_for_timeout(5000)` before every action.\n**Good:** Use `wait_for_selector`, `wait_for_url`, `expect_response`, or `wait_for_load_state`. Hardcoded waits are flaky and slow.\n\n### No Error Recovery\n**Bad:** Linear script that crashes on first failure.\n**Good:** Wrap each page interaction in try\u002Fexcept. Take error-state screenshots. Implement retry with exponential backoff.\n\n### Ignoring robots.txt\n**Bad:** Scraping without checking robots.txt directives.\n**Good:** Fetch and parse robots.txt before scraping. Respect `Crawl-delay`. Skip disallowed paths. Add your bot name to User-Agent if running at scale.\n\n### Storing Credentials in Scripts\n**Bad:** Hardcoding usernames and passwords in Python files.\n**Good:** Use environment variables, `.env` files (gitignored), or a secrets manager. Pass credentials via CLI arguments.\n\n### No Rate Limiting\n**Bad:** Hammering a site with 100 requests\u002Fsecond.\n**Good:** Add random delays between requests (1-3s for polite scraping). Monitor for 429 responses. Implement exponential backoff.\n\n### Selector Fragility\n**Bad:** Relying on auto-generated class names (`.css-1a2b3c`) or deep nesting (`div > div > div > span:nth-child(3)`).\n**Good:** Use data attributes, semantic HTML, or text-based locators. Test selectors in browser DevTools first.\n\n### Not Cleaning Up Browser Instances\n**Bad:** Launching browsers without closing them, leading to resource leaks.\n**Good:** Always use `try\u002Ffinally` or async context managers to ensure `browser.close()` is called.\n\n### Running Headed in Production\n**Bad:** Using `headless=False` in production\u002FCI.\n**Good:** Develop with headed mode for debugging, deploy with `headless=True`. Use environment variable to toggle: `headless = os.environ.get(\"HEADLESS\", \"true\") == \"true\"`.\n\n## Cross-References\n\n- **playwright-pro** — Browser testing skill. Use for E2E tests, test assertions, test fixtures. Browser Automation is for data extraction and workflow automation, not testing.\n- **api-test-suite-builder** — When the website has a public API, hit the API directly instead of scraping the rendered page. Faster, more reliable, less detectable.\n- **performance-profiler** — If your automation scripts are slow, profile the bottlenecks before adding concurrency.\n- **env-secrets-manager** — For securely managing credentials used in authenticated automation workflows.\n","# Browser Automation - POWERFUL\n\n## Overview\n\nThe Browser Automation skill provides comprehensive tools and knowledge for building production-grade web automation workflows using Playwright. This skill covers data extraction, form filling, screenshot capture, session management, and anti-detection patterns for reliable browser automation at scale.\n\n**When to use this skill:**\n- Scraping structured data from websites (tables, listings, search results)\n- Automating multi-step browser workflows (login, fill forms, download files)\n- Capturing screenshots or PDFs of web pages\n- Extracting data from SPAs and JavaScript-heavy sites\n- Building repeatable browser-based data pipelines\n\n**When NOT to use this skill:**\n- Writing browser tests or E2E test suites — use **playwright-pro** instead\n- Testing API endpoints — use **api-test-suite-builder** instead\n- Load testing or performance benchmarking — use **performance-profiler** instead\n\n**Why Playwright over Selenium or Puppeteer:**\n- **Auto-wait built in** — no explicit `sleep()` or `waitForElement()` needed for most actions\n- **Multi-browser from one API** — Chromium, Firefox, WebKit with zero config changes\n- **Network interception** — block ads, mock responses, capture API calls natively\n- **Browser contexts** — isolated sessions without spinning up new browser instances\n- **Codegen** — `playwright codegen` records your actions and generates scripts\n- **Async-first** — Python async\u002Fawait for high-throughput scraping\n\n## Core Competencies\n\n### 1. Web Scraping Patterns\n\n**Selector priority (most to least reliable):**\n1. `data-testid`, `data-id`, or custom data attributes — stable across redesigns\n2. `#id` selectors — unique but may change between deploys\n3. Semantic selectors: `article`, `nav`, `main`, `section` — resilient to CSS changes\n4. Class-based: `.product-card`, `.price` — brittle if classes are generated (e.g., CSS modules)\n5. Positional: `nth-child()`, `nth-of-type()` — last resort, breaks on layout changes\n\nUse XPath only when CSS cannot express the relationship (e.g., ancestor traversal, text-based selection).\n\n**Pagination strategies:** next-button, URL-based (`?page=N`), infinite scroll, load-more button. See [data_extraction_recipes.md](references\u002Fdata_extraction_recipes.md) for complete pagination handlers and scroll patterns.\n\n### 2. Form Filling & Multi-Step Workflows\n\nBreak multi-step forms into discrete functions per step. Each function fills fields, clicks \"Next\"\u002F\"Continue\", and waits for the next step to load (URL change or DOM element).\n\nKey patterns: login flows, multi-page forms, file uploads (including drag-and-drop zones), native and custom dropdown handling. See [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for complete API reference on `fill()`, `select_option()`, `set_input_files()`, and `expect_file_chooser()`.\n\n### 3. Screenshot & PDF Capture\n\n- **Full page:** `await page.screenshot(path=\"full.png\", full_page=True)`\n- **Element:** `await page.locator(\"div.chart\").screenshot(path=\"chart.png\")`\n- **PDF (Chromium only):** `await page.pdf(path=\"out.pdf\", format=\"A4\", print_background=True)`\n- **Visual regression:** Take screenshots at known states, store baselines in version control with naming: `{page}_{viewport}_{state}.png`\n\nSee [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for full screenshot\u002FPDF options.\n\n### 4. Structured Data Extraction\n\nCore extraction patterns:\n- **Tables to JSON** — Extract `\u003Cthead>` headers and `\u003Ctbody>` rows into dictionaries\n- **Listings to arrays** — Map repeating card elements using a field-selector map (supports `::attr()` for attributes)\n- **Nested\u002Fthreaded data** — Recursive extraction for comments with replies, category trees\n\nSee [data_extraction_recipes.md](references\u002Fdata_extraction_recipes.md) for complete extraction functions, price parsing, data cleaning utilities, and output format helpers (JSON, CSV, JSONL).\n\n### 5. Cookie & Session Management\n\n- **Save\u002Frestore cookies:** `context.cookies()` and `context.add_cookies()`\n- **Full storage state** (cookies + localStorage): `context.storage_state(path=\"state.json\")` to save, `browser.new_context(storage_state=\"state.json\")` to restore\n\n**Best practice:** Save state after login, reuse across scraping sessions. Check session validity before starting a long job — make a lightweight request to a protected page and verify you are not redirected to login. See [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for cookie and storage state API details.\n\n### 6. Anti-Detection Patterns\n\nModern websites detect automation through multiple vectors. Apply these in priority order:\n\n1. **WebDriver flag removal** — Remove `navigator.webdriver = true` via init script (critical)\n2. **Custom user agent** — Rotate through real browser UAs; never use the default headless UA\n3. **Realistic viewport** — Set 1920x1080 or similar real-world dimensions (default 800x600 is a red flag)\n4. **Request throttling** — Add `random.uniform()` delays between actions\n5. **Proxy support** — Per-browser or per-context proxy configuration\n\nSee [anti_detection_patterns.md](references\u002Fanti_detection_patterns.md) for the complete stealth stack: navigator property hardening, WebGL\u002Fcanvas fingerprint evasion, behavioral simulation (mouse movement, typing speed, scroll patterns), proxy rotation strategies, and detection self-test URLs.\n\n### 7. Dynamic Content Handling\n\n- **SPA rendering:** Wait for content selectors (`wait_for_selector`), not the page load event\n- **AJAX\u002FFetch waiting:** Use `page.expect_response(\"**\u002Fapi\u002Fdata*\")` to intercept and wait for specific API calls\n- **Shadow DOM:** Playwright pierces open Shadow DOM with `>>` operator: `page.locator(\"custom-element >> .inner-class\")`\n- **Lazy-loaded images:** Scroll elements into view with `scroll_into_view_if_needed()` to trigger loading\n\nSee [playwright_browser_api.md](references\u002Fplaywright_browser_api.md) for wait strategies, network interception, and Shadow DOM details.\n\n### 8. Error Handling & Retry Logic\n\n- **Retry with backoff:** Wrap page interactions in retry logic with exponential backoff (e.g., 1s, 2s, 4s)\n- **Fallback selectors:** On `TimeoutError`, try alternative selectors before failing\n- **Error-state screenshots:** Capture `page.screenshot(path=\"error-state.png\")` on unexpected failures for debugging\n- **Rate limit detection:** Check for HTTP 429 responses and respect `Retry-After` headers\n\nSee [anti_detection_patterns.md](references\u002Fanti_detection_patterns.md) for the complete exponential backoff implementation and rate limiter class.\n\n## Workflows\n\n### Workflow 1: Single-Page Data Extraction\n\n**Scenario:** Extract product data from a single page with JavaScript-rendered content.\n\n**Steps:**\n1. Launch browser in headed mode during development (`headless=False`), switch to headless for production\n2. Navigate to URL and wait for content selector\n3. Extract data using `query_selector_all` with field mapping\n4. Validate extracted data (check for nulls, expected types)\n5. Output as JSON\n\n```python\nasync def extract_single_page(url, selectors):\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        context = await browser.new_context(\n            viewport={\"width\": 1920, \"height\": 1080},\n            user_agent=\"Mozilla\u002F5.0 ...\"\n        )\n        page = await context.new_page()\n        await page.goto(url, wait_until=\"networkidle\")\n        data = await extract_listings(page, selectors[\"container\"], selectors[\"fields\"])\n        await browser.close()\n    return data\n```\n\n### Workflow 2: Multi-Page Scraping with Pagination\n\n**Scenario:** Scrape search results across 50+ pages.\n\n**Steps:**\n1. Launch browser with anti-detection settings\n2. Navigate to first page\n3. Extract data from current page\n4. Check if \"Next\" button exists and is enabled\n5. Click next, wait for new content to load (not just navigation)\n6. Repeat until no next page or max pages reached\n7. Deduplicate results by unique key\n8. Write output incrementally (don't hold everything in memory)\n\n```python\nasync def scrape_paginated(base_url, selectors, max_pages=100):\n    all_data = []\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        page = await (await browser.new_context()).new_page()\n        await page.goto(base_url)\n\n        for page_num in range(max_pages):\n            items = await extract_listings(page, selectors[\"container\"], selectors[\"fields\"])\n            all_data.extend(items)\n\n            next_btn = page.locator(selectors[\"next_button\"])\n            if await next_btn.count() == 0 or await next_btn.is_disabled():\n                break\n\n            await next_btn.click()\n            await page.wait_for_selector(selectors[\"container\"])\n            await human_delay(800, 2000)\n\n        await browser.close()\n    return all_data\n```\n\n### Workflow 3: Authenticated Workflow Automation\n\n**Scenario:** Log into a portal, navigate a multi-step form, download a report.\n\n**Steps:**\n1. Check for existing session state file\n2. If no session, perform login and save state\n3. Navigate to target page using saved session\n4. Fill multi-step form with provided data\n5. Wait for download to trigger\n6. Save downloaded file to target directory\n\n```python\nasync def authenticated_workflow(credentials, form_data, download_dir):\n    async with async_playwright() as p:\n        browser = await p.chromium.launch(headless=True)\n        state_file = \"session_state.json\"\n\n        # Restore or create session\n        if os.path.exists(state_file):\n            context = await browser.new_context(storage_state=state_file)\n        else:\n            context = await browser.new_context()\n            page = await context.new_page()\n            await login(page, credentials[\"url\"], credentials[\"user\"], credentials[\"pass\"])\n            await context.storage_state(path=state_file)\n\n        page = await context.new_page()\n        await page.goto(form_data[\"target_url\"])\n\n        # Fill form steps\n        for step_fn in [fill_step_1, fill_step_2]:\n            await step_fn(page, form_data)\n\n        # Handle download\n        async with page.expect_download() as dl_info:\n            await page.click(\"button:has-text('Download Report')\")\n        download = await dl_info.value\n        await download.save_as(os.path.join(download_dir, download.suggested_filename))\n\n        await browser.close()\n```\n\n## Tools Reference\n\n| Script | Purpose | Key Flags | Output |\n|--------|---------|-----------|--------|\n| `scraping_toolkit.py` | Generate Playwright scraping script skeleton | `--url`, `--selectors`, `--paginate`, `--output` | Python script or JSON config |\n| `form_automation_builder.py` | Generate form-fill automation script from field spec | `--fields`, `--url`, `--output` | Python automation script |\n| `anti_detection_checker.py` | Audit a Playwright script for detection vectors | `--file`, `--verbose` | Risk report with score |\n\nAll scripts are stdlib-only. Run `python3 \u003Cscript> --help` for full usage.\n\n## Anti-Patterns\n\n### Hardcoded Waits\n**Bad:** `await page.wait_for_timeout(5000)` before every action.\n**Good:** Use `wait_for_selector`, `wait_for_url`, `expect_response`, or `wait_for_load_state`. Hardcoded waits are flaky and slow.\n\n### No Error Recovery\n**Bad:** Linear script that crashes on first failure.\n**Good:** Wrap each page interaction in try\u002Fexcept. Take error-state screenshots. Implement retry with exponential backoff.\n\n### Ignoring robots.txt\n**Bad:** Scraping without checking robots.txt directives.\n**Good:** Fetch and parse robots.txt before scraping. Respect `Crawl-delay`. Skip disallowed paths. Add your bot name to User-Agent if running at scale.\n\n### Storing Credentials in Scripts\n**Bad:** Hardcoding usernames and passwords in Python files.\n**Good:** Use environment variables, `.env` files (gitignored), or a secrets manager. Pass credentials via CLI arguments.\n\n### No Rate Limiting\n**Bad:** Hammering a site with 100 requests\u002Fsecond.\n**Good:** Add random delays between requests (1-3s for polite scraping). Monitor for 429 responses. Implement exponential backoff.\n\n### Selector Fragility\n**Bad:** Relying on auto-generated class names (`.css-1a2b3c`) or deep nesting (`div > div > div > span:nth-child(3)`).\n**Good:** Use data attributes, semantic HTML, or text-based locators. Test selectors in browser DevTools first.\n\n### Not Cleaning Up Browser Instances\n**Bad:** Launching browsers without closing them, leading to resource leaks.\n**Good:** Always use `try\u002Ffinally` or async context managers to ensure `browser.close()` is called.\n\n### Running Headed in Production\n**Bad:** Using `headless=False` in production\u002FCI.\n**Good:** Develop with headed mode for debugging, deploy with `headless=True`. Use environment variable to toggle: `headless = os.environ.get(\"HEADLESS\", \"true\") == \"true\"`.\n\n## Cross-References\n\n- **playwright-pro** — Browser testing skill. Use for E2E tests, test assertions, test fixtures. Browser Automation is for data extraction and workflow automation, not testing.\n- **api-test-suite-builder** — When the website has a public API, hit the API directly instead of scraping the rendered page. Faster, more reliable, less detectable.\n- **performance-profiler** — If your automation scripts are slow, profile the bottlenecks before adding concurrency.\n- **env-secrets-manager** — For securely managing credentials used in authenticated automation workflows.\n",{"name":5,"description":7},"main","0b48cbb0e6d5be6ff68c0f6b28bc0c65ee21c4a1","engineering\u002Fbrowser-automation",2,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill","https:\u002F\u002Fgithub.com\u002Fafaizalam2003","https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fmdfaizalam2003\u002F","2026-04-01 04:35:43",{"source":8,"skillName":5,"commands":61,"prompt":71,"zipAvailable":72},[62,65,68],{"tool":63,"command":64},"npx","npx skills add afaizalam2003\u002FMaster-claude-skill --skill browser-automation",{"tool":66,"command":67},"bunx","bunx skills add afaizalam2003\u002FMaster-claude-skill --skill browser-automation",{"tool":69,"command":70},"pnpm","pnpm dlx skills add afaizalam2003\u002FMaster-claude-skill --skill browser-automation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002FSKILL.md",true,[74,80,84,88,94,98],{"path":75,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":78,"sourceHtmlUrl":79},"references\u002Fanti_detection_patterns.md","REFERENCE","参考文档",13964,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Freferences\u002Fanti_detection_patterns.md",{"path":81,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fdata_extraction_recipes.md",19037,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Freferences\u002Fdata_extraction_recipes.md",{"path":85,"resourceType":76,"resourceTypeDesc":77,"sizeBytes":86,"sourceHtmlUrl":87},"references\u002Fplaywright_browser_api.md",13232,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Freferences\u002Fplaywright_browser_api.md",{"path":89,"resourceType":90,"resourceTypeDesc":91,"sizeBytes":92,"sourceHtmlUrl":93},"scripts\u002Fanti_detection_checker.py","SCRIPT","脚本",22037,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Fscripts\u002Fanti_detection_checker.py",{"path":95,"resourceType":90,"resourceTypeDesc":91,"sizeBytes":96,"sourceHtmlUrl":97},"scripts\u002Fform_automation_builder.py",10127,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Fscripts\u002Fform_automation_builder.py",{"path":99,"resourceType":90,"resourceTypeDesc":91,"sizeBytes":100,"sourceHtmlUrl":101},"scripts\u002Fscraping_toolkit.py",7800,"https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fengineering\u002Fbrowser-automation\u002Fscripts\u002Fscraping_toolkit.py",[103,133,158,183,212,232],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":114,"publishedAt":47},"01M6KC4E8MYR9DBAZTAXHKYMMH","soc2-compliance","Use when the user asks to prepare for SOC 2 audits, map Trust Service Criteria, build control matrices, collect audit evidence, perform gap analysis, or assess SOC 2 Type I vs Type II readiness.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fsoc2-compliance\u002FSKILL.md",[109],{"categoryNo":110,"categoryName":111,"taxonomyType":112,"taxonomyTypeDesc":113},"01JBSEEDCAT000000000000037","律师与法务","PROFESSION","适用职业",[115,116,117,118,119,120,121,122,37,123,124,125,126,127,128,129,130,131,132,32],"compliance","soc2","soc2 compliance","audit evidence","audit","collect","soc","analysis","evidence","trust","build","map","control","type","gap","readiness","gap analysis","assess",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":139,"publishedAt":47},"01M6KC461R3GKGZ1TAYHVR1MW9","risk-management-specialist","Medical device risk management specialist implementing ISO 14971 throughout product lifecycle. Provides risk analysis, risk evaluation, risk control, and post-production information analysis. Use when user mentions risk management, ISO 14971, risk analysis, FMEA, fault tree analysis, hazard identification, risk control, risk matrix, benefit-risk analysis, residual risk, risk acceptability, or post-market risk.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Frisk-management-specialist\u002FSKILL.md",[],[140,141,142,143,144,145,146,147,122,37,148,149,150,151,152,153,154,155,156,157],"risk","risk management specialist","management","risk management","risk evaluation","product","Evaluation","lifecycle","risk matrix","iso","medical","iso 14971","identification","information","post-production","tree","matrix","implementing",{"skillNo":159,"name":160,"nameZh":6,"description":161,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":162,"categories":163,"keywords":164,"publishedAt":47},"01M6KC3HF5ZFSSP2NZF68PEPFB","regulatory-affairs-head","Senior Regulatory Affairs Manager for HealthTech and MedTech companies. Prepares FDA 510(k), De Novo, and PMA submission packages; analyzes regulatory pathways for new medical devices; drafts responses to FDA deficiency letters and Notified Body queries; develops CE marking technical documentation under EU MDR 2017\u002F745; coordinates multi-market approval strategies across FDA, EU, Health Canada, PMDA, and NMPA; and maintains regulatory intelligence on evolving standards. Use when users need to plan or execute FDA submissions, navigate 510(k) or PMA approval processes, achieve CE marking, prepare pre-submission meeting materials, write regulatory strategy documents, respond to agency queries, or manage compliance documentation for medical device market access.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fregulatory-affairs-head\u002FSKILL.md",[],[165,166,167,168,169,170,171,172,173,37,174,175,176,177,178,179,180,181,182,32],"regulatory affairs head","regulatory","head","documents","devices","health","plan","write","strategy","standards","queries","manage","strategies","de","users","respond","medical devices","manager",{"skillNo":184,"name":185,"nameZh":6,"description":186,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":187,"categories":188,"keywords":195,"publishedAt":47},"01M6KC34Z8M1559EY1CXYBJB1A","quality-manager-qms-iso13485","ISO 13485 Quality Management System implementation and maintenance for medical device organizations. Provides QMS design, documentation control, internal auditing, CAPA management, and certification support. Use when working with medical device quality systems, preparing for ISO 13485 audits, managing regulatory compliance documentation, setting up corrective actions, or building audit preparation programs. Useful for quality management, audit preparation, regulatory compliance, medical device documentation, and corrective action workflows.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fquality-manager-qms-iso13485\u002FSKILL.md",[189,192],{"categoryNo":190,"categoryName":191,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000015","视觉设计",{"categoryNo":193,"categoryName":194,"taxonomyType":112,"taxonomyTypeDesc":113},"01JBSEEDCAT000000000000032","设计师",[196,197,182,198,42,199,200,201,202,37,203,204,205,206,207,208,209,210,32,211],"quality manager qms iso13485","quality","design","validation","actions","certification","system","maintenance","document","systems","records","ISO 13485","organizations","corrective action","internal audit","preparation",{"skillNo":213,"name":214,"nameZh":6,"description":215,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":216,"categories":217,"keywords":219,"publishedAt":47},"01M6KC2FXHBHXYX2ZZ17Y3GDB4","quality-manager-qmr","Senior Quality Manager Responsible Person (QMR) for HealthTech and MedTech companies. Provides quality system governance, management review leadership, regulatory compliance oversight, and quality performance monitoring per ISO 13485 Clause 5.5.2.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fquality-manager-qmr\u002FSKILL.md",[218],{"categoryNo":110,"categoryName":111,"taxonomyType":112,"taxonomyTypeDesc":113},[220,197,182,221,222,223,224,225,115,226,202,227,166,228,229,230,149,207,32,231],"quality manager qmr","review","policy","performance","governance","monitoring","regulatory compliance","oversight","cost","culture","leadership","person",{"skillNo":233,"name":234,"nameZh":6,"description":235,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":236,"categories":237,"keywords":243,"publishedAt":47},"01M6KC1VRR5JTDBNZANHPS6VR6","quality-documentation-manager","Document control system management for medical device QMS. Covers document numbering, version control, change management, and 21 CFR Part 11 compliance. Use for document control procedures, change control workflow, document numbering, version management, electronic signature compliance, or regulatory documentation review.","https:\u002F\u002Fgithub.com\u002Fafaizalam2003\u002FMaster-claude-skill\u002Fblob\u002Fmain\u002Fra-qm-team\u002Fquality-documentation-manager\u002FSKILL.md",[238,241,242],{"categoryNo":239,"categoryName":240,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},{"categoryNo":110,"categoryName":111,"taxonomyType":112,"taxonomyTypeDesc":113},[244,245,197,182,221,246,247,119,202,248,249,250,251,252,150,253,254,255,256,32],"documentation","quality documentation manager","workflow","change","change management","list","master","system management","documentation review","cfr","21 cfr part 11","change control","retention"]