[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M749CC93R4WXKPM8ZV7GAGDH":3,"skill-same:01M749CC93R4WXKPM8ZV7GAGDH":48},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":32,"updatedAt":33,"install":34,"resources":47},"01M749CC93R4WXKPM8ZV7GAGDH","ab-test-analysis",null,"Analyze A\u002FB test results with statistical significance, sample size validation, confidence intervals, and ship\u002Fextend\u002Fstop recommendations. Use when evaluating experiment results, checking if a test reached significance, interpreting split test data, or deciding whether to ship a variant.","phuryn\u002Fpm-skills","phuryn","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F7837354?v=4","01M727RY5RDZZWVRDN2X5F4B1W","Pawel Huryn","01M727RY6H19NE8H5PABTR7A59",27991,"MIT","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-data-analytics\u002Fskills\u002Fab-test-analysis\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: ab-test-analysis\ndescription: \"Analyze A\u002FB test results with statistical significance, sample size validation, confidence intervals, and ship\u002Fextend\u002Fstop recommendations. Use when evaluating experiment results, checking if a test reached significance, interpreting split test data, or deciding whether to ship a variant.\"\n---\n\n## A\u002FB Test Analysis\n\nEvaluate A\u002FB test results with statistical rigor and translate findings into clear product decisions.\n\n### Context\n\nYou are analyzing A\u002FB test results for **$ARGUMENTS**.\n\nIf the user provides data files (CSV, Excel, or analytics exports), read and analyze them directly. Generate Python scripts for statistical calculations when needed.\n\n### Instructions\n\n1. **Understand the experiment**:\n   - What was the hypothesis?\n   - What was changed (the variant)?\n   - What is the primary metric? Any guardrail metrics?\n   - How long did the test run?\n   - What is the traffic split?\n\n2. **Validate the test setup**:\n   - **Sample size**: Is the sample large enough for the expected effect size?\n     - Use the formula: n = (Z²α\u002F2 × 2 × p × (1-p)) \u002F MDE²\n     - Flag if the test is underpowered (\u003C80% power)\n   - **Duration**: Did the test run for at least 1-2 full business cycles?\n   - **Randomization**: Any evidence of sample ratio mismatch (SRM)?\n   - **Novelty\u002Fprimacy effects**: Was there enough time to wash out initial behavior changes?\n\n3. **Calculate statistical significance**:\n   - **Conversion rate** for control and variant\n   - **Relative lift**: (variant - control) \u002F control × 100\n   - **p-value**: Using a two-tailed z-test or chi-squared test\n   - **Confidence interval**: 95% CI for the difference\n   - **Statistical significance**: Is p \u003C 0.05?\n   - **Practical significance**: Is the lift meaningful for the business?\n\n   If the user provides raw data, generate and run a Python script to calculate these.\n\n4. **Check guardrail metrics**:\n   - Did any guardrail metrics (revenue, engagement, page load time) degrade?\n   - A winning primary metric with degraded guardrails may not be a true win\n\n5. **Interpret results**:\n\n   | Outcome | Recommendation |\n   |---|---|\n   | Significant positive lift, no guardrail issues | **Ship it** — roll out to 100% |\n   | Significant positive lift, guardrail concerns | **Investigate** — understand trade-offs before shipping |\n   | Not significant, positive trend | **Extend the test** — need more data or larger effect |\n   | Not significant, flat | **Stop the test** — no meaningful difference detected |\n   | Significant negative lift | **Don't ship** — revert to control, analyze why |\n\n6. **Provide the analysis summary**:\n   ```\n   ## A\u002FB Test Results: [Test Name]\n\n   **Hypothesis**: [What we expected]\n   **Duration**: [X days] | **Sample**: [N control \u002F M variant]\n\n   | Metric | Control | Variant | Lift | p-value | Significant? |\n   |---|---|---|---|---|---|\n   | [Primary] | X% | Y% | +Z% | 0.0X | Yes\u002FNo |\n   | [Guardrail] | ... | ... | ... | ... | ... |\n\n   **Recommendation**: [Ship \u002F Extend \u002F Stop \u002F Investigate]\n   **Reasoning**: [Why]\n   **Next steps**: [What to do]\n   ```\n\nThink step by step. Save as markdown. Generate Python scripts for calculations if raw data is provided.\n\n---\n\n### Further Reading\n\n- [A\u002FB Testing 101 + Examples](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fab-testing-101-for-pms)\n- [Testing Product Ideas: The Ultimate Validation Experiments Library](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fthe-ultimate-experiments-library)\n- [Are You Tracking the Right Metrics?](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fare-you-tracking-the-right-metrics)\n","## A\u002FB Test Analysis\n\nEvaluate A\u002FB test results with statistical rigor and translate findings into clear product decisions.\n\n### Context\n\nYou are analyzing A\u002FB test results for **$ARGUMENTS**.\n\nIf the user provides data files (CSV, Excel, or analytics exports), read and analyze them directly. Generate Python scripts for statistical calculations when needed.\n\n### Instructions\n\n1. **Understand the experiment**:\n   - What was the hypothesis?\n   - What was changed (the variant)?\n   - What is the primary metric? Any guardrail metrics?\n   - How long did the test run?\n   - What is the traffic split?\n\n2. **Validate the test setup**:\n   - **Sample size**: Is the sample large enough for the expected effect size?\n     - Use the formula: n = (Z²α\u002F2 × 2 × p × (1-p)) \u002F MDE²\n     - Flag if the test is underpowered (\u003C80% power)\n   - **Duration**: Did the test run for at least 1-2 full business cycles?\n   - **Randomization**: Any evidence of sample ratio mismatch (SRM)?\n   - **Novelty\u002Fprimacy effects**: Was there enough time to wash out initial behavior changes?\n\n3. **Calculate statistical significance**:\n   - **Conversion rate** for control and variant\n   - **Relative lift**: (variant - control) \u002F control × 100\n   - **p-value**: Using a two-tailed z-test or chi-squared test\n   - **Confidence interval**: 95% CI for the difference\n   - **Statistical significance**: Is p \u003C 0.05?\n   - **Practical significance**: Is the lift meaningful for the business?\n\n   If the user provides raw data, generate and run a Python script to calculate these.\n\n4. **Check guardrail metrics**:\n   - Did any guardrail metrics (revenue, engagement, page load time) degrade?\n   - A winning primary metric with degraded guardrails may not be a true win\n\n5. **Interpret results**:\n\n   | Outcome | Recommendation |\n   |---|---|\n   | Significant positive lift, no guardrail issues | **Ship it** — roll out to 100% |\n   | Significant positive lift, guardrail concerns | **Investigate** — understand trade-offs before shipping |\n   | Not significant, positive trend | **Extend the test** — need more data or larger effect |\n   | Not significant, flat | **Stop the test** — no meaningful difference detected |\n   | Significant negative lift | **Don't ship** — revert to control, analyze why |\n\n6. **Provide the analysis summary**:\n   ```\n   ## A\u002FB Test Results: [Test Name]\n\n   **Hypothesis**: [What we expected]\n   **Duration**: [X days] | **Sample**: [N control \u002F M variant]\n\n   | Metric | Control | Variant | Lift | p-value | Significant? |\n   |---|---|---|---|---|---|\n   | [Primary] | X% | Y% | +Z% | 0.0X | Yes\u002FNo |\n   | [Guardrail] | ... | ... | ... | ... | ... |\n\n   **Recommendation**: [Ship \u002F Extend \u002F Stop \u002F Investigate]\n   **Reasoning**: [Why]\n   **Next steps**: [What to do]\n   ```\n\nThink step by step. Save as markdown. Generate Python scripts for calculations if raw data is provided.\n\n---\n\n### Further Reading\n\n- [A\u002FB Testing 101 + Examples](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fab-testing-101-for-pms)\n- [Testing Product Ideas: The Ultimate Validation Experiments Library](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fthe-ultimate-experiments-library)\n- [Are You Tracking the Right Metrics?](https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fare-you-tracking-the-right-metrics)\n",{"name":5,"description":7},"main","9af7bfd0bc47fdad43c2eaa7a43cd13918e7aaef","pm-data-analytics\u002Fskills\u002Fab-test-analysis",2885,"https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills","https:\u002F\u002Fwww.productcompass.pm\u002Fp\u002Fpm-skills-2-red-team-ship","https:\u002F\u002Fgithub.com\u002Fphuryn","https:\u002F\u002Fwww.productcompass.pm\u002F","AI PM Coach, ex-CPO, Builder | 400K+ Follow","2026-11-09 07:10:04",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add phuryn\u002Fpm-skills --skill ab-test-analysis",{"tool":40,"command":41},"bunx","bunx skills add phuryn\u002Fpm-skills --skill ab-test-analysis",{"tool":43,"command":44},"pnpm","pnpm dlx skills add phuryn\u002Fpm-skills --skill ab-test-analysis","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-data-analytics\u002Fskills\u002Fab-test-analysis\u002FSKILL.md",true,[],[49,56,63,70,77,84],{"skillNo":50,"name":51,"nameZh":6,"description":52,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":19},"01M749K1XJGDQ1FA90BGD6MJ6Z","review-resume","Comprehensive PM resume review and tailoring against 10 best practices including XYZ+S formula, keyword optimization, job-specific tailoring, and structure. Use when reviewing a PM resume, preparing for job applications, or improving resume impact.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-toolkit\u002Fskills\u002Freview-resume\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":19},"01M749JVQ3SER9X0V501S82BRX","privacy-policy","Draft a detailed privacy policy covering data types, jurisdiction, GDPR and compliance considerations, and clauses needing legal review. Use when creating a privacy policy, updating data protection documentation, or preparing for compliance.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-toolkit\u002Fskills\u002Fprivacy-policy\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":67,"categories":68,"keywords":69,"publishedAt":19},"01M749JRFCNDADS61HFWMBHJJD","grammar-check","Identify grammar, logical, and flow errors in text and suggest targeted fixes without rewriting the entire text. Use when proofreading content, checking writing quality, or reviewing a draft.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-toolkit\u002Fskills\u002Fgrammar-check\u002FSKILL.md",[],[],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M749JMXFX9KSXV4JP6J1K7QX","draft-nda","Draft a detailed Non-Disclosure Agreement between two parties covering information types, jurisdiction, and clauses needing legal review. Use when creating confidentiality agreements or preparing an NDA for a partnership.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-toolkit\u002Fskills\u002Fdraft-nda\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M749JGY34DGZ9BGFS2YCET6N","value-proposition","Design a detailed value proposition using a 6-part JTBD template — Who, Why, What before, How, What after, Alternatives. Use when creating a value proposition, analyzing customer value delivery, or articulating why customers should choose your product.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-product-strategy\u002Fskills\u002Fvalue-proposition\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M749JE27D1APC1PWWPFMF2GN","swot-analysis","Perform a detailed SWOT analysis — strengths, weaknesses, opportunities, and threats with actionable recommendations. Use when doing strategic assessment, competitive analysis, or evaluating a product or business position.","https:\u002F\u002Fgithub.com\u002Fphuryn\u002Fpm-skills\u002Fblob\u002Fmain\u002Fpm-product-strategy\u002Fskills\u002Fswot-analysis\u002FSKILL.md",[],[]]