coreyhainesco launched ASO A/B skill
- Corey Haines’ open-source `ab-test-setup` skill is live in his `marketingskills` repo, packaging A/B test planning rules for AI agents into a reusable workflow. - The skill’s guidance centers on pre-set sample sizes, one-variable tests, and no early stopping; its reference tables show 72,000 visitors per variant at 5% baseline conversion. - It extends Haines’ fast-growing marketing-skills library, which reached 38 skills in the latest release and now spans ASO, CRO, and analytics. (github.com)
Corey Haines has published an `ab-test-setup` skill in his open-source `marketingskills` repository, giving AI agents a template for planning statistically valid A/B tests. (github.com) The skill is aimed at users who want to plan experiments, compare variants, or build a growth-testing program, and the current metadata lists it at version 1.2.0. (github.com) Its instructions tell agents to start with a hypothesis, test one variable at a time, pre-determine sample size, and avoid peeking at results and stopping early. (github.com) The supporting sample-size guide spells out the inputs behind that advice: baseline conversion rate, minimum detectable effect, 95% significance, and 80% statistical power. (github.com) The reference tables show how quickly test size grows when baseline conversion is low. At a 5% conversion rate, detecting a 10% lift calls for about 72,000 visitors per variant; at 3%, the same lift needs about 120,000. (github.com) That matters for app-store work because App Store Optimization A/B tests often fail on basics like short runtimes, mixed traffic sources, or unclear success metrics. AppTweak says tests should run at least seven days and warns that mixing paid and organic traffic can skew results. (apptweak.com) The same logic applies directly to app-store listings. SplitMetrics says native tools for this work are Apple’s Product Page Optimization in App Store Connect and Store Listing Experiments in Google Play Console. (splitmetrics.com) Haines’ broader repository has been expanding quickly. GitHub shows the project at 24,800 stars, and the latest release notes said the library had grown to 38 skills, after adding ASO audit work in an earlier release and more skills afterward. (github.com 1) (github.com 2) So the launch is less a standalone calculator than a new building block in a larger AI marketing toolkit: one that tries to turn experimentation rules into default behavior before a test ever goes live. (github.com)
