Before launching any A/B test: Check visual rendering (no flicker, works on mobile), functional correctness (links, forms work), tracking accuracy (conversions fire), statistical setup (sample size calculated), and technical compatibility (CSP, caching, SEO). Use this checklist to catch issues before they bias your results.
Why QA Matters
Common issues that ruin A/B tests:
- • Flicker: Biases results, hurts UX
- • Broken tracking: One variant doesn't track conversions → false results
- • Sample ratio mismatch: 60/40 split instead of 50/50 → invalid test
- • Mobile bugs: Variant breaks on mobile → loses 50% of traffic
The Complete Checklist
Visual QA
Functional QA
Tracking QA
Statistical QA
Technical QA
Critical Checks (Never Skip)
Test on real devices
Browser emulators miss real device issues
Verify tracking in both variants
Complete a test conversion in each variant
Check console for errors
JavaScript errors can break tracking or functionality
Monitor first 24 hours closely
Catch issues early before they contaminate results
Download Checklist
Get the checklist as a printable PDF or Notion template:
Run Quality Tests
ExperimentHQ includes built-in QA warnings for common issues like flicker and tracking problems.