SUTVA (Stable Unit Treatment Value Assumption): One user's treatment shouldn't affect another's outcome. Violated in: marketplaces, social networks, referral programs. When violated, standard A/B tests give biased results. Solutions: cluster randomization, geo-based splits, or time-based experiments.
What is SUTVA?
SUTVA is a core assumption of A/B testing:
"The treatment assigned to one unit does not affect the outcome of another unit."
In plain English: What you show User A shouldn't affect User B's behavior.
When SUTVA is Violated
Marketplace (Uber, Airbnb)
Violation: Driver availability affects rider experience
Solution: Geo-based randomization
Social networks
Violation: User A sees content from User B in different variant
Solution: Cluster randomization
Two-sided platforms
Violation: Supply side affects demand side
Solution: Time-based or market-based splits
Referral programs
Violation: User A refers User B, both in different variants
Solution: Household/network-level randomization
Real Example: Marketplace
Scenario: Testing a new driver incentive on Uber.
- • Variant A (control): Normal driver pay
- • Variant B: +20% driver pay
The problem: Drivers in Variant B are more available. This affects riders in Variant A (control) who get faster pickups because more drivers are online.
Result: Your test underestimates the true effect because the control group benefits from spillover.
Solutions
Cluster Randomization
Randomize by city, region, or social cluster instead of individual users
Time-Based Experiments
Run variant A for 2 weeks, then variant B for 2 weeks (switchback design)
Accept the Bias
For most websites, SUTVA violations are minimal—proceed with standard A/B testing
When to Worry About SUTVA
High Risk:
- • Marketplaces
- • Social networks
- • Referral programs
- • Shared inventory systems
Low Risk:
- • E-commerce product pages
- • SaaS pricing pages
- • Landing pages
- • Most marketing sites
For Most Sites
SUTVA violations are rare outside of marketplaces and social networks. Standard A/B testing with ExperimentHQ works fine.