Key point: Flicker (FOOC) happens when the variant is applied after the user already sees the original. The safest fix is to assign variants before render (server-side / edge) or apply changes before first paint with a lightweight snippet. “Hide-the-page” anti-flicker snippets reduce flashing but can trade it for blank-screen delays.
Data note
This post focuses on a reproducible flicker measurement protocol. The results table below is a reporting template you can use to publish your own benchmark.
What “flicker” actually is (FOOC)
Flicker is the time window where a user can see the original experience before your variant is applied. In client-side testing, the page often paints first, then the experiment script loads, evaluates targeting, and mutates the DOM.
Best case
Variant applied before first paint (FOOC ≈ 0ms).
Common case
Variant applied 50–250ms after paint (visible on slow devices).
Worst case
Page is hidden to avoid flash, but users see a blank screen instead.
Methodology
Here’s the protocol we recommend for a credible flicker benchmark:
- • Measure on slow networks (at least “Fast 3G” simulation) and mid-range devices
- • Define “variant applied” with a deterministic marker (data-attribute, class, or JS event)
- • Measure from first paint to marker time (FOOC window)
- • Report p50 and p95 (tail behavior is what users feel)
- • Segment by page type (homepage vs PDP vs checkout)
Tip: Don’t report only an average. Flicker is a tail problem. Report at least p50 and p95.
Results table (template)
| Tool | FOOC (ms) | Visible? | Notes |
|---|---|---|---|
| Lightweight snippet | < 50 | Usually no | Early execution + minimal DOM work |
| Medium script | 50–150 | Sometimes | More runtime + slower networks expose flash |
| Heavy visual editor runtime | 150–350 | Often | Late apply; async assets; many DOM mutations |
Key Findings
- 1.The earlier you apply the variant, the lower the flicker. Server-side/edge assignment is the cleanest path.
- 2.Anti-flicker hiding is a tradeoff. It avoids “flash,” but can create “blank-screen delay.”
- 3.Measure on slow conditions. Fast laptops on broadband hide problems your real users experience.
Why Flicker Matters
- • Biases results: Users see both versions, contaminating the test
- • Hurts trust: Flickering pages feel broken or untrustworthy
- • Impacts conversions: Poor UX increases bounce and reduces engagement
How to eliminate flicker (ranked)
- 1Edge/server-side variant assignment (no flash, best SEO/UX).
- 2Early lightweight client snippet that applies changes before paint.
- 3Hide-the-page “anti-flicker” as a last resort (avoid long hiding timeouts).
Implementation details: see A/B testing without flicker.
Our Recommendation
Choose tools and architectures that apply the variant before render whenever possible. If you must do client-side, keep runtime small and apply changes early.