Summary
A/B testing assigns users to experiment variants before the page renders, eliminating the layout shift that client-side swaps produce. Stable bucketing via a hashed user ID keeps the same user in the same variant across sessions. Mutex groups prevent two experiments from colliding on the same surface. Guardrail metrics and sample ratio mismatch checks catch broken experiments early.
Jump to the interview angleA/B testing runs two (or more) variants of a feature against separate user segments to measure causal effect on a metric. The assignment decision must happen server-side — either in an edge middleware or origin handler — before HTML is generated. Client-side swaps (hiding the control, injecting the variant after load) cause a flash of the original content that harms both user experience and measurement accuracy. The canonical approach: hash the user's stable ID with the experiment ID to derive a deterministic bucket, then rewrite the request at the edge so the correct variant is rendered on the first byte.
Edge middleware variant assignment (Next.js)
Middleware runs before the cache and before the React render tree. It reads the user's ID cookie, hashes it into a bucket, and rewrites the URL so the correct variant page is served without a client-side swap.
// middleware.ts (Next.js 15 — runs on Vercel Edge Runtime)
import { NextRequest, NextResponse } from "next/server";
/** Deterministic 32-bit hash (FNV-1a variant). */
function fnv32a(str: string): number {
let h = 0x811c9dc5;
for (let i = 0; i < str.length; i++) {
h ^= str.charCodeAt(i);
h = (h * 0x01000193) >>> 0;
}
return h;
}
/** Returns 0..99 bucket for (userId, experimentId). */
function bucket(userId: string, experimentId: string): number {
return fnv32a(userId + ":" + experimentId) % 100;
}
const EXPERIMENT_ID = "checkout-cta-v2";
const VARIANT_THRESHOLD = 50; // 50 % control, 50 % treatment
export function middleware(req: NextRequest) {
const url = req.nextUrl.clone();
if (!url.pathname.startsWith("/checkout")) {
return NextResponse.next();
}
const userId =
req.cookies.get("uid")?.value ?? crypto.randomUUID();
const b = bucket(userId, EXPERIMENT_ID);
const variant = b < VARIANT_THRESHOLD ? "control" : "treatment";
// Rewrite to variant-specific segment — no client flicker.
url.pathname = `/checkout/${variant}`;
const res = NextResponse.rewrite(url);
// Persist anonymous ID so bucketing stays stable.
if (!req.cookies.get("uid")) {
res.cookies.set("uid", userId, {
httpOnly: true,
sameSite: "lax",
maxAge: 60 * 60 * 24 * 365,
});
}
// Log exposure for analysis pipeline.
res.headers.set("X-Experiment", `${EXPERIMENT_ID}:${variant}`);
return res;
}FNV-1a hashing is fast and produces uniform distribution across 100 buckets. The URL rewrite happens before Next.js renders, so the browser never sees the control variant when assigned to treatment.
Experiment integrity checklist
- Assign at the edge, not in client JS — avoids flicker and measurement noise from bots.
- Hash user ID + experiment ID together so two experiments produce independent bucket sequences.
- Use mutex groups when two experiments touch the same UI surface to prevent interaction effects.
- Run a chi-square SRM check before reading any metric — a p-value < 0.01 means the split is broken.
- CUPED regresses a pre-experiment covariate (e.g., last week's revenue) to cut required sample size by ~30–50 %.
SRM invalidates your results
If you targeted a 50/50 split but measured 47/53, stop reading metrics. SRM means users are being bucketed or filtered asymmetrically — a common cause is caching the rewrite response without varying on the assignment cookie. Always set Vary: Cookie (or equivalent) on cached experiment routes.
Interview angle
Interviewers probe how you avoid flicker and keep bucketing stable. Walk through edge-middleware rewrite, hashing the user ID into a bucket, mutex groups for exclusivity, and SRM checks before reading results.
Soundbite: "Hash the user ID at the edge, rewrite the URL, and check for SRM before trusting any metric."
Key terms
- Variant assignment
- Placing a user in experiment group A or B, done once per experiment before the response is sent.
- Stable bucketing
- Hashing user ID + experiment ID so the same user always lands in the same variant across sessions.
- Mutex group
- A set of experiments that share a user-traffic pool; each user enters at most one experiment in the group.
- Sample ratio mismatch (SRM)
- When measured variant traffic differs significantly from the intended split, signaling a broken assignment.
- CUPED
- Controlled-experiment Using Pre-Experiment Data — reduces variance by regressing out pre-experiment covariate signal.