Yesterday I was railing against being in a hurry to launch A/B tests. (Clearly it hits a nerve with me 😅) I've always figured it was a symptom of a broader problem, and the interwebs confirms this. …
You don’t need a hypothesis
There, I said it. A/B test hypotheses, as they're commonly written, are wasted keystrokes that don't do enough to ensure your test results will be valid or that your analysis will be accurate. Let's …
Dealing with pressure to call tests early (3/3)
In Part 1 we tried to see things from the perspective of the execs, clients, media buyers, and bystanders who might favor calling a test as soon as you see lift. They have budgets, and goals to hit - …
Continue Reading about Dealing with pressure to call tests early (3/3) →
Dealing with pressure to call tests early (1/3)
You already know that calling a test too early is a Bad Thing. It can lead to Type I and Type II errors, leaving your conversion rates worse than they could've been, or even worse than where they …
Continue Reading about Dealing with pressure to call tests early (1/3) →