5 published
Blog
Notes from the field: what worked, what broke, and what the numbers actually said.
PostLogistic Regression for Hypothesis Testing: Maximum Likelihood EstimationWhere LL-Null, Log-Likelihood and the LLR p-value in a logistic regression come from, built up from a 12-observation experiment you can check by hand.PostNot A/B Testing Everything is FineLeading voices in experimentation suggest that you test everything. Some inconvenient truths about A/B testing suggest it’s better not to.PostCaveats of Testing Elements below the FoldOnly a fraction of visitors scroll far enough to see a change below the fold. Here is what that does to your sample size and test duration.PostA/B Testing Go-no-Go Part 2: ImpactA 20% uplift on a small segment can move the overall metric by less than 1%. How to weigh an experiment's reach before you run it.PostA/B Testing Go-no-Go Part 1: NoiseWhy users who never see the change dilute an A/B test, and how much sensitivity that noise costs you. A worked example with real numbers.