Lesson 4: Sample size calculation playground - level I
In this lesson, you build intuition for sample size calculation by using an interactive sample size calculator playground. By changing parameters that we have discussed in this course, you can see how the sample size changes.
This is the first level of the sample size courses, so a lot of the parameters that you can affect in a real experiment are fixed here and cannot be changed.
The idea with this lesson is simple, you change the parameters in the sample size calculator and see how the sample size changes. In this lessons there are more questions than for usual lessons. Most can be answered by simply trying things out in the sample size calculator.
Turn on the 'Show detailed formulas' option in the sample size calculator to see the formulas used to calculate the sample size. This can help you understand how the different parameters affect the sample size.
If you double the relative MDE/NIM (from 1% to 2%), what happens to the required sample size?
What happens to the required sample size if you increase the baseline variance while keeping all other parameters constant?
If you want to increase the power from 0.8 to 0.9, what happens to the required sample size?
If power is 80% and alpha is 10%, what increases the sample size the most?
If your baseline variance is very small (near zero), what tends to happen to the required sample size?
Notes for Nerds
The Confidence sample size calculator takes into account a whole range of parameters when calculating the required sample size. If you want to learn about how Confidence handles risk management, you can read more about it in this blog post.