Proper experimental design can save you a lot of headaches and wasted effort. One experimental design tool is often called a Power Analysis. A Power Analysis lets you determine if your design will have enough power to detect an effect. Statistical power is the probability of detecting a trend, given a trend actually exists. Importantly, statistical power is intricately linked with three other quantities: significance threshold, sample size, and the effect size. Given three of four of the quantities, you can calculate the forth using formulas available for common experimental situations.
We previously discussed this in a tutorial. R has several tools available to help us to perform a Power Analysis.
Answers to the exercises are available here.
Install and then load the pwr package. This package contains functions for Power Analysis.
pwr.t.test() to calculate the minimum number of samples required for an effect size (d) of 0.3, a significance level of 0.05, and a power of 0.8 for a two-sample test.
pwr.t.test(), but now calculate the effect size that could be detected. Use 50 samples, a significance level of 0.05, and power of 0.8.
pwr.t.test(), but now calculate the power given 50 samples, an effect size (d) of 0.3, and a significance level of 0.05.
Examine the effect of sample size on statistical power. Use
pwr.t.test() to calculate the statistical power of different numbers of samples. Then, plot statistical power versus sample size. Use an effect size of 0.3 and a significance level of 0.05.
Similar to the previous problem, plot statistical power versus the effect size. Use 100 samples and a significance threshold of 0.05.
The pwr package also includes other commands, including one for ANOVAs,
pwr.anova.test(). Use this command to calculate the power for an experiment with 6 groups, a sample size of 50 per group, a significance level of 0.05, and an effect size of 0.2.
Load the PlantGrowth data that is built into R. This data consists of plant weights for a control and two different treatment groups. There are 10 replicates per group, for a total of 30 samples. Calculate the effect size of this data using the
anova() command and the formula:
Where pi is the fraction of the total samples in group i, k is the total number of groups, μi is the mean of all samples in group i, and μ is the mean of all samples. Lastly, σ is the error variance within the group.
Calculate the statistical power obtained using an ANOVA for the PlantGrowth data. Use the effect size calculated in the last problem, along with the number of samples, number of groups, and a significance threshold of 0.05.
Explore other commands found in the pwr package.