Bonferroni Method

Background:

We know ANOVA is good for testing if there is any difference between the mean value among different groups.

Null Hypothesis: $x0=x1=x2....$
If the p-value for ANOVA <0.05, we know there is at least one group have different mean values compared to others.However, we do not know which groups have significant mean values. If we would like to know which paired groups have significant differences between the mean values, we will use the Bonferroni Method.

Bonferroni’s Method:

It is a stronger version of ANOVA; it is used for multiple hypothesis testing to find out the mean value of which pair treatments are significantly different from each other.
Null Hypothesis: $x1=x2$, $x2=x3$, … ,$xn=x1$

Bonferroni Confidence Intervals and P-value:

Since there are multiple groups, if we conduct the Confidence level 1-α individually, therefore, we need to make adjusts according to the number of treatment groups so we can achieve a 1-α overall Confidence interval by using the significance level of α/m. If the Bonferroni confidence interval contains 0 then it means the mean value of these group not significant, otherwise, the mean value of the two groups are significant. For Example, If we want to achieve a 95% CI of 2 treatment groups, A and B, compare with Placebo individually, then we need to set the individual group to be 97.5% CI.

Similarly, for a P-value to be statistically significant, we need to consider the number of groups. In the previous example, we need P-value < 0.025 to demonstrate the mean value is different between Treatment A/B and Placebo.

Note: If there are k groups, then there are k(k-1)/2 pairwise differences to consider.

Example:

Suppose we have 4 Treatment Group and Pain Score Value, we want to know which groups have significant mean values.

Solution:

proc anova data=one ;
class treatment;
model value=treatment ;
means treatment / alpha=0.0125 bon cldiff;
run;

title "Mean Value by Treatment";
proc sgplot data=two;
vbar treatment/response=mean
barwidth=0.6;
run;

Outcome:

From the ANOVA result, we can see Treatment 1-2, Treatment 2-4 and Treatment 1-3 have the significant difference in mean value. Our plot visually supported the result as well.

Summary:

ANOVA method will tell us if there’s the difference between the mean value for each group, and Bonferroni’s method investigates one more step to check which pair of mean values is significantly different from each other. We can use the Confidence Interval and P-value to determine the result, When m is too large, too many treatment groups, then Bonferroni is not recommended.

Alternative Method for Clinical Trial Studies, Dunnett Method, it is the best method for treatment VS comparison.

Thanks very much to Renee Wu.5 for sharing and go through the Bonferroni Method with me!

Happy Studying! 🤡