Keywords

Assumptions; one-way ANOVA; two-way ANOVA

11.1 Introduction

In this chapter we briefly introduce Analysis of variance (ANOVA) which is statistical test used to determine if more than two population means are equal. The test uses the F-distribution (probability distribution) function and information about the variances of each population (within) and grouping of populations (between) to help decide if variability between and within each population is significantly different. It is based on the comparison of the average value of a common component. The method of ANOVA tests the hypotheses that:

$\begin{array}{cc}{H}_0& ={\mu }_1={\mu }_2=\dots ={\mu }_k\\ H_1& =\text{Not all the means are equal}\end{array}$

The purpose of ANOVA test statistic is

The between-sample variance or error is the average of the square variations of each population mean from the mean or all the data and is an estimate of σ² only if the null hypothesis, H₀ is true.

When the null hypothesis is false this variance is relatively large and by comparing it with the within-sample variance we can tell statistically whether H₀ is true or not.

The ANOVA technique helps in performing the test in one go and, therefore, is considered to be important technique of analysis. The basic principle underlying the technique is that the total variation in the dependent variable is broken into two parts—one which can be attributed to some specific causes and the other that may be attributed to chance. The one which is attributed to the specific causes is known as the variation between the samples and the one which is attributed to chance is called the variation within the samples. In ANOVA it is assumed that each of the samples drawn from a normal population and each of these populations has an equal variance.

The measure of variability used in the analysis of variance is called a “mean square,” i.e.,

$\mathrm{Mean}s\mathrm{quare}=\frac{\mathrm{Sum}\mathrm{of}\mathrm{squared}\mathrm{deviations}\mathrm{from}\mathrm{mean}}{\mathrm{Degrees}\mathrm{of}\mathrm{freedom}}$

The ANOVA table stands for Analysis of Variance, which is used as a statistical technique or test in detecting the difference in population means or whether or not the means of different groups are all equal when there are more than two populations.

11.2 Assumptions

11.3 One-Way ANOVA

The one-way ANOVA is also called a single-factor analysis of variance because there is only one independent variable or factor. The independent population means are equal.

The one-way ANOVA compares the means of the samples or groups in order to make inferences about the population means of the independent variables are equal. It involves single independent variable.

When using one-way analysis of variance, the process of looking up the resulting value of F in an F-distribution table, is proven to be reliable under the following assumptions:

The assumption that the groups follow the normal curve is the usual one made in most significance tests, though here it is somewhat stronger in that it is applied to several groups at once. Of course many distributions do not follow the normal curve, so here is one reason that ANOVA may give incorrect results. It would be wise to consider whether it is reasonable to believe that the groups’ distributions follow the normal curve.

Of course the different population averages imposes no restriction on the use of ANOVA; the null hypothesis, as usual, allows us to do the computations that yield F.

The third assumption, that the populations’ standard deviations are equal, is important in principle, and it can only be approximately checked by using as bootstrap estimates the sample standard deviations. In practice statisticians feel safe in using ANOVA if the largest sample SD is not larger than twice the smallest.

Completely randomized design involves the testing of equality of means of two or more groups. In this design there is only one independent variable and one dependent variable. The dependent variable is metric whereas the independent variable is a categorical variable. We briefly explain the steps involved in one-way ANOVA.

Step 1: Set up null hypothesis (H₀) and alternative hypothesis (H₁).

Step 2: Find the total T of all observations in all the samples, i.e.,

$\sum X_1+\sum X_2+\dots +\sum X_k.$

Step 3: Find the value of the correction factor $\frac{T^2}{N},$ where

$N=n_1+n_2+\dots n_k$

Step 4: Calculate the sum of squares of deviations SST where

$\mathrm{SST}=\sum X_1^2+\sum X_2^2+\dots +\sum X_k^2-\frac{T^2}{N}$

Step 5: Find the sum of squares of deviations between the samples SSB where

$\mathrm{SSB}=\left[\frac{\sum X_1^2}{n_1}+\frac{\sum X_2^2}{n_2}+\dots +\frac{\sum X_k^2}{n_k}\right]-\frac{T^2}{N}$

Step 6: Calculate the mean square deviations within the samples (SSW)where

$\text{SSW}=\text{SST}–\text{SSB}$

Step 7: Find degrees of freedom v₁=df₁=k−1 (k=number of columns) and degrees of freedom

$v_2={\mathrm{df}}_2=N-k$

Step 8: Find the mean square deviation between the samples (MSB), where

$\mathrm{MSB}=\frac{\mathrm{SSB}}{v_1}$

and the mean square deviation within the samples (MSW), where

$\mathrm{MSW}=\frac{\mathrm{SSW}}{v_2}$

Step 9: In this step calculate the F statistic by applying the formula

$F=\frac{\mathrm{MSB}}{\mathrm{MSW}}\text{when MSB}>\text{MSW}$

and

$F=\frac{\mathrm{MSW}}{\mathrm{MSB}}\text{when MSW}>\text{MSB}$

Step 10: In this Final step, compare calculated value of F with the tabulated value. If the calculated value is less than the tabulated value of F, accept the null hypothesis and if the calculated value is greater than table value of F, reject the null hypothesis.

Table value of F, reject the null hypothesis.

ANOVA Table

Source of variation	Sum of scores	Degrees of freedom	Mean square	Statistic—F
Between varieties (column means)	SSB SSW	df1=v1 df2=v2	MSB MSW	$F=\frac{\mathrm{MSB}}{\mathrm{MSW}}$

The method is explained with the help of an example given below:

Example 11.1: Three varieties of A, B, C wheat were shown in 4 plots each and the following yields in tonnes per acre were obtained:

(The table value of F=at 5% level of significance for (2, 9) degrees of freedom is 2.165).

Test the significance of difference between yields of the varieties.

Solution: Null Hypothesis H₀: Varieties of wheat are not significantly different from each other in their yielding capacities.

Alternative Hypothesis H₁: Varieties of wheat are significantly different from each other in their yielding capacities.

We have the following table:

Sample A		Sample B		Sample C
X1	${\mathit{X}}_1^2$	X2	${\mathit{X}}_2^2$	X3	${\mathit{X}}_3^2$
7	49	3	9	3	9
4	16	5	25	5	25
6	36	5	25	4	16
8	64	7	49	2	4

$n_1=4,n_2=4,n_3=4,N=n_1+n_2+n_3=4+4+4=12\mathrm{and}k=3$

$\sum X_1=25,\sum X_2=20,\sum X_3=15$

$\sum X_1^2=165,\sum X_2^2=108,\sum X_3^2=61$

$T=\sum X_1+\sum X_2+\sum X_3=25+20+15=60$

$T^2={(60)}^2=3600$

$\mathrm{Correction}\mathrm{factor}=\frac{T^2}{N}=\frac{3600}{12}=300$

$\begin{array}{ll}\mathrm{Total}\mathrm{sum}\mathrm{of}\mathrm{squares}(\text{SST})\hfill & =\sum X_1^2+\sum X_2^2+\sum X_3^2-\frac{T^2}{N}\hfill \\ \hfill & =165+108+61–300=334–300=34\hfill \end{array}$

$\mathrm{Degrees}\mathrm{of}\mathrm{freedom}={\mathrm{df}}_1=v_1=k–1=3–1=2$

$\mathrm{Degrees}\mathrm{of}\mathrm{freedom}={\mathrm{df}}_2=v_2=N–k=12–3=9$

$\begin{array}{l}\mathrm{Sum}\mathrm{of}\mathrm{squares}\mathrm{between}\mathrm{the}\text{samples}(\text{SSB})\\ =\left[\frac{\sum X_1^2}{n_1}+\frac{\sum X_2^2}{n_2}+\dots +\frac{\sum X_k^2}{n_k}\right]-\frac{T^2}{N}\\ =\left[\frac{{(25)}^2}{4}+\frac{{(20)}^2}{4}+\dots +\frac{{(15)}^2}{4}\right]-300\\ =\frac{1250}{4}-300=312.5-300=12.5\end{array}$

Sum of squares=within the varieties (SSW)=SSW−SSB=34−12.5=21.5

Mean square between varieties=$\mathrm{MSB}=\frac{\mathrm{SSB}}{v_1}=\frac{12.5}{2}=6.25$

Mean square within varieties $\mathrm{MSW}=\frac{\mathrm{SSW}}{v_2}=\frac{21.5}{2}=2.39$ (approx.)

Test statistic=$F=\frac{\mathrm{MSB}}{\mathrm{MSW}}=\frac{6.25}{2.39}=2.615$

ANOVA Table

Source of variation	Sum of scores	Degrees of freedom	Mean square	Statistic—F
Between varieties (column means)	SSB=12.5	df1=v1=2	MSB=6.25	$\begin{array}{cc}F& =\frac{\mathrm{MSB}}{\mathrm{MSW}}\\ & =2.615\end{array}$
Within samples (errors)	SSW=21.5	df2=v2=9	MSW=2.39
Total	34.0

Since the calculated value of F is less than the table value of F for (2, 9) df at 5% level of significance we accept Null hypothesis.

11.4 Working Rule

The following steps are involved in the short-cut method for ANOVA:

Step 1: Set up null hypothesis (H₀) and alternative hypothesis (H₁)

Step 2: Find the sum of the values of all items of all samples and denote it by T
i.e.,

$T=\sum X_1+\sum X_2+\dots +\sum X_k$

Step 3: Find the value of correction factor=$\frac{T^2}{N}$

Step 4: Find the sum of squares of all items of all samples

Step 5: Find the total sum of squares SST, where

$\mathrm{SST}=\sum X_1^2+\sum X_2^2+\dots +\sum X_k^2-\frac{T^2}{N}$

Step 6: Find the total sum of squares between the samples SSC

$=\left[\frac{\sum X_1^2}{n_1}+\frac{\sum X_2^2}{n_2}+\dots +\frac{\sum X_k^2}{n_k}\right]-\frac{T^2}{N}$

Step 7: Find the sum of squares within the samples SSE
where

$\text{SSE}=\text{SST}-\text{SSC}$

Step 8: Set up ANOVA table and calculate F, which is the test statistic.

Source of variation	Sum of scores	Degrees of freedom	Variance ratio	F
Between columns	SSC	c−1	MSC	$F_C=\frac{\mathrm{MSC}}{\mathrm{MSE}}$
Between rows	SSR	r−1	MSR
Error	SSE	(c−1) (r−1)	MSE	$F_R=\frac{\mathrm{MSR}}{\mathrm{MSE}}$

Example 11.2:

(For (3,9) df F_0.05=4.76 and for (2, 6) df F_0.05=5.14).

Solution:

Null hypothesis

X1	${\mathit{X}}_{\mathbf{1}}^{\mathbf{2}}$	${\mathit{X}}_{\mathbf{2}}$	${\mathit{X}}_{\mathbf{2}}^{\mathbf{2}}$	X3	${\mathit{X}}_{\mathbf{3}}^{\mathbf{2}}$	X4	${\mathit{X}}_{\mathbf{4}}^{\mathbf{2}}$
3	16	4	16	6	36	6	36
6	36	4	16	5	25	3	09
6	36	6	36	4	16	7	49
$\sum X_1=15$	$\sum X_1^2=15$	$\sum X_2=14$	$\sum X_2^2=68$	$\sum X_3=15$	$\sum X_3^2=77$	$\sum X_4=16$	$\sum X_4^2=94$

We have

$\sum X_1=15,\sum X_2=14,\sum X_3=15,\sum X_4=16,$

$n_1=3,n_2=3.n_3=3,n_4=3,N=3+3+3+3=12$

$T=\sum X_1+\sum X_2+\sum X_3+\sum X_4=15+14+15+16=60$

$\mathrm{Correction}\mathrm{factor}=\frac{T^2}{N}=\frac{{(60)}^2}{12}=\frac{3600}{12}=300$

Total sum of squares

$\begin{array}{l}=SST=\sum X_1^2+\sum X_2^2+\dots +\sum X_k^2-\frac{T^2}{N}\hfill \\ =88+68+77+94–300\hfill \\ =320–300=20\hfill \end{array}$

Sum of squares between the samples (i.e., between columns) (SSC)

$\begin{array}{l}=\left[\frac{\sum X_1^2}{n_1}+\frac{\sum X_2^2}{n_2}+\dots +\frac{\sum X_k^2}{n_k}\right]-\frac{T^2}{N}\hfill \\ =\left[\frac{{\left(15\right)}^2}{3}+\frac{{\left(14\right)}^2}{3}+\frac{{\left(15\right)}^2}{3}+\frac{{\left(16\right)}^2}{3}\right]-300\hfill \\ =\frac{902}{3}-300=300.67-300=0.67=303.5–300=3.5\hfill \end{array}$

Sum of squares within the samples (between rows) SSR

$=\left[\frac{{(19)}^2}{4}+\frac{{(18)}^2}{4}+\frac{{(23)}^2}{4}\right]-300$

Error=SST – (SSC+SSR)=20−(0.67+3.5)=15.83

Source of variation	Sum of squares	Degrees of freedom (v)	Variance ratio	F
Between columns (yields)	0.67	v1=c–1=3	$\begin{array}{cc}\mathrm{MSC}& =\frac{\mathrm{MSC}}{v_1}\\ & =0.223\end{array}$	$\begin{array}{cc}{F}_C& =\frac{\mathrm{MSC}}{\mathrm{MSE}}\\ & =0.076\end{array}$
Between rows	3.50	v2=2	$\begin{array}{cc}\mathrm{MSR}& =\frac{\mathrm{SSR}}{R-1}\\ & =1.75\end{array}$	$\begin{array}{cc}{F}_R& =\frac{\mathrm{MSR}}{\mathrm{MSE}}\\ & =0.597\end{array}$
Error	15.83	6	$\begin{array}{cc}\mathrm{MSE}& =\frac{15.83}{6}\\ & =2.930\end{array}$

For (3, 6) df, the calculated value of F is less than the table value of F, hence we accept null hypothesis, i.e., there is no significant difference in the yields of four varieties of wheat.

For (3, 6) df, the calculated value of F is less than the table value of F, hence we accept null hypothesis, i.e., there is no significant difference in the yields of four varieties of wheat.

For (2, 6) df, the calculated value of F is less than the table value of F, hence we accept null hypothesis, i.e., there is no significant difference in the y plots of land with regard to yield.

Exercise 11.1

Test (a) whether the mean productivity is the same for the different machine types, and (b) whether the three workers differ with respect to mean productivity. Table value of $F_C$ at 5% level for (2, 4) df is 6.95 (And the table value of $F_R=6.95$).