Chapter 17
Non-Parametric Statistics
IN THIS CHAPTER
Introducing non-parametric statistics
Testing independent samples
Testing related samples
Correlating ranks
The statistical methods I cover in earlier chapters have a couple of things in common. First, we assume ratio (or at least interval) data. (If you don’t know what that means, reread the section entitled “Types of data” in Chapter 1.) Second, we can use sample statistics to estimate parameters of the sampling distribution, and we use the central limit theorem to characterize the nature of the distribution so that we can test hypotheses.
Sometimes we have to analyze nominal data or ordinal data (again, Chapter 1). And sometimes we can’t specify the distribution of the variable we’re working with.
To deal with these cases, statisticians have developed non-parametric statistical tests. The list of these tests is long, and growing all the time. Many of them require special lookup tables for hypothesis tests.
I want to avoid those special tables. So, to make the cut for this chapter, a test had to either (a) test hypotheses via a well-known distribution built into Excel or (b) work with a well-known distribution when samples are large. Of the non-parametric tests that fit the bill, I chose six classical ones.
Because Excel has no special data analysis tools or built-in functions for these tests, the general plan is to translate the test formulas into Excel formulas and then use an Excel statistical function (NORM.DIST or CHISQ.DIST.RT) to perform the hypothesis test.
The easiest way to tell you about each test is to show you the finished spreadsheet and describe the steps it takes to get there.
Independent Samples
The non-parametric tests I show you in this section are analogous to the independent groups t-test and to the one-factor analysis of variance.
Two samples: Mann-Whitney U test
The Mann-Whitney U test is one of the best-known non-parametric tests. You use it to test whether two independent samples come from the same population in situations where you don’t have interval or ratio data (and thus can’t use the t-test).
When the data are ordinal, statisticians work with the ranks of the data rather than with the original data points, and that’s what this test does.
Imagine a study in which 40 people are randomly assigned to watch either a stressful movie or a cartoon. Then they have to rate how stressed they feel on a scale of 1 (least stressed) to 100 (most stressed).
Figure 17-1 shows a spreadsheet with the data in columns A and B. The null hypothesis is that the two groups came out of the same population. The alternative hypothesis is that the stress would be higher after the stressful movie, so this can be a one-tailed test at α = .05.
FIGURE 17-1: The Mann-Whitney U test.
The first order of business is to rank each score (in ascending order) in the context of all scores. If the null hypothesis is true, the high ranks and the low ranks (and all those in between) should be distributed about equally through the two groups.
I can base the formula for the Mann-Whitney U test on either the ranks in column A or the ranks in column B.
If I choose A, the formula is
If I choose B, it’s
RA is the sum of the ranks in column A, RB is the sum of the ranks in column B, NA is the number of scores in column A, and NB is the number of scores in column B.
With large samples, the sampling distribution of U is approximately a normal distribution with
and
Back to Figure 17-1. The ranks for the scores in columns A and B appear in columns D and E. How did I get them? Easy. In cell D2, I type
=RANK.AVG(A2,$A$2:$B$21,1)
Then I autofill columns D and E.
Now it’s a matter of translating either U-formula into an Excel formula. For clarity, I define NA as the name of the value in cell G2 — the number of scores in column A, and NB for the value in H2 — the number of scores in column B.
I choose to base the U-formula on column A, so in cell H4 I type
=NA*NB +((NA*(NA+1))/2)-E22
I press Enter, and the result, 271, appears.
Now for the test. For the mean of the sampling distribution, I type
=(NA*NB)/2
into cell H5 and
=SQRT((NA*NB*(NA+NB+1))/12)
into H6. To calculate the probability associated with U, I type
=NORM.DIST(H4,H5,H6,TRUE)
into H8. The result, 0.97, means that the probability of U being less than the calculated value is greater than 0.95. So the probability of a result more extreme is less than .05, and I can reject the null hypothesis.
More than two samples: Kruskal-Wallis one-way ANOVA
FarKlempt Robotics, Inc., surveys its employees about their level of satisfaction with their jobs. They ask developers, managers, maintenance workers, and tech writers to rate job satisfaction on a scale of 1 (least satisfied) to 100 (most satisfied). Six employees are in each category. Figure 17-2 shows a spreadsheet with the data in columns A through D, rows 1–7. The null hypothesis is that the samples all come from the same population. The alternative hypothesis is that they do not.
FIGURE 17-2: The Kruskal–Wallis One-Way Analysis of Variance.
The appropriate non-parametric test is the Kruskal-Wallis One-Way Analysis of Variance. As with the Mann-Whitney, I start by ranking all 24 scores in ascending order. Again, if the null hypothesis true, the ranks should be distributed about equally throughout the groups.
The formula for this statistic is
N is the total number of scores, and n is the number of scores in each group. To keep things easy, I specified the same number of scores in each group, but that’s not necessary for this test. R is the sum of the ranks in a group. H is distributed approximately as chi-square with df = number of groups — 1, when each n is greater than 5.
Looking back at Figure 17-2, the ranks for the data are in rows 9–15 of columns A through D. Row 16 holds the sums of the ranks in each group. I defined N_Total as the name for the value in cell F2, the total number of scores. I defined n_group as the name for the value in G2, the number of scores in each group.
To calculate H, I type
=(12/(N_Total*(N_Total+1)))*(SUMSQ(A16:D16)/n_group)-3*(N_Total+1)
into cell G6.
For the hypothesis test, I type
=CHISQ.DIST.RT(G6,3)
into G7. The result is less than .05, so I reject the null hypothesis.
Matched Samples
The non-parametric tests I show you in this main section are analogous to the matched groups t-test and to the Repeated Measures Analysis of Variance.
Two samples: Wilcoxon matched-pairs signed ranks
This test works with the differences between matched pairs. It goes a bit further than that, though. In addition to the direction of the differences, the Wilcoxon test considers the sizes of the differences. So this test is useful when you have ordinal data and enough precision that you can rank the differences between pairs.
Here’s an example. A social worker studies 26 pairs of identical twins. Each child in a twin-pair either attends a public school or is home-schooled. At the end of a school year, the social worker rates all the children on sociability on a scale of 1 (least sociable) to 100 (most sociable). The null hypothesis is that the two groups don’t differ. The alternative hypothesis is that they do.
Figure 17-3 shows the data in columns A and B, rows 1-27. The interpair differences are in column C, and the ranks of the differences are in column D.
FIGURE 17-3: The Wilcoxon matched-pairs signed ranks test.
Now things get interesting. The next step is to see how many pairs have a negative difference, and how many have a positive difference. If the null hypothesis is true, the sum of the ranks for the positive differences should be about the same as the sum of the ranks for the negative differences.
You work with the less frequent category and add up its ranks. I refer to that sum as T. If the number of pairs, N, is larger than 25, then T is normally distributed with
and
On the spreadsheet, I calculate the number of positive differences by typing
=COUNTIF(C2:C27,">0")
in cell G4 and the number of negative differences by typing
=COUNTIF(C2:C27,"<0")
in G5.
The results show 8 negative differences and 18 positive differences, so the next step is to add up the ranks of the negative differences. To do that, I type
=SUMIF(C2:C27,"<0",D2:D27)
in cell G7.
To test the hypotheses, I have to calculate the mean and the standard deviation of the sampling distribution. I define N as the label for cell F2, which holds the number of twin-pairs.
So I type
=(N*(N+1))/4
into cell G9 for the mean and type
=SQRT((N*(N+1)*(2*N +1))/24)
into cell G10 for the standard deviation.
And finally, to test the hypotheses, I type
=NORMDIST(G7,G9,G10,TRUE)
into G12. The very low value (way less than .05) indicates that I can reject the null hypothesis.
More than two samples: Friedman two-way ANOVA
With ordinal data and more than two matched samples, the Friedman two-way ANOVA is the appropriate non-parametric test. In this test, we rank the data within each sample.
Figure 17-4 shows an example. Twenty people rate their knowledge of economics, geography, and history on a scale of 1 (least knowledge) to 10 (most knowledge). The data are in columns A through D and rows 1–21. Think of economics, geography, and history as three matched samples. They’re “matched” because the three numbers in each row represent the data for the same person. The ranks within each sample are in columns F through H.
FIGURE 17-4: The Friedman two-way analysis of variance.
The null hypothesis is that the three different areas of knowledge yield no differences, and the alternative hypothesis is that they do. If the null hypothesis is true, the sums of the ranks for each area should be about the same.
Friedman referred to the test statistic as χr2, so I will too. The formula is
N is the number of individuals, and k is the number of groups (economics, geography, and history). With a large enough sample (more than 9), χr2 is distributed as chi-square with k-1 degrees of freedom.
To determine the ranks in columns F through H, I type
=RANK.AVG(B2,$B2:$D2,1)
into F2. Note the dollar sign ($) to the left of each column, but not to the left of each row number. This allows me to autofill all the cells from F2 to H21. Then I sum the ranks in row 22.
For clarity in the Excel formula, I define Total_N as the label for J2 (the number of people in the study) and group as the label for K2 (the number of areas of knowledge).
To calculate χr2, I type
=(12/(Total_N*groups*(groups+1))*SUMSQ(F22:H22))-3*Total_N*(groups+1)
into cell K7.
To test the hypotheses, I type
=CHISQ.DIST.RT(K7,groups-1)
into K8. The value is lower than .05, so I reject the null hypothesis.
More than two samples: Cochran’s Q
Wait. What? Another test for more than two samples? This one’s a different animal. The other tests in this chapter work with ordinal data. This one works with nominal data.
The spreadsheet in Figure 17-5 holds the data for a study of 20 people solving anagram problems. The anagrams are either easy (column B), of medium difficulty (column C), or difficult (column D). If a person solves an anagram within one minute, that’s a “Success” denoted by 1. If not, it’s a “Failure”, denoted by 0.
FIGURE 17-5: Cochran’s Q test.
The null hypothesis is that the three difficulty levels yield no differences. The alternative hypothesis is that they differ. If the null hypothesis is true, the sums for the three conditions (cells B22, C22, and D22) will be about equal.
The formula for this test is
According to longstanding usage for this test, G represents a column sum, L represents a row sum, and k is the number of conditions (three, in this example). The sampling distribution of Q approximates chi-square with k-1 degrees of freedom.
For clarity in the upcoming Excel formula, I define L as the name for the row sums in column E, G as the name for the column sums in row 22, and k as the name for the value in cell G2 (the number of conditions).
The formula for Q in cell H5 is thus:
=(k*(k-1)*DEVSQ(G))/(k*SUM(L)-SUMSQ(L))
Need a refresher on DEVSQ? Go to Chapter 5.
To test the hypotheses, I type
=CHISQ.DIST.RT(H5,k-1)
into cell H6. The value is lower than .05, which indicates that I should reject the null hypothesis.
Correlation: Spearman’s rS
Spearman’s correlation coefficient, rS, was the earliest non-parametric test based on ranks. For a sample of individuals each measured on two variables, the idea is to rank each score within its own variable. Then, for each individual subtract one rank from the other. If correlation is perfect (in the positive direction), all the differences are zero.
Figure 17-6 shows an example of what I mean. An industrial psychologist rated the sociability of 20 employees of the FarDrate Timepiece Corporation. The scale ranged from 1 (least sociable) to 100 (most sociable). Each FarDrate employee also rated his or her job satisfaction on a scale of 1 (least satisfaction) to 80 (most satisfaction). The null hypothesis is that sociability is not correlated with job satisfaction. The alternative hypothesis is that these two variables are correlated.
FIGURE 17-6: Spearman’s rS.
The data are in columns B and C, and the ranks are in columns E and F. The differences between each pair of ranks are in column G.
The formula is
where d is an interpair difference. As is the case with the regular correlation coefficient (see Chapter 15), if the null hypothesis is true, the value of rS should be around zero.
To calculate the ranks in column E, I type
=RANK.AVG(B2,$B$2:$B$21,1)
into E2 and autofilled. For the ranks in column E, I type
=RANK.AVG(C2,$C$2:$C$21,1)
into F2 and autofilled.
I don’t have to type a complicated Excel formula into cell J4 to calculate the correlation coefficient. Why? Because Excel and mathematical statistics team up for a swell surprise: All I have to do is type
=CORREL(E2:E21,F2:F21)
into J4. That’s all there is to it. Using CORREL on the ranks gives the same answer as the formula I just showed you. (So it isn’t really necessary to calculate the interpair rank differences in column G.)
The hypothesis test is also familiar if you’ve read Chapter 15. Calculate
N is the number of pairs, and the test has N-2 degrees of freedom.
I define Number_of_pairs as the name for the value in cell I2. So I type
=J4*SQRT(Number_of_Pairs-2)/SQRT(1-J4^2)
into J6 and
=T.DIST.2T(J6,Number_of_Pairs-2)
into J7. I use the two-tailed t distribution function because I didn’t know the correlation’s direction in advance. And once again, the low p-value tells me to reject the null hypothesis.
A Heads-Up
A couple of things I’d like you to be aware of. First, I didn’t put any tied ranks in these examples. As it turns out, ties present a few wrinkles for the rank-based statistics (except for the Friedman two-way ANOVA), and I wanted to avoid all that.
Second, be aware that additional non-parametric tests are lurking in the remainder of the book. In Chapter 18, you find hypothesis testing based on the binomial distribution. In Chapter 22, I tell you about a way to test the independence of two nominal variables.