Looking for a creative way to teach ANOVA

I taught a class like yours when I was a graduate student over 30 years ago. I remember how I taught some of it, but not exactly how I taught ANOVA. I was happy with the class, and the students were happy, too. They brought refreshments on the last day of class, and gave me a card with kind comments on it. They were happy because most were afraid of taking a statistics class, but were comfortable in class and understood more than they thought they would. Since it was a successful class, I’ll share how I might have taught the lesson on ANOVA.

At that time, for a college class, I did mostly lecture. But I did encourage questions a lot, and if there were no questions I would sometimes say, “If you have no questions, then I’ll ask you questions,” and get someone to explain an important point. This is one way that I highlighted important points. When lecturing I might ask a question or pose a hypothetical situation for students to consider before explaining, because we process information better when it is used to answer a specific question or resolve a specific problem. If I asked a question when lecturing, we might or might not have had some whole class discussions, but less than I would have now. (The class included about 30 students.) In the spirit of collaboration, others can suggest revisions to improve this lesson and to create different versions.

I tried to get students to understand the concepts, not just to do calculations. A class in ANOVA would build on understanding of concepts taught earlier, primarily:

1. The mean as the middle of a distribution and the best estimate of a set of scores based on the “least squares” criterion. It is computed as the sum of scores divided by the number of scores.

2. The deviation score as the distance a score is above or below the mean. It is computed as scoreminus mean (x minus x-bar). This is a simple but key concept that shows how the mean is in the middle of interval data, since the sum of the positive deviations equals the sum of the negative deviations. It is the basis for the least squares criterion, where error is measured as the sum of squared deviations from the estimate. The error is lower when computed from the mean than from any other point. The deviation score is also the basis for computing variance.

3. Variance as a measure of dispersion, the extent that scores are spread away from the middle of the distribution. It is also a measure of error if the mean is taken as the estimate for a group. It is computed as the mean of the squared deviations (the “mean-square”): the sum of squared deviations divided by the number of scores, or by the degrees of freedom if estimating the population variance from sample data.

4. The logic of hypothesis testing including especially the concepts of the null hypothesis, the experimental hypothesis, alternative hypotheses which are eliminated by a good research design, and alpha error. I remember using an analogy of trying to disprove someone’s claim that he was psychic. Everyone could understand that process, so mapping the terms of hypothesis testing to aspects of that process made the terms easy to understand.

Before doing any computation, I want to highlight several concepts: the mean of each group as the value that represents the score for that group, the variance within a group as how good an estimate its mean is as a group score, and the variance of the group means (between-group variance) as a measure of how spread apart the groups are from the grand mean and from each other.

Then I want the students to think about, if they were researchers doing a study with three groups, each getting a different treatment, and they expected each group to be significantly different from each other, would they want the within group variances to be larger or smaller, why? Would they want the between group variance to be larger or smaller, why? I would expect they would want the within groups variances to be smaller because a tighter distribution for each group results in less overlap between group scores. The larger the within variance, the less accurate is the mean as a score representing its group, and a larger separation between means is needed to have confidence in group differences. They would want the between group variance to be larger because that happens when the means of the groups are spread farther apart. (I want the students to think about it, but this is straight forward so I wouldn’t have spent much time on discussion. I would have quickly explained or summed up the discussion.)

Next I want the students to think about the within group variances relative to the between group variance. I ask the students to visualize the within group variances staying the same, but the between group variance starting small then getting larger and larger. I ask the students to describe what they see. I expect that they would say the groups of scores move apart, being greatly overlapped at first then separating into more distinct groups. At some point the groups become significantly different. (I would spend a little more time on discussion of the relative sizes of the variances, on this and the next questions before lecturing on the calculations, because this is conceptually important and more complex than the first set of questions.)

If there is time, I would ask the class to visualize the reverse situation--the within group variance staying the same this time, meaning the means of the groups stay the same, but the within group variances starting out very large then becoming smaller and smaller. What do they see?

Before doing computations, I would ask the class, “If you did a study and the means of the three groups were farther apart than you expected, with two treatment groups doing better than the control group to a surprising extent, and one treatment group doing much better than the other, you would expect the statistical test to be significant. Thinking in terms of the variances, is there a situation where the statistical test is NOT significant despite the large differences in group means?” I would hope that, after the visualizations that we did of the variances changing relative to one another, the students would say that if the within group variances were extremely large, then seemingly large differences in group means may be relatively small, and there would be much overlap in group scores. For group scores to be significantly separated, the between group variance, and therefore the differences among group means, should be large relative to the within group variances, which measures the accuracy of the means as the groups’ scores. This would be a logical segue to the F-statistic, which I would explain to the class is what is used in ANOVA.

The questions and limited discussions get the students to understand the issues. I would have then just lectured on partitioning the variance and doing all the calculations, but relating it to the concepts and the discussion. I think the calculations make sense and are easier to remember when the issues are discussed first conceptually, without any numbers attached. One point in particular that would need emphasis is using degrees of freedom instead of n-size to compute a sample variance as an estimate of population variance. Otherwise, the means and sum of square calculations are similar to what they had already done in earlier parts of the class. The F-statistic calculation should make sense after discussing the importance of the relationship between the between-group and within-group variances. Calculations were shown in the textbook, and I don’t remember doing any handouts, but it would make sense to have a handout with all the calculations done for a small set of data, and a table for the F-statistic with the degrees of freedom and critical values of F highlighted for the particular data contained in the handout.

Then I think it is important to review effect size, n-size and the power of the test, and the difference between statistical significance and practical significance—all of which might have been covered with hypothesis testing in the introduction to inferential statistics. I don’t know that I actually did a review of these points in that class I taught (probably not), but I think it is more important for students to know these factors that affect the results of a study than it is to know how to compute the F-statistic from raw data, since they would use a computer to do the calculations if they actually analyzed data. These factors that affect the result of a study are important to know if they actually did a study using ANOVA, or even if they only need this knowledge to evaluate studies in their field.

The class always had homework, so they would have had to do computations from raw data. I had a system where they turned in their homework at the department office and I reviewed and wrote comments on each one, and returned them by the next class. Classes were three times a week.

I remember that I gave an optional extra credit assignment to copy the Results section of a journal article in their field and to discuss the article in terms of what we covered in class, to demonstrate their understanding of the course material.

If I taught the class now, I would incorporate what technology we had available in the class. I would also try to take a few minutes for small group work to get the students to think more deeply about real world issues related to statistical testing, especially for the social science graduate students in the class. Because of time constraints, they might first discuss the question in class, but then write up their responses as homework or for optional extra credit. For example, I would refer back to the last hypothetical situation regarding large group differences that were not statistically significant. “If you did a study and your group differences were large but not statistically significant, and you were confident in your theory and thought the study should be redone, what changes would you consider to improve the study? Remember what we discussed about why large group differences might not be statistically significant.” They might research it online and that’s fine. My goal is to get them to think more deeply about the issues, not necessarily come up with solutions all on their own.