Leigh Nataro teaches elementary statistics and business calculus at Moravian College in Bethlehem, PA. Leigh has been an AP Exam Reader and Table Leader and was on the AP Statistics Instructional Design Team, where she helped to tag items for the AP Classroom question bank. In addition to leading AP Statistics workshops, Leigh is also a Desmos Certified Presenter. Leigh can be reached on Twitter at @mathteacher24.
We often teach students about Type I and Type II errors after they have learned about conducting a full significance test. The Course and Exam description for AP Statistics describes these errors in a chart like the one shown here:
Although this chart helps students see the four possibilities of what could happen with significance test conclusions, I find my students don’t confuse the errors when they learn the following two statements basic statements:
Type I error: Rejecting null, when null is true
Type II error: Not rejecting null, when null is false
If students can recite what a Type I error is, then a Type II error is simply the opposite.
However, if students do not fully understand the meaning of “reject the null hypothesis” and “fail to reject the null hypothesis” when drawing a conclusion for a significance test, they can become very confused when trying to describe Type I and Type II errors for a given scenario.
I ran into this exact situation a few years ago while leading a Saturday session for NMSI in New York City. Some of the students had not done significance testing at all yet and that made it even more challenging to get the students to understand these ideas. To introduce them to the concepts of Type I and Type II errors, I created an example that most teenagers could relate to and it was devoid of any statistical context. Here is that example:
If I had done a good job of teaching Cassie the importance of oral hygiene and I had seen her brush her teeth several times successfully by herself, I would naturally assume the null hypothesis was true when I asked her if she brushed her teeth. But what if Cassie would rather watch her favorite show on tv or continue playing with her toys? She could tell me that she brushed her teeth, when in fact she did not. If I assumed she brushed her teeth when she did not, I would have made an error. This would be a Type II error, not rejecting the null hypothesis when the null hypothesis is false.
Could I have made another error in this scenario? The students are quick to say that I could have said Cassie did not brush her teeth when she did. This is rejecting the null, when the null is true, which would be a Type I error.
Finally, I ask the students what evidence would lead me to reject the null in favor of the alternative? In other words, what evidence would lead me to think Cassie did not brush her teeth.
Here are some ideas they generated:
● Cassie’s breath is not minty fresh.
● Cassie was in the bathroom for less than 30 seconds.
● I could not hear the water in the bathroom being turned on.
● Cassie’s toothbrush is dry.
● There is not a big glob of toothpaste in the sink. (This one I suggested.)
We talk about the fact that the first two pieces of evidence - minty fresh breath and 30 seconds - could lead us to believe that Cassie did not brush her teeth, when in fact she did brush them, albeit poorly. We would reject the null hypothesis when the null hypothesis was true and make a Type I error.
Next, we talk about the consequences of the errors. Since this is an example that students can relate to, it is easy for them to identify consequences. A Type I error consequence could be accusing Cassie of lying when in fact she did not lie. This could lead to a lack of trust. A Type II error would be not catching Cassie in a lie and the potential consequence could be cavities!
Student Practice
It is also important for students to see Type I and Type II errors in a research lens and I share the following with them:
When I teach this concept, we spend a full fifty-minute class period working on identifying Type I and Type II errors and their consequences. All of the problems we consider are devoid of statistical context and this helps the students to focus on just understanding and identifying Type I and Type II errors. During the spring of 2020, we switched to synchronous virtual classes and I had students work in breakout rooms on a copy of this Google document. This allowed me to identify in real-time any groups that were struggling. It also gave me a set of student work to share when we debriefed the activity near the end of class. An answer key for the document can be found here.
ACTIVITY DOWNLOADS: Identifying Type I and Type II Errors
Answer Key: PDF