Observational Studies vs. Experiments (Topic 3.2)
Chapter 4 - Day 6
Unit 3
Chapter 4
Day 1
Day 2
Day 3
Day 4
Day 5
Day 6
Day 7
Day 8
Day 9
Day 10
Day 11
Day 12
Day 13
Day 14
All Units
Learning Targets
-
Explain the concept of confounding and how it limits the ability to make cause-and-effect conclusions.
-
Distinguish between an observational study and an experiment, and identify the explanatory and response variables in each type of study.
-
Identify the experimental units and treatments in an experiment.
Activity: Does SAT Prep Produce Higher Scores?
Activity:
Answer Key:
When trying to decide if the SAT prep class caused an improvement in scores, we must recognize that those students who sign up for a prep class are likely different than the rest of the student population. They may be harder working, have more time for studying, and care more about their SAT score than those students who did not sign up. We don’t know if the difference in the average SAT scores (1220 vs 1050) is because of these confounding variables or because of the SAT prep class. We suggest making a visible list of these confounding variables.
Some students will suggest that there should be an SAT taken before and after the prep class. This is a matched pairs design and will be discussed later. For now, we want to focus on basic experimental design.
Students will likely use a paragraph to explain how they would design an experiment. In the debrief of the activity, show them an outline of the experiment:
When students draw these outlines, they often want to skip the step that shows “Group 1” and “Group 2”. This step is very important to show the purpose of the random assignment. The random assignment is hopefully equally distributing the confounding variables into the two groups. Some hard working students go in Group 1 and some in Group 2, and the same for all other confounding variables. When the treatment is applied to each group, the groups are now different…and the only difference between the groups is the treatment. So if there is a significant difference in SAT scores between the two groups, we can say that the SAT prep class caused the higher scores. In a nutshell, random assignment allows us to show causation.
Random Sample vs Random Assignment
These two concepts allow us to make different conclusions.
Random Sample: A random sample should be representative of the population from which it was taken, so a random sample allows us to generalize our conclusion to the population.
In the SAT prep class example, we have 44 student volunteers. Thus we cannot generalize our conclusion to all students. Instead we can only make a conclusion for these 44 students or other students like the ones who volunteered for the experiment.
Random Assignment: Random assignment hopefully will equally distribute the various levels of the confounding variables into the treatment groups, so that only difference between groups is the treatment. If there is a significant difference between the groups, the random assignment allows us to conclude that the treatment caused the difference.
Luke's Lesson Notes
Here is a brief video highlighting some key information to help you prepare to teach this lesson.