This blog post was written by Luke Wilcox, who was a Table Leader for Question #2.
A developer wants to know whether adding fibers to concrete used in paving driveways will reduce the severity of cracking, because any driveway with severe cracks will have to be repaired by the developer. The developer conducts a completely randomized experiment with 60 new homes that need driveways. Thirty of the driveways will be randomly assigned to receive concrete that contains fibers, and the other 30 driveways will receive concrete that does not contain fibers. After one year, the developer will record the severity of cracks in each driveway on a scale of 0 to 10, with 0 representing not cracked at all and 10 representing severely cracked.
Part (a)
Based on the information provided about the developer’s experiment, identify each of the following: experimental units, treatments, response variable.
WOULD THIS GET CREDIT?
Experimental units are the individuals (which may be people or other objects of study) that are assigned treatments. In this scenario, the treatments are not actually being assigned to the new homes, they are being assigned to driveways. For the treatments, consider what is actually being given to the experimental units (driveways). Each driveway is not receiving just “fibers”, but more specifically, “concrete with fibers”. For the response variable, we are looking for the outcome that is being measured after the treatments have been administered. The response “cracks or no cracks” is a categorical variable, which is incorrect. The outcome actually being measured in this experiment is the quantitative variable of “severity of cracking”.
Teaching Tips:
Practice vocabulary when teaching experimental design. It is often better to discuss these terms in a context rather than simply having students memorize definitions.
Require students to be specific in their language on free response.
Teach students the difference between categorical and quantitative variables early in the course and come back to the idea often. One place to remind students of this idea later in the course is with inference, as we use proportions for categorical variables and means for quantitative variables.
Part (b)
Describe an appropriate method the developer could use to randomly assign concrete that contains fibers and concrete that does not contain fibers to the 60 driveways.
The rubric is looking for two components
(1) Describe how to correctly use a random process to assign treatments, such that every possible random assignment is equally likely.
(2) The random process results in an equal number of driveways assigned for each treatment.
WOULD THIS GET CREDIT?
Response 1:
Label each driveway with a number from 1 to 60. Use a random number generator to get 30 numbers and assign concrete with fibers to the homes with labels that match these numbers. The rest of the driveways get concrete with no fibers.
Response 2:
Label each driveway with a number from 1 to 60. Write these numbers on identical slips of paper and put them in a hat. Grab 30 slips of paper and assign concrete with fibers to the homes with labels that match the selected addresses. The rest of the driveways get concrete with no fibers.
Response 3:
Go to each driveway and toss a coin. If the coin lands heads, the driveway gets concrete with fibers, and if the coin lands tails, the driveway gets concrete without fibers.
Response 4:
Go to each driveway and toss a coin. If the coin lands heads, the driveway gets concrete with fibers, and if the coin lands tails, the driveway gets concrete without fibers. Once one of the treatment groups gets 30 driveways, all other driveways go in the other treatment group.
Response 5:
Use a random number generator to assign the numbers 1-60 (no repeats) to the 60 driveways. The driveways assigned 1-30 get concrete with fibers and the those assigned 31-60 get concrete without fibers.
Response 1 is the random number generator approach. It is missing two necessary pieces: (1) The boundary values of the random number generator must be specified i.e. “get 30 numbers between 1 and 60” and (2) it is missing the idea that numbers must be selected without repeats i.e. “no repeats” or “unique” numbers.
Response 2 is the names-in-a-hat approach. It is missing two necessary pieces: (1) The papers must be “mixed well” and (2) the slips of paper are selected “without replacement”.
Response 3 is the coin-toss approach. It receives credit for the first component because each random assignment is equally likely, but it loses credit for the second component because it does not guarantee an equal number of driveways to get each treatment.
Response 4 is the coin-toss-fill-until-full approach. This response loses credit for the first component because every random assignment is not equally likely (it’s more likely that the last few driveways end up in the same group) and earns credit for the second component because it correctly guarantees equal-sized groups.
Response 5 earns full credit because it (1) describes a random process such that every possible random assignment is equally likely and (2) it results in an equal number of driveways to each treatment.
Teaching Tips:
For full credit in describing how to do random assignment, the explanation must be clear enough that a non-statistical person could read it and correctly carry out the procedure in the same way.
The random number generator approach (or random digits table) needs (1) labels for experimental units, (2) boundaries for random numbers, and (3) without replacement.
The hat approach needs (1) labels for experimental units, (2) mix well, and (3) without replacement.
Part (c)
Suppose the developer finds that there is a statistically significant reduction in the mean severity of cracks in driveways using the concrete that contains fibers compared to the driveways using concrete that does not contain fibers.
(c) In terms of the developer’s conclusion, what is the benefit of randomly assigning the driveways to either the concrete that contains fibers or the concrete that does not contain fibers.
The rubric is looking for two components
(1) Random assignment allows a conclusion about causation.
(2) The response is in context.
WOULD THIS GET CREDIT?
Response 1 sounds like causation, but was NOT given credit because it was mostly just copying words from the prompt.
Response 2 was given partial credit because it discussed confounding variables in context, but did not make an argument for causation.
Responses 3 and 4 were both given credit, as the language of “caused” and “attributed to fibers” were strong enough arguments for causation.
Teaching Tips:
Be sure students understand the difference between random sampling (which allows generalization to a larger population) and random assignment (which allows us to conclude causation) – see this lesson.
When checking conditions for formal inference, consider having students check the condition AND provide the “so what”. Random sample --> so we can generalize to the population. Random assignment --> so we can show causation.