Evaluating statistical claims, observational studies, and experiments: Worksheet with Answers

Researchers recruited 39 willing participants from among volunteers at a community centre; this was not a random sample. Participants chose whether to use a new practice schedule. Researchers observed a statistically significant association with task accuracy. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.
The study establishes causation. Generalization to the stated population is not justified by this sampling method.
The study supports an association, but does not establish causation. The findings automatically generalize to the stated population.
The study establishes causation. The findings automatically generalize to the stated population.

Researchers recruited 36 willing participants from among households in a town; this was not a random sample. Participants chose whether to use a reminder message. Researchers observed a statistically significant association with water use. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.
The study establishes causation. Generalization to the stated population is not justified by this sampling method.
The study supports an association, but does not establish causation. The findings automatically generalize to the stated population.
The study establishes causation. The findings automatically generalize to the stated population.

Researchers recruited 33 willing participants from among adult runners in a city; this was not a random sample. Participants chose whether to use a new warm-up routine. Researchers observed a statistically significant association with running speed. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.
The study establishes causation. Generalization to the stated population is not justified by this sampling method.
The study supports an association, but does not establish causation. The findings automatically generalize to the stated population.
The study establishes causation. The findings automatically generalize to the stated population.

Researchers recruited 30 willing participants from among students at a university; this was not a random sample. Participants chose whether to use a new study method. Researchers observed a statistically significant association with exam scores. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.
The study establishes causation. Generalization to the stated population is not justified by this sampling method.
The study supports an association, but does not establish causation. The findings automatically generalize to the stated population.
The study establishes causation. The findings automatically generalize to the stated population.

Researchers randomly selected a representative sample of volunteers at a community centre. Participants chose whether to use a new practice schedule. Researchers observed a statistically significant association with task accuracy. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.
The study establishes causation. The findings can be generalized to the stated population.
The study supports an association, but does not establish causation. Random sampling prevents generalization to the stated population.
The study establishes causation. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of households in a town. Participants chose whether to use a reminder message. Researchers observed a statistically significant association with water use. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.
The study establishes causation. The findings can be generalized to the stated population.
The study supports an association, but does not establish causation. Random sampling prevents generalization to the stated population.
The study establishes causation. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of adult runners in a city. Participants chose whether to use a new warm-up routine. Researchers observed a statistically significant association with running speed. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.
The study establishes causation. The findings can be generalized to the stated population.
The study supports an association, but does not establish causation. Random sampling prevents generalization to the stated population.
The study establishes causation. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of students at a university. Participants chose whether to use a new study method. Researchers observed a statistically significant association with exam scores. Which conclusion is best supported by the study design?

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.
The study establishes causation. The findings can be generalized to the stated population.
The study supports an association, but does not establish causation. Random sampling prevents generalization to the stated population.
The study establishes causation. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of volunteers at a community centre. Participants were randomly assigned to use a new practice schedule or to a control group. A statistically significant difference in task accuracy was observed. Which conclusion is best supported by the study design?

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.
Random assignment provides no basis for a causal comparison. The findings can be generalized to the stated population.
The experiment supports a causal conclusion for its participants. Random sampling prevents generalization to the stated population.
Random assignment provides no basis for a causal comparison. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of households in a town. Participants were randomly assigned to use a reminder message or to a control group. A statistically significant difference in water use was observed. Which conclusion is best supported by the study design?

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.
Random assignment provides no basis for a causal comparison. The findings can be generalized to the stated population.
The experiment supports a causal conclusion for its participants. Random sampling prevents generalization to the stated population.
Random assignment provides no basis for a causal comparison. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of adult runners in a city. Participants were randomly assigned to use a new warm-up routine or to a control group. A statistically significant difference in running speed was observed. Which conclusion is best supported by the study design?

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.
Random assignment provides no basis for a causal comparison. The findings can be generalized to the stated population.
The experiment supports a causal conclusion for its participants. Random sampling prevents generalization to the stated population.
Random assignment provides no basis for a causal comparison. Random sampling prevents generalization to the stated population.

Researchers randomly selected a representative sample of students at a university. Participants were randomly assigned to use a new study method or to a control group. A statistically significant difference in exam scores was observed. Which conclusion is best supported by the study design?

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.
Random assignment provides no basis for a causal comparison. The findings can be generalized to the stated population.
The experiment supports a causal conclusion for its participants. Random sampling prevents generalization to the stated population.
Random assignment provides no basis for a causal comparison. Random sampling prevents generalization to the stated population.

Answer key and explanations

1. Answer

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.

Without random assignment, other differences between the groups may explain the association. A volunteer sample alone does not justify generalization to the entire population.

2. Answer

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.

Without random assignment, other differences between the groups may explain the association. A volunteer sample alone does not justify generalization to the entire population.

3. Answer

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.

Without random assignment, other differences between the groups may explain the association. A volunteer sample alone does not justify generalization to the entire population.

4. Answer

The study supports an association, but does not establish causation. Generalization to the stated population is not justified by this sampling method.

Without random assignment, other differences between the groups may explain the association. A volunteer sample alone does not justify generalization to the entire population.

5. Answer

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.

Without random assignment, other differences between the groups may explain the association. A representative random sample supports generalization to the population sampled.

6. Answer

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.

Without random assignment, other differences between the groups may explain the association. A representative random sample supports generalization to the population sampled.

7. Answer

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.

Without random assignment, other differences between the groups may explain the association. A representative random sample supports generalization to the population sampled.

8. Answer

The study supports an association, but does not establish causation. The findings can be generalized to the stated population.

Without random assignment, other differences between the groups may explain the association. A representative random sample supports generalization to the population sampled.

9. Answer

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.

Random assignment reduces confounding and supports a causal comparison. A representative random sample supports generalization to the population sampled.

10. Answer

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.

Random assignment reduces confounding and supports a causal comparison. A representative random sample supports generalization to the population sampled.

11. Answer

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.

Random assignment reduces confounding and supports a causal comparison. A representative random sample supports generalization to the population sampled.

12. Answer

The experiment supports a causal conclusion for its participants. The findings can be generalized to the stated population.

Random assignment reduces confounding and supports a causal comparison. A representative random sample supports generalization to the population sampled.