Introduction
Every data set on the SAT comes from somewhere — a survey, an experiment, an observational study. Your job is to judge what conclusions that data actually supports. This skill appears in the Problem-Solving and Data Analysis domain (≈15% of the test) and typically asks you to choose the one conclusion that is both logically supported and appropriately scoped. Getting this right means knowing three rules cold.
Core Concept
Rule 1 — Random Sampling → Generalization to the Sampled Population
When a sample is chosen randomly from a population, you can extend the results to that specific population and no further. The key phrase is the population from which the sample was selected.
A researcher randomly selects 300 adults from voter rolls in Dane County and finds that 62% support a new transit proposal. → You may conclude that roughly 62% of all registered voters in Dane County support the proposal. You may not conclude anything about voters statewide, because the sample only came from Dane County.
The scope of generalization is always bounded by the sampling frame — the list or group from which subjects were actually drawn.
Rule 2 — Random Assignment → Causal Relationship
When subjects are randomly assigned to treatment and control groups in an experiment, any difference in outcomes can be attributed to the treatment itself. Random assignment balances out all other variables (age, health, motivation, etc.) between groups on average, eliminating the influence of confounders.
100 volunteers are randomly assigned to either a new study-skills workshop (treatment) or their regular routine (control). After 4 weeks, the treatment group's test scores rose 10 points more. → Because of random assignment, there is evidence that the workshop caused the score increase.
This is the only design that produces evidence of causation.
Rule 3 — No Random Assignment → Association Only
An observational study records data without manipulating anything. Because subjects are not randomly assigned, any observed relationship between variables could be explained by confounding variables — other differences between the groups that the researcher did not control. The result: observational studies show association, never causation.
| Feature | Result |
|---|---|
| Random sample from population P | Findings generalize to population P |
| Random assignment to groups | Evidence of a causal relationship |
| No random assignment (observational) | Association only — no causation |
| Non-random (biased) sample | Results may not generalize to any broader group |
Biased Samples
A sample is biased when the method of selection makes certain members of the population more or less likely to be chosen. Common biased designs:
- Voluntary response (self-selected): only motivated people respond, skewing results.
- Convenience sample: only easily accessible subjects are included.
- Undercoverage: some parts of the population have no chance of being selected.
Biased samples cannot reliably represent the population, regardless of sample size.
Unlock the full Evaluating Statistical Claims note with Nova
You're reading the preview. Unlock the complete note — every worked example, examiner pitfall and practice question — plus 24/7 AI tutoring from Nova that teaches directly from these notes.