Concluding and evaluating practice questions.

On a bar chart comparing two group means, the error bars overlap considerably. What does this suggest?

• The experiment was carried out incorrectly
• The difference may not be significant
• One mean is certainly much larger than the other
• The data contain no random error at all

Answer: The difference may not be significant

Large overlap of error bars indicates the two means may not differ significantly, so any apparent difference could be due to chance rather than a real effect.

Plants grown in the dark also happened to be kept at a lower temperature than those grown in the light. What problem does this create for the conclusion?

• Temperature is a confounding variable
• There are too many repeats in the data
• The dependent variable was not measured
• The sample size was far too large to analyse

Answer: Temperature is a confounding variable

Because temperature changed along with light, it is a confounding variable, so any difference in growth cannot be attributed to light alone.

A student measured reaction rate only once at each temperature. What is the best way to improve the reliability of the data?

• Repeat each measurement and find a mean
• Test at just one temperature instead
• Use a wider range of temperatures
• Record every result to more decimal places

Answer: Repeat each measurement and find a mean

Repeating readings and averaging reduces the effect of random error and reveals anomalies; extra decimal places or a wider range change precision or scope but not reliability.

A drug trial included only young male volunteers. What is the main limitation of this sample?

• Results may not apply to other groups
• The sample size was far too large
• There were too many repeats taken
• The independent variable was not changed

Answer: Results may not apply to other groups

An unrepresentative sample limits how far conclusions can be generalised; effects seen in young males may not hold for women, older people or other groups.

A test comparing two treatment means returns a p-value of 0.03. What does that value mean?

• A difference this large would rarely arise by chance alone
• There is a 3 per cent chance that the measurements are wrong
• The treatment accounts for 97 per cent of the difference seen
• The result would be repeated in 97 out of 100 further studies

Answer: A difference this large would rarely arise by chance alone

The p-value is calculated on the assumption that there is no real difference, and it reports how often a difference at least this large would then be seen: about three times in a hundred. It says nothing about the size of the effect and nothing about whether the study would replicate.

A balance was never zeroed before an experiment. How would that show in the results?

• Every mass shifted the same way, so the trend still holds
• The masses scattered more widely than they should have
• Only the largest masses would have been affected by it
• The trend would be reversed compared with the true one

Answer: Every mass shifted the same way, so the trend still holds

A constant offset moves all the values together, so differences between treatments and the shape of any relationship survive intact while the absolute values do not. This is why such an error can pass unnoticed in a study that compares groups and be fatal in one that reports a value.