Careless Statistics

Each of the statements below is drawn from a real source. Note the potential for misinterpretation due to possible sources of bias or error, and omissions of important facts.

  1. In a study of schoolboys, an educator discovered a positive correlation between hand size and quality of handwriting. (Age is the causal factor which drives both hand size and handwriting quality.)
  2. A recent report showed a strong relationship between the incomes of adults, and the quality of their childhood education (measured by such statistics as student-teacher ratios and teacher salaries). (But perhaps family and neighborhood affluence drive both educational quality and economic success.)
  3. One-third of the women students at Johns Hopkins University during its first year married faculty members.(There were only three coeds that year.)
  4. In order to justify a proposed rent increase, a landlord indicated that his maintenance expenses have risen by a greater percentage than his rental income over the past few years. (But his maintenance expenses may be a negligible fraction of his total costs.)
  5. A school administered a series of comprehensive standardized examinations to a class of students at the ends of two successive years. It was found that the students who did best on the tests at the end of the first year tended to not maintain that level of performance on the second-year tests, and it was therefore concluded that the second-year teacher failed to adequately stimulate the better students. (Or, this was simply regression to the mean.)
  6. A union claimed that the average monthly earnings of plant employees had fallen 8 percent in the past year. But management showed that every employee was making more than he/she was the year before. (Senior employees had left the firm, and new, younger, lower-paid workers had been brought in.)
  7. From 1984 to 1985, the national mean SAT quantitative-skills score dropped (and a number of editorials lamented the decline in U.S. educational standards). Yet the mean score for “whites” increased, as did the mean score for “non-whites.” (The proportion of non-whites taking the exam increased; non-whites had lower average scores in both years.)
  8. A North Carolina study of smoking habits revealed that moderate smokers have a greater life expectancy than either heavy smokers or nonsmokers of the same age. The obvious conclusion is that a little smoking is good for you. (The “nonsmoker” group lumped together both elective (healthy) nonsmokers, and forced nonsmokers with chronic health problems.
  9. The president of an institution proposed to set a minimum salary of $12,000 for employees in a particular category. His assistant found that there were 250 employees in this category, and their average annual salary was $11,000. Therefore, he predicted a payroll increase of about $250,000. (If half were making $13,000, and half $9,000, the increase would be $375,000.)
  10. A Minnesota study has shown that 61 percent of those involved in accidents have spent more than 10 years behind the wheel. The study also showed that 21 percent of those involved in accidents had six to ten years’ driving experience, and 17 percent, one to five. “Apparently drivers become more complacent about their driving as the years go by,” said Spaulding Southhall of the National Chicago Safety Conference. “As a consequence their records become worse.” (Or maybe there are simply many more drivers with more than ten years of experience.)
  11. A random sample of 30 children is selected at a school, and each child is asked the size of his/her family. The average of the resulting responses is taken as an estimate of mean family size for families represented in the school. (Large families, with several children in the school, are much more likely to be represented in the sample than are smaller families.)
  12. A painting contractor observed that, on the contracts he obtained, his actual costs tend to exceed his prior estimates. He began to worry that his cost-estimation procedures were faulty. (He may be suffering from the “Winner’s Curse” – He only obtains those contracts on which his estimate is lower than those of the other contractors bidding for the job.)
  13. The final report of the 1950 U.S. census contained a table which classified students by age and school grade. This table indicated that, of the students in the first year of elementary school, the vast majority were between five and nine years of age. However, 7,790 were 13-year-olds, 12,765 were 14-year-olds, and 8,885 were 15-year-old. (The census questionnaire was badly worded, and a number of “first year in high school” responses were miscoded in the field as “first year in school.”)
  14. The same census reported 1670 14-year-old widowers, but only 810 17-year-old widowers. (Systematic keypunching errors – an omitted column early in the coding – turned household heads in their 40’s (respectively, in their 70’s) into widowers in their teens (14 and 17, respectively). There were more opportunities for error among the (larger) first group.)