Common logical fallacies and reasoning errors - Sampling and...

Learning Outcomes

After reading this article, you will be able to identify and analyse sampling and survey errors as major logical fallacies on the LSAT. You will be able to spot sampling flaws in arguments, recognise how unrepresentative or biased samples undermine conclusions, and apply appropriate strategies to answer LSAT-style questions focused on these reasoning errors.

LSAT Syllabus

For LSAT, you are required to understand how arguments can fail when they rely on flawed or biased samples. This article focuses your revision on the following points:

• Recognising when a survey or study is used as evidence for a generalisation
• Identifying assumptions about the representativeness of samples
• Analysing how flawed samples affect argument validity
• Approaching LSAT questions that employ or test these specific fallacies

Test Your Knowledge

Attempt these questions before reading this article. If you find some difficult or cannot remember the answers, remember to look more closely at that area during your revision.

1. A study of 15 business owners is cited as proof that all business owners favour a new tax law. What is the main logical flaw in this reasoning?
2. If a survey selects only customers who left positive reviews, what error is likely present?
3. True or false? A sample is always representative if it is large.
4. Which approach best addresses a sampling flaw in a LSAT question?
   a) Accept the results if the survey is recent
   b) Check if the sample is representative of the group claimed in the conclusion
   c) Focus on whether the sample is random
   d) Ignore all details about how the sample was selected

Introduction

Sampling and survey errors are a frequent source of flawed reasoning on the LSAT. Many arguments or studies present statistics or survey data and then draw broad conclusions. For exam purposes, it is essential to determine whether the argument incorrectly assumes the sample is representative of the wider group described in the conclusion.

Key Term: sampling error
An error made when the selected sample does not reflect the relevant characteristics of the overall population, undermining the strength of conclusions drawn from that sample.

Key Term: biased sample
A sample chosen in a way that excludes or overrepresents certain parts of the population, resulting in misleading or unreliable conclusions.

Common Sampling Fallacies

LSAT questions often test your ability to spot specific reasoning errors tied to the use of surveys or statistics. The most typical flaws include:

• Assuming a small or non-random sample can represent a much broader population.
• Drawing general conclusions from survey data that is not shown to be representative.
• Overlooking the method by which the sample is selected, such as self-selection or other bias.

Key Term: representativeness
The degree to which a sample's characteristics match those of the population about which a claim is made.

Worked Example 1.1

A stimulus states: "A survey of 40 shoppers in one luxury mall showed that nearly all prefer brand-name products. Therefore, most Americans prefer brand-name products."

What is the reasoning flaw?

Answer:
The argument assumes the preferences of luxury mall shoppers represent those of all Americans. The sample is both small and selected from a specific group, making it unrepresentative. This undermines the validity of the conclusion.

Spotting Sampling and Survey Flaws

Sampling and survey fallacies are identified by tracing how an argument moves from data about a group (the sample) to a broader conclusion. Key warning signs include:

• Details about the sample that differ from the group in the conclusion.
• Surveys that use narrow, local, or self-selected groups.
• Arguments that generalise from anecdotes ("in my experience...") or a single case.

Key Term: statistical generalisation
Using data from a specific group to make a broad claim about a larger group, often without justification of how that data applies beyond the sample.

Worked Example 1.2

A study claims: "86% of respondents in an online poll support remote working. Thus, the majority of workers want to work from home."

What is the flaw in this argument?

Answer:
The online poll may attract only those who have strong opinions about remote work or spend more time online. It likely excludes many workers without internet access or interest in such polls, creating a bias. The conclusion extends beyond the sample data.

Revision Tip

When evaluating surveys or statistics on the LSAT, always check:

• Who is in the sample?
• How were they selected?
• Does the argument provide any justification for why the sample reflects the group in the conclusion?

Exam Warning

Do not assume that a large sample is necessarily representative. A large, but biased or otherwise unrepresentative sample, still leads to flawed conclusions.

Common LSAT Question Stems and Strategies

For these questions, the correct answer often:

• Challenges or questions the representativeness of the sample.
• Points out that the sample excludes relevant groups.
• Shows a key difference between the sample group and the population referred to in the conclusion.

Use process of elimination to discard answers that don't address the mismatch between sample and conclusion.

Worked Example 1.3

The argument states: "Our restaurant receives high praise from diners who take our feedback survey cards. Therefore, we must provide excellent overall service."

What is the sampling error?

Answer:
Only customers who fill out the feedback cards are considered. This self-selected group may be those with extreme experiences (very positive or negative), not typical diners. The conclusion unjustifiably applies the feedback of a subset to all customers.

Summary

Key Point Checklist

This article has covered the following key knowledge points:

• Sampling and survey errors appear frequently as logical flaws on the LSAT
• The main flaw is unjustified generalisation from a non-representative sample
• Key warning signs include details about sampling methods and group differences
• The correct answer often points out how the sample does not match the broader group claimed in the conclusion
• Large samples may still be biased and lead to flawed arguments

Key Terms and Concepts

• sampling error
• biased sample
• representativeness
• statistical generalisation