Unit 5 · Data Collection and Cleaning
5-1 Methods of Data Collection
Before you can trust what data tells you, you have to trust how it was gathered. Good analysis moves through three stages — collection, then organization, then analysis — and if the wrong data is collected at the start, no careful work later can rescue it. Some data you gather yourself, for your exact purpose; some already exists, collected by other people. And when a whole group is too large to ask, you ask a part of it — but only a fair part lets you speak for the whole.
Guiding question
How do you collect data you can trust, and how can a small sample fairly represent a whole population?
Learning outcome. Tell primary data from secondary data and say when each one is the right choice, explain how sampling works, and explain how a biased sample can give a confident but wrong answer.
1 · The data collection stage
Data analysis goes in this order, after the purpose has been made clear:
Data collection is the first stage. In it you plan what data will be gathered and how, in line with the purpose of the analysis.
2 · Why collection decides everything
If data that does not match the purpose is collected, it is impossible to draw appropriate conclusions — no matter how carefully the organization and analysis are carried out afterwards. That is why collection is the stage that decides the quality of the whole analysis.
1 · Primary data
Primary data — data that is newly collected to fit a specific purpose. For example: surveys, interviews, observations, experiments.
Strength
The items collected can be designed to match the purpose, giving excellent originality and reliability.
Constraint
Collection takes time, effort and cost.
2 · Secondary data
Secondary data — data that has already been collected and made public by others, and is used in analysis. For example: open datasets, existing statistics, industry reports.
Strength
Large amounts of data can be obtained in a short time.
Constraints
It may not perfectly match your purpose, and bias or omissions may have arisen while someone else edited or summarized it.
3 · Using both together
Primary data is high-value — it can be collected to match your own purpose, so it has excellent reliability and originality, and it can serve as the core of the analysis. The value of secondary data is different: it lets you easily use large-scale data and existing statistics that primary data alone could never give you.
In real analysis the two are combined, depending on the purpose and the constraints. It is not a choice of one or the other.
1 · Population and sample
The entire group that is the target of the analysis is the population. The part of the population that is actually surveyed is the sample.
Complete survey
Surveying the entire population. Also called a census.
Sample survey
Surveying a chosen sample.
2 · Sampling
Sampling is selecting a sample from the population. Depending on the sampling method, the reliability of the conclusions varies greatly.
When surveying the whole group is difficult, examine a sample to estimate the whole. It matters that the sample is a miniature of the population — if there is bias, accurate inference is not possible.
3 · Random sampling
A typical technique is random sampling. It makes it likely that the sample represents the population, which is what allows the results to be applied to the population as a whole.
| Method | Description |
|---|---|
| Simple random sampling | Choose a sample from the population purely by chance, without bias |
| Stratified sampling | Divide the population into strata such as gender or age group, and randomly choose from each stratum |
4 · Which one to use
Simple random sampling is the most basic method and can be used even when little is known about the population. Stratified sampling can reflect the make-up of the population (such as the ratio of gender and age group) inside the sample, making it easier to reduce bias when the differences between strata affect the analysis.
Build a sample that mirrors the school
In stratified sampling you split the school into groups (strata) and choose from each one. Set how many students you take from each group. Try to make your sample a small copy of the school.
Your share
33% vs 50% real
Your share
33% vs 35% real
Your share
33% vs 15% real
Your estimate is 35.00 minutes. Your shares do not match the school’s yet, so this is not a miniature of the population. Keep adjusting.
Notice what this does not depend on: how many students you take in total. 2 · 7 · 3 works as well as 10 · 7 · 3. What matters is the shares.
1 · Sampling bias
Sampling bias occurs when the method of selecting a sample is biased and does not correctly reflect the actual state of the population. When sampling bias is present, the conclusions cannot be applied to the population as a whole.
Watch it happen — the arithmetic stays correct the whole time
How long does a student at this school travel to get here?
There are 240 students. You cannot ask all of them, so you will ask 30. Choose how you pick those 30.
Press the button. Then try the same thing with a different method and compare.
2 · The three typical biases
Typical examples of bias. Self-selection and time/location bias are about who is surveyed (selection bias). Response bias is about how those surveyed answer (measurement error).
| Type of bias | Description | Example |
|---|---|---|
| Self-selection bias | Bias caused by only those who want to respond actually responding | In a web survey, only people with high interest respond |
| Response bias | Not a selection problem: people already surveyed give answers that look desirable rather than accurate | Reporting a “study time” longer than it really was |
| Time/location bias | Bias caused by the timing or location of the survey being limited | Surveying in front of a station on weekday afternoons, missing office workers and students |
3 · Reducing it
It is difficult to eliminate sampling bias completely, but it can be reduced by carefully designing the survey: how subjects are chosen, how questions are worded, and how the survey is conducted.
You now know what stratified sampling is. Let us actually build one, with small numbers, so you could do it in an exam.
🧮 Choosing 60 students to speak for a school of 600
- 1
The school has 600 students: 300 in Grade 10, 200 in Grade 11, 100 in Grade 12. You can only ask 60.
Why this step: The 600 are the POPULATION — the whole group you want to know about. The 60 are the SAMPLE — the part you actually ask. Getting these two words the right way round is worth an easy mark.
- 2
Work out what fraction of the school you are asking: 60 ÷ 600 = 0.1, which is one tenth.
Why this step: Every group must be represented in the same proportion as it appears in the school. Finding that one fraction first means you only calculate it once.
- 3
Take one tenth of each group: Grade 10 → 300 × 0.1 = 30. Grade 11 → 200 × 0.1 = 20. Grade 12 → 100 × 0.1 = 10.
Why this step: This is what “divide the population into strata and randomly choose from each” means in practice. Each grade is a stratum, and each keeps its real share of the school.
Most people slip here: Do NOT take 20 from each grade. Equal numbers from unequal groups is the commonest mistake — it would give Grade 12 (only 100 students) the same voice as Grade 10 (300 students), which misrepresents the school.
- 4
Check they add up: 30 + 20 + 10 = 60. ✓
Why this step: A quick check that costs seconds and catches an arithmetic slip. If the parts do not total your sample size, something has gone wrong.
- 5
Now choose those 30, 20 and 10 AT RANDOM from within each grade.
Why this step: This is the step people forget, and without it the whole thing collapses. Stratified sampling is random sampling done inside each stratum — if you pick the first 30 Grade 10 students on the register, or your friends, the proportions are right and the sample is still biased.
Where the comparison breaks down: You can stir a soup; you cannot stir a population. That is exactly why sampling needs a method — random or stratified — to do the job that stirring does in the pot.
The reason is the whole of this lesson. The magazine drew its names from telephone directories and car-registration lists — and in 1936 those were lists of people who could afford telephones and cars. The sample was not the population; it was the wealthier part of it. Two and a half million of the wrong people is still the wrong people.
The Literary Digest presidential poll of 1936, the standard example of sampling bias.
How the real paper is built
The ministry sets half multiple choice and half written answer in every subject. The two halves are not the same skill. In the multiple-choice half you have to spot the wrong options quickly — and the wrong options are built out of the mistakes students really make. In the written half nobody gives you the answer to recognise; you have to produce it, in your own words, with a reason.
Practise both. Being good at one half and weak at the other costs you exactly half the paper.
- data collection
- — the first stage: plan what to gather and how.
- primary data
- — collected fresh, by you, for this purpose.
- secondary data
- — already collected and made public by others.
- population
- — the whole group being analysed.
- sample
- — the part actually surveyed.
- complete survey
- — surveying everyone; also called a census.
- sampling
- — selecting a sample from the population.
- simple random sampling
- — picked purely by chance, without bias.
- stratified sampling
- — split into strata, then pick from each.
- sampling bias
- — an unfair sample that misrepresents the whole.
Worked Example
(1) Mark each statement with ○ if it is true or × if it is false.
A Primary data refers to data that has already been collected and made public by others.
B Selecting a sample from a population is called sampling.
C Stratified sampling is a method in which the population is divided into strata, and a sample is randomly chosen from each stratum.
D When sampling bias is present, the conclusions obtained may not be applicable to the population as a whole.
(2) Match each description with a sampling method: A Stratified sampling, B Simple random sampling.
a Choose a sample from the population by chance, without bias.
b Divide the population into strata such as gender or age group, and randomly choose from each stratum.
1 · Answer the following questions.
2(1) · Which is the most appropriate method of collecting primary data?
Mark ○ for the one that collects primary data, and × for the ones that reuse someone else’s.
Use statistical data published by a government agency.
Create a survey to fit one’s own purpose and collect responses.
Cite figures from a report issued by an industry association.
Use survey data from a study conducted earlier by someone else as is.
2(2) · Choose the most appropriate sampling method for each survey.
Match each description with the choice that fits it.
Investigating study time across all students in a school, taking into account differences in tendencies between grades
Grasping the opinions of all eligible voters in a city without bias
2(3) · Which is an example in which sampling bias has occurred?
Mark × for the one where bias has occurred, and ○ for the ones that are sound.
Surveying 100 students randomly chosen from all students in the school by drawing lots.
Dividing all residents into age-group strata and surveying a randomly chosen sample from each stratum.
Approaching passers-by in front of the station on weekday afternoons and reporting the results as “the opinions of all citizens.”
Designing the wording of a survey question carefully so that respondents can answer truthfully.
Think as an Engineer — Investigate & Decide
Your school wants to know how long students typically spend commuting to school each day, but you cannot survey everyone. You design how to choose and question a sample.
1 · Collect data (plan the sample).
Decide how many students you would ask and how you would pick them so the sample fairly represents the whole school. As a quick pilot, tally the commute of 8–10 classmates. State the population and the sample clearly.2 · Analyze the risk of bias.
Name one group your method might silently drop — for example students who were absent that day, or students who never open an online survey link. Explain how missing that group would skew the estimated commute, and say which bias it is closest to (self-selection, response, or time/location).3 · Decide.
Choose simple random or stratified sampling for this survey and justify the choice from the risk you found in step 2 — think about whether commute really differs between identifiable subgroups here. In pairs, compare methods and agree on the single change that would most reduce bias.
In a New Context
A shopping mall surveys only visitors on Saturday afternoon and reports the results as “what all our customers think.” Name the bias, explain who is missed, and describe one change that would make the sample fairer.
Lesson Question — Answered
Trustworthy analysis begins with collection. Once the purpose is clear you choose primary data — gathered fresh for that purpose, with high reliability but more cost — or secondary data, already collected by others, fast to obtain but perhaps mismatched or biased, and often you combine both.
When the population — the whole group analysed — is too large, you study a sample through sampling, aiming for a miniature of the population. Random sampling makes that likely: simple random sampling picks purely by chance, while stratified sampling splits the population into strata and picks from each to mirror its make-up.
The danger is sampling bias, where a skewed selection cannot speak for the whole. It is reduced by careful design of who is chosen, how questions are worded, and how the survey is run.
2 · Read the passage and fill in the blanks (a)–(d).
In data analysis, it is important to plan what data to collect according to the purpose. The data to be collected can be classified into two types: ( a ), which is newly collected, and ( b ), which has already been collected and made public by others. Also, when investigating the entire population is difficult, ( c ) is performed to select a part of the population. At this time, if bias arises in the way of selection, ( d ) occurs, and the analysis results may not be applicable to the population as a whole.
3(1) · Which is the most appropriate example of secondary data?
Mark ○ for the one that is secondary data, and × for the ones that are primary.
Survey responses collected from students in one’s own class.
Population statistics published by the government.
Traffic on a school commute route observed and recorded by oneself.
Records of an interview designed and conducted by oneself.
3(2) · Which most appropriately describes sampling methods?
Mark ○ for the correct description, and × for the ones that are wrong.
Simple random sampling is a method in which the surveyor preferentially chooses subjects that are easy to gather.
Stratified sampling is a method in which the population is divided into strata, and a sample is randomly chosen from each stratum.
Simple random sampling is a method in which the population is divided into strata, and a representative is chosen from each stratum.
A complete survey is a method in which only part of the population is surveyed.
3(3) · Which is NOT an appropriate way to reduce sampling bias?
Mark ○ for the ones that reduce bias, and × for the one that does not.
Choose subjects without bias from the entire population.
Limit the timing and location of the survey, and gather responses only from people in specific conditions.
Choose the sample taking into account the strata of the population (age group, gender, etc.).
Design the wording of survey questions so that respondents can answer truthfully.
Key takeaway
Collect the right data (primary or secondary), study a fair sample of the population, use random sampling — simple or stratified — so the sample mirrors the whole, and guard against sampling bias. Correct arithmetic on a biased sample is still a wrong answer.
Ask each other the same question two ways: “how many hours did you study yesterday?” and “between 0 and 1, 1 and 2, or more than 2 hours?” See whether the wording changes the answers you get. That difference is response bias in miniature.