Module S: Statistics
Module S Learning Objectives:
- Distinguish between a population and a sample
- Categorize data as either qualitative or quantitative
- Correctly identify the type of sampling method used
- Correctly identify the potential bias in a statistical study
- Determine whether a statistical study is an experiment or an observational study
- Classify the type of experimental study
- Use bar graphs and pie charts to graphically display data
- Use a histogram to graphically display data
- Identify distribution shapes from histograms
- Calculate and interpret the mean, median, and mode
- Compute and interpret the range and standard deviation
- Calculate and interpret the quartiles, 5 number summary, and boxplots
- Be familiar with properties of the normal distribution
- Be able to apply and interpret the Empirical Rule to a normally distributed variable
- Calculate and interpret the meaning of z-scores
- Find the area under specified regions of the standard normal distribution
- Use z-scores to find the percentage of observations with a specific location for a normally distributed variable
Section S.1 – Introduction to Statistics
Section S.2 – Presenting Data Graphically
Section S.3 – Describing a Distribution and Measures of Central Tendency
Section S.4 – Measures of Variation, Quartiles, Five Number Summary, and Box Plots
Section S.5 – The Normal Distribution and The Empirical Rule
Section S.6 – Standard Scores (z-scores), Finding Percentages with Normally Distributed Variables
Section S.1 – Introduction to Statistics
It is important to be able to properly evaluate the data and claims that bombard you every day. If you cannot distinguish sound from faulty reasoning, then you are vulnerable to manipulation and to decisions that are not in your best interest. Statistics provides tools that you need in order to react intelligently to information you hear or read. In this sense, Statistics is one of the most important things that you can study.
To be more specific, here are some claims that we have heard on several occasions: (We are not saying that each one of these claims is true!)
• 4 out of 5 dentists recommend Dentyne.
• Almost 85% of lung cancers in men and 45% in women are tobacco-related.
• Condoms are effective 94% of the time.
• Native Americans are significantly more likely to be hit crossing the streets than are people of other ethnicities.
• People tend to be more persuasive when they look others directly in the eye and speak loudly and quickly.
• Women make 75 cents to every dollar a man makes when they work the same job.
• A surprising new study shows that eating egg whites can increase one’s life span.
• People predict that it is very unlikely there will ever be another baseball player with a batting average over 400.
• There is a 70% chance that in a room full of 30 people that at least two people will share the same birthday.
• 79.48% of all statistics are made up on the spot.
All of these claims are statistical in character. We suspect that some of them sound familiar; if not, we bet that you have heard other claims like them. Notice how diverse the examples are; they come from psychology, health, law, sports, business, etc. Indeed, data and data-interpretation show up in discourse from virtually every facet of contemporary life.
Statistics are often presented in an effort to add credibility to an argument or advice. You can see this by paying attention to television advertisements. Many of the numbers thrown about in this way do not represent careful statistical analysis. They can be misleading, and push you into decisions that you might find cause to regret. These chapters will help you learn statistical essentials. It will make you into an intelligent consumer of statistical claims.
You can take the first step right away. To be an intelligent consumer of statistics, your first reflex must be to question the statistics that you encounter. The British Prime Minister Benjamin Disraeli famously said, “There are three kinds of lies — lies, damned lies, and statistics.” This quote reminds us why it is so important to understand statistics. So let us invite you to reform your statistical habits from now on. No longer will you blindly accept numbers or findings. Instead, you will begin to think about the numbers, their sources, and most importantly, the procedures used to generate them.
We have put the emphasis on defending ourselves against fraudulent claims wrapped up as statistics. Just as important as detecting the deceptive use of statistics is the appreciation of the proper use of statistics. You must also learn to recognize statistical evidence that supports a stated conclusion. Statistics are all around you, sometimes used well, sometimes not. We must learn how to distinguish the two cases.
| Populations and Samples |
Before we begin gathering and analyzing data we need to characterize the population we are studying. If we want to study the amount of money spent on textbooks by a typical first-year college student, our population might be all first-year students at your college. Or it might be:
- All first-year students at public colleges and universities in the state of Arizona.
- All first-year community college students in the state of Arizona.
- All first-year students at all colleges and universities in the state of Arizona.
- All first-year students at all colleges and universities in the entire United States.
| Population |
| The population of a study is the group the collected data is intended to describe. It is the entire group of objects or individuals of interest in a statistical study. |
Why is it important to specify the population? We might get different answers to our question as we vary the population we are studying. First-year students at Arizona State University might take slightly more diverse courses than those at your college, and some of these courses may require less popular textbooks that cost more; or, on the other hand, the University Bookstore might have a larger pool of used textbooks, reducing the cost of these books to the students. Whichever the case (and it is likely that some combination of these and other factors are in play), the data we gather from your college will probably not be the same as that from Arizona State University. Particularly when conveying our results to others, we want to be clear about the population we are describing with our data.
| Example 1 |
|
A poll is conducted to determine whether people intend to vote for or against an upcoming proposition in Maricopa County. What is the population? |
|
The population of interest in this case is all registered voters in Maricopa County. |
| Example 2 |
|
A newspaper website contains a poll asking people their opinion on a recent news article. What is the population? |
|
While the intended population may have been all people, the real population of the survey is readers of the website. |
The previous example demonstrates a potential problem with how the data used in a statistical study is actually collected. We will come back to this concept when we learn about sampling methods later in this chapter.
If we were able to gather data on every member of our population, say the average (we will define “average” more carefully in a subsequent section) amount of money spent on textbooks by each first-year student at your college during the 2009-2010 academic year, the resulting number would be called a parameter.
| Parameter |
| A parameter is a value (average, percentage, etc.) calculated using all the data from a population. |
We seldom see parameters, however, since surveying an entire population is usually very time-consuming and expensive, unless the population is very small or we already have the data collected.
| Census |
| A survey of an entire population is called a census. |
You are probably familiar with two common censuses: the official government Census that attempts to count the population of the U.S. every ten years, and voting, which asks the opinion of all eligible voters in a district. The first of these demonstrates one additional problem with a census: the difficulty in finding and getting participation from everyone in a large population, which can bias, or skew, the results.
There are occasionally times when a census is appropriate, usually when the population is fairly small. For example, if the manager of Starbucks wanted to know the average number of hours her employees worked last week, she should be able to pull up payroll records or ask each employee directly.
Since surveying an entire population is often impractical, we usually select a sample to study;
| Sample |
| A sample is a smaller subset of the entire population, ideally one that is fairly representative of the whole population. |
We will discuss sampling methods in greater detail in a later section. For now, let us assume that samples are chosen in an appropriate manner. If we survey a sample, say 100 first-year students at your college, and find the average amount of money spent by these students on textbooks, the resulting number is called a statistic.
| Statistic |
| A statistic is a value (average, percentage, etc.) calculated using the data from a sample. |
| Example 3 |
| A researcher wanted to know how citizens of Tacoma felt about a voter initiative. To study this, she goes to the Tacoma Mall and randomly selects 500 shoppers and asks them their opinion. 60% indicate they are supportive of the initiative. What is the sample and population? Is the 60% value a parameter or a statistic? |
|
The sample is the 500 shoppers questioned. The population is less clear. While the intended population of this survey was Tacoma citizens, the effective population was mall shoppers. There is no reason to assume that the 500 shoppers questioned would be representative of all Tacoma citizens. The 60% value was based on the sample, so it is a statistic. |
| You Try S.1.A |
|
a. To determine the average length of trout in a lake, researchers catch 20 fish and measure them. What is the sample and population in this study? b. A college reports that the average age of their students is 28 years old. Is this a statistic or a parameter? |
How to Mess Things Up Before You Start
There are number of ways that a study can be ruined before you even start collecting data. The first we have already explored – sampling or selection bias, which is when the sample is not representative of the population. One example of this is voluntary response bias, which is bias introduced by only collecting data from those who volunteer to participate. This is not the only potential source of bias.
| Sources of Bias |
|
Sampling bias – when the sample is not representative of the population Voluntary response bias – the sampling bias that often occurs when the sample is volunteers Self-interest study – bias that can occur when the researchers have an interest in the outcome Response bias – when the responder gives inaccurate responses for any reason Perceived lack of anonymity – when the responder fears giving an honest answer might negatively affect them Non-response bias – when people refusing to participate in the study can influence the validity of the outcome Loaded questions – when the question wording influences the response |
| Example 4 |
|
Consider a recent study which found that chewing gum may raise math grades in teenagers5. This study was conducted by the Wrigley Science Institute, a branch of the Wrigley chewing gum company. This is an example of a self-interest study; one in which the researches have a vested interest in the outcome of the study. While this does not necessarily ensure that the study was biased, it certainly suggests that we should subject the study to extra scrutiny. |
| Example 5 |
|
A survey asks people “when was the last time you visited your doctor?” This might suffer from response bias, since many people might not remember exactly when they last saw a doctor and give inaccurate responses. |
Sources of response bias may be innocent, such as bad memory, or as intentional as pressuring by the pollster. Consider, for example, how many voting initiative petitions people sign without even reading them.
| Example 6 |
|
A survey asks participants a question about their interactions with members of other races. Here, a perceived lack of anonymity could influence the outcome. The respondent might not want to be perceived as racist even if they are, and give an untruthful answer. |
| Example 7 |
|
An employer puts out a survey asking their employees if they have a drug abuse problem and need treatment help. Here, a perceived lack of anonymity could influence the outcome. Answering truthfully might have consequences; responses might not be accurate if the employees do not feel their responses are anonymous or fear retribution from their employer. |
| Example 8 |
|
A telephone poll asks the question “Do you often have time to relax and read a book?”, and 50% of the people called refused to answer the survey. It is unlikely that the results will be representative of the entire population. This is an example of non-response bias, introduced by people refusing to participate in a study or dropping out of an experiment. When people refuse to participate, we can no longer be so certain that our sample is representative of the population. |
| Example 9 |
|
A survey asks “do you support funding research of alternative energy sources to reduce our reliance on high-polluting fossil fuels?” This is an example of a loaded or leading question – questions whose wording leads the respondent towards an answer. |
Loaded questions can occur intentionally by pollsters with an agenda, or accidentally through poor question wording. Also a concern is question order, where the order of questions changes the results. A psychology researcher provides an example6:
| You Try S.1.B |
|
In each situation, identify a potential source of bias: a. A survey asks how many sexual partners a person has had in the last year b. A radio station asks readers to phone in their choice in a daily poll. c. A substitute teacher wants to know how students in the class did on their last test. The teacher asks the 10 students sitting in the front row to state their latest test score. d. High school students are asked if they have consumed alcohol in the last two weeks. e. The Beef Council releases a study stating that consuming red meat poses little cardiovascular risk. f. A poll asks “Do you support a new transportation tax, or would you prefer to see our public transportation system fall apart?” |
Experiments
So far, we have primarily discussed observational studies – studies in which conclusions would be drawn from observations of a sample or the population. In some cases, these observations might be unsolicited, such as studying the percentage of cars that turn right at a red light even when there is a “no turn on red” sign. In other cases, the observations are solicited, like in a survey or a poll.
In contrast, it is common to use experiments when exploring how subjects react to an outside influence. In an experiment, some kind of treatment and/or control is applied to the subjects and the results are measured and recorded.
| Observational Studies and Experiments |
|
An observational study is a study based on observations or measurements. An experiment is a study in which the effects of a treatment are measured. |
Here are some examples of experiments:
| Example 10 |
|
a. A pharmaceutical company tests a new medicine for treating Alzheimer’s disease by administering the drug to 50 elderly patients with recent diagnoses. The treatment here is the new drug. |
| You Try S.1.C |
|
Determine which of the scenarios below describe an observational study and which describe an experiment? If it is an experiment, identify the treatment. a. The weights of 30 randomly selected people are measured b. Subjects are asked to do 20 jumping jacks, and then their heart rates are measured c. 20 people are given a concentration test after drinking a cup of coffee. |
When conducting experiments, it is essential to isolate the treatment being tested.
Suppose a middle school (junior high) finds that their students are not scoring well on the state’s standardized math test. They decide to run an experiment to see if an alternate curriculum would improve scores. To run the test, they hire a math specialist to come in and teach a class using the new curriculum. To their delight, they see an improvement in test scores.
The difficulty with this scenario is that it is not clear whether the curriculum is responsible for the improvement, or whether the improvement is due to a math specialist teaching the class. This is called confounding – when it is not clear which factor or factors caused the observed effect. Confounding is the downfall of many experiments, though sometimes it is hidden.
| Confounding |
| Confounding occurs when there are two or more potential variables that could have caused the outcome and it is not possible to determine which actually caused the result. |
| Example 11 |
|
A drug company study about a weight loss pill might report that people lost an average of 8 pounds while using their new drug. However, in the fine print you find a statement saying that participants were encouraged to also diet and exercise. It is not clear in this case whether the weight loss is due to the pill, to diet and exercise, or a combination of both. In this case confounding has occurred. |
| Example 12 |
|
Researchers conduct an experiment to determine whether students will perform better on an arithmetic test if they listen to music during the test. They first give the student a test without music, then give a similar test while the student listens to music. In this case, the student might perform better on the second test, regardless of the music, simply because it was the second test and they were warmed up. |
There are a number of measures that can be introduced to help reduce the likelihood of confounding. The primary measure is to use a control group.
| Control Group |
| When using a control group, the participants are divided into two or more groups, typically a control group and a treatment group. The treatment group receives the treatment being tested; the control group does not receive the treatment. |
Ideally, the groups are otherwise as similar as possible, isolating the treatment as the only potential source of difference between the groups. For this reason, the method of dividing groups is important. Some researchers attempt to ensure that the groups have similar characteristics (same number of females, same number of people over 50, etc.), but it is nearly impossible to control for every characteristic. Because of this, random assignment is very commonly used.
| Example 13 |
|
To determine if a two-day prep course would help high school students improve their scores on the SAT test, a group of students was randomly divided into two subgroups. The first group, the treatment group, was given a two-day prep course. The second group, the control group, was not given the prep course. Afterwards, both groups were given the SAT. |
| Example 14 |
|
A company testing a new plant food grows two crops of plants in adjacent fields, the treatment group receiving the new plant food and the control group not. The crop yield would then be compared. By growing them at the same time in adjacent fields, they are controlling for weather and other confounding factors. |
Sometimes not giving the control group anything does not completely control for confounding variables. For example, suppose a medicine study is testing a new headache pill by giving the treatment group the pill and the control group nothing. If the treatment group showed improvement, we would not know whether it was due to the medicine in the pill, or a response to have taken any pill. This is called a placebo effect.
| Placebo Effect |
|
The placebo effect is when the effectiveness of a treatment is influenced by the patient’s perception of how effective they think the treatment will be, so a result might be seen even if the treatment is ineffectual. |
| Example 15 |
|
A study found that when doing painful dental tooth extractions, patients told they were receiving a strong painkiller while actually receiving a saltwater injection found as much pain relief as patients receiving a dose of morphine.7 |
To control for the placebo effect, a placebo, or dummy treatment, is often given to the control group. This way, both groups are truly identical except for the specific treatment given.
| Placebo and Placebo Controlled Experiments |
|
A placebo is a dummy treatment given to control for the placebo effect. An experiment that gives the control group a placebo is called a placebo controlled experiment. |
| Example 16 |
| a. In a study for a new medicine that is dispensed in a pill form, a sugar pill could be used as a placebo.
b. In a study on the effect of alcohol on memory, a non-alcoholic beer might be given to the control group as a placebo. c. In a study of a frozen meal diet plan, the treatment group would receive the diet food, and the control could be given standard frozen meals stripped of their original packaging. |
In some cases, it is more appropriate to compare to a conventional treatment than a placebo. For example, in a cancer research study, it would not be ethical to deny any treatment to the control group or to give a placebo treatment. In this case, the currently acceptable medicine would be given to the second group, called a comparison group in this case. In our SAT test example, the non-treatment group would most likely be encouraged to study on their own, rather than be asked to not study at all, to provide a meaningful comparison.
When using a placebo, it would defeat the purpose if the participant knew they were receiving the placebo.
| Blind Studies |
| A blind study is one in which the participant does not know whether or not they are receiving the treatment or a placebo.
A double-blind study is one in which neither the participants, nor those interacting with the participants, know who is in the treatment group and who is in the control group. |
| Example 17 |
|
In a study about anti-depression medicine, you would not want the psychological evaluator to know whether the patient is in the treatment or control group either, as it might influence their evaluation, so the experiment should be conducted as a double-blind study. |
It should be noted that not every experiment needs a control group.
| Example 18 |
|
If a researcher is testing whether a new fabric can withstand fire, she simply needs to torch multiple samples of the fabric – there is no need for a control group. |
| Example 19 |
|
To test a new lie detector, two groups of subjects are given the new test. One group is asked to answer all the questions truthfully, and the second group is asked to lie on one set of questions. The person administering the lie detector test does not know what group each subject is in. In this example, the truth-telling group could be considered the control group, but really both groups are treatment groups here, since it is important for the lie detector to be able to correctly identify lies, and also not identify truth telling as lying. This study is blind, since the person running the test does not know what group each subject is in. |
References
[5] Reuters. http://news.yahoo.com/s/nm/20090423/od_uk_nm/oukoe_uk_gum_learning. Retrieved 4/27/09
[6] Swartz, Norbert. http://www.umich.edu/~newsinfo/MT/01/Fal01/mt6f01.html. Retrieved 3/31/2009
[7] Levine JD, Gordon NC, Smith R, Fields HL. (1981) Analgesic responses to morphine and placebo in individuals with postoperative pain. Pain. 10:379-89.
Section S.1 Answers to You Try Problems
S.1.A
a. The sample is the 20 fish caught. The population is all fish in the lake. The sample may be somewhat unrepresentative of the population since not all fish may be large enough to catch the bait.
b. This is a parameter, since the college would have access to data on all students (the population)
S.1.B
a. Response bias – historically, men are likely to over-report, and women are likely to under-report to this question.
b. Voluntary response bias – the sample is self-selected
c. Sampling bias – the sample may not be representative of the whole class
d. Lack of anonymity
e. Self-interest study
f. Loaded question
S.1.C
a. Observational study
b. Experiment; the treatment is the jumping jacks
c. Experiment; the treatment is coffee
