Statistics for Social Science
Statistics for Social Science

Statistics for Social Science

Lead Author(s): Stephen Hayward

Student Price: Contact us to learn more

Statistics for Social Science takes a fresh approach to the introductory class. With learning check questions, embedded videos and interactive simulations, students engage in active learning as they read. An emphasis on real-world and academic applications help ground the concepts presented. Designed for students taking an introductory statistics course in psychology, sociology or any other social science discipline.

This content has been used by 8,525 students

What is a Top Hat Textbook?

Top Hat has reimagined the textbook – one that is designed to improve student readership through interactivity, is updated by a community of collaborating professors with the newest information, and accessed online from anywhere, at anytime.


  • Top Hat Textbooks are built full of embedded videos, interactive timelines, charts, graphs, and video lessons from the authors themselves
  • High-quality and affordable, at a significant fraction in cost vs traditional publisher textbooks
 

Key features in this textbook

Our Statistics for Social Science textbook allows students to manipulate data, visualize the effects discussed, and explore Lightboard videos that feature instructor explanations to reinforce concepts and calculations.
Top Hat’s interactive offering includes a complementary module on using R software for data management, graphics, and conducting statistical analyses with examples and practice questions.
Built-in assessment questions embedded throughout chapters so students can read a little, do a little, and test themselves to see what they know!

Comparison of Social Sciences Textbooks

Consider adding Top Hat’s Statistics for Social Sciences textbook to your upcoming course. We’ve put together a textbook comparison to make it easy for you in your upcoming evaluation.

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

Cengage

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

Sage

Gregory Privitera, Essentials Statistics for the Behavioral Sciences, 2nd Edition

Pricing

Average price of textbook across most common format

Up to 40-60% more affordable

Lifetime access on any device

$200.83

Hardcover print text only

$239.95

Hardcover print text only

$92

Hardcover print text only

Always up-to-date content, constantly revised by community of professors

Content meets standard for Introduction to Anatomy & Physiology course, and is updated with the latest content

In-Book Interactivity

Includes embedded multi-media files and integrated software to enhance visual presentation of concepts directly in textbook

Only available with supplementary resources at additional cost

Only available with supplementary resources at additional cost

Only available with supplementary resources at additional cost

Customizable

Ability to revise, adjust and adapt content to meet needs of course and instructor

All-in-one Platform

Access to additional questions, test banks, and slides available within one platform

Pricing

Average price of textbook across most common format

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Up to 40-60% more affordable

Lifetime access on any device

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

$200.83

Hardcover print text only

Pearson

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

$239.95

Hardcover print text only

Sage

McConnell, Brue, Flynn, Principles of Microeconomics, 7th Edition

$92

Hardcover print text only

Always up-to-date content, constantly revised by community of professors

Constantly revised and updated by a community of professors with the latest content

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

Pearson

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

Sage

Gregory Privitera, Essentials Statistics for the Behavioral Sciences, 2nd Edition

In-book Interactivity

Includes embedded multi-media files and integrated software to enhance visual presentation of concepts directly in textbook

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

Pearson

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

Sage

Gregory Privitera, Essentials Statistics for the Behavioral Sciences, 2nd Edition

Customizable

Ability to revise, adjust and adapt content to meet needs of course and instructor

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

Pearson

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

Sage

Gregory Privitera, Essentials Statistics for the Behavioral Sciences, 2nd Edition

All-in-one Platform

Access to additional questions, test banks, and slides available within one platform

Top Hat

Steve Hayward et al., Statistics for Social Sciences, Only one edition needed

Pearson

Agresti, Statistical Methods for the Social Sciences, 5th Edition

Pearson

Gravetter et al., Essentials of Statistics for The Behavioral Sciences, 9th Edition

Sage

Gregory Privitera, Essentials Statistics for the Behavioral Sciences, 2nd Edition

About this textbook

Lead Authors

Steve HaywardRio Salado College

A lifelong learner, Steve focused on statistics and research methodology during his graduate training at the University of New Mexico. He later founded and served as CEO of Center for Performance Technology, providing instructional design and training development support to larger client organizations throughout the United States. Steve is presently lead faculty member for statistics at Rio Salado College in Tempe, Arizona.

Joseph F. Crivello, PhDUniversity of Connecticut

Joseph Crivello has taught Anatomy & Physiology for over 34 years, and is currently a Teaching Fellow and Premedical Advisor of the HMMI/Hemsley Summer Teaching Institute.

Contributing Authors

Susan BaileyUniversity of Wisconsin

Deborah CarrollSouthern Connecticut State University

Alistair CullumCreighton University

William Jerry HauseltSouthern Connecticut State University

Karen KampenUniversity of Manitoba

Adam SullivanBrown University

Explore this textbook

Read the fully unlocked textbook below, and if you’re interested in learning more, get in touch to see how you can use this textbook in your course today.

Introduction to Statistics

A visualization of Social Network Analysis, where the networks and connections of social structures are investigated. [1]


Chapter Objectives

After completing this chapter you will be able to:

  • Distinguish between a population and a sample.
  • Identify parameters and statistics.
  • Differentiate descriptive statistics and inferential statistics.
  • Distinguish between qualitative data and quantitative data.
  • Classify data as nominal, ordinal, interval, and ratio.
  • Design a statistical study and collect data.
  • Design an experiment and create a sample.


What is 'Statistics'?

Statistics deals with the collection, presentation, analysis, and interpretation of data. Some consider it a branch of mathematics, but in relation to its extensive role in the social sciences, it might be better considered as an analysis and decision-making tool that makes use of mathematical processes.

As Jeri Mulrow, American Statistical Association's Vice President, put it, “...statistics are everywhere! They are all around us in our daily lives. It is important to be able to think critically about all of the data and information that surround us. Statistics and statistical thinking help us to make sense out of all of it.”

So, why statistics, anyway? Why do these things matter? Well, it turns out that they matter a lot! Watch as host Eric Newburger uses graphics, pictures, and stories to illustrate the relevance of statistics and how many different things can be learned about the nation and its communities through the study of statistics.


Populations and Samples

Social science research is typically focused on being able to draw conclusions about entire groups of people that are identified as populations. These groups, or populations, may be large, for example, everyone living in a certain country, or they may be much smaller, for example, the students at a particular college.

Researchers are often interested in being able to draw conclusions about these populations but are most often unable to obtain data from each and every member of the population, as is done with a census. Therefore, they must rely on estimates based on data from a subset of the population, identified as a sample. The data from the sample enables researchers to draw conclusions about the population based on the data at hand when they are unable to conduct an actual census.

A population can be defined in many different ways. A population may be very large, such as the population of Canada, or it may be a much smaller group, such as the population of a postal code. It could also be all people with brown hair. It could even be all people with brown hair who are in this statistics class!

sample is a subset, or part, of a population, selected in such a way as to be representative of the population it is drawn from. A sample is used in statistical decision-making to represent a population. There are various ways of selecting samples from populations; some of those will be introduced later in this chapter.

In statistics, a sample is represented by data derived from observations, outcomes, responses, measurements or counts. This data is often referred to as scores.

Examples of a Population

A human learning researcher collects data from a sample of 40 babies to see how quickly they adapt to changes in the stimulus that indicates the availability of food. The population of interest is babies in general.

An admissions advisor at a large university is interested in knowing the average starting salary of graduates from the university’s MBA program and conducts a survey to obtain salary data. His population of interest is graduates of the MBA program.

The United States Census Bureau conducts a census of the country’s residents every ten years. The population of interest is all people living in the United States.

Examples of a Sample

A pollster collects data on responses to questions about the latest health care proposal, Amerimed, for a newspaper article. When the article is printed, the headline reads, "60% of U.S. residents agree with the Amerimed health care proposal." The population here is all residents of the U.S. but the data had to come from a sample, a subset of the population, since it would be impossible for the pollster to obtain data from each and every resident of the U.S.

The Dean of Students at a community college obtains test scores from three statistics classes in order to estimate average scores for all students taking statistics at the college. In this study, the students in the three statistics classes represent a sample taken from the population of all students enrolled in statistics classes. 



1.01

A survey is conducted of all residents of an apartment building to determine the average family size of all residents. The residents represent a ________.

A

Population

B

Sample


1.02

All residents of an apartment building are surveyed to determine average annual income. The data are used to estimate average household income for people living in that neighborhood. The people surveyed represent a ________.


1.03

Match the descriptions in the left column to the correct description in the right column.

Premise
Response
1

One city in each state or province in a country

A

Sample

2

The age of one person per row in a cinema

B

Population

3

All 50 state capitals in the U.S.

C

Population

4

The score of every student taking the SAT test last year

D

Sample



Parameters and Statistics

parameter is a (numerical) description of a measurable population characteristic. A parameter provides information about an entire population. If we know the value of a population parameter, we know what that value is for the population as a whole.

statistic is a (numerical) description of a sample characteristic. A statistic provides information about a portion, or subset, of a population. If another sample was selected from the population, the statistic in question most likely will have a different value than before, since no two samples are likely to be exactly the same. This variability is an important consideration when drawing conclusions based on data obtained from a sample, as we'll see in future lessons.

Examples of a Parameter

A social worker would like to know the average income of residents of a low-income apartment project and conducts a census by going door-to-door to obtain data. The population is residents of the project and the income data is a parameter since it represents the entire population.

A statistics instructor scored all of the midterm exams for his statistics class and presented the data to the class during a review of the exam results. In this case, the statistics class is a population and the data the instructor presented represented parameters since it provided information about the entire population.

Examples of a Statistic

We saw previously that the pollster who collected data on responses to questions about the latest health care proposal was collecting data from a sample, or subset, of a larger population. The data collected was a statistic since it provided information obtained from a sample selected to represent the larger population.

A school nurse selected 40 sixth-grade students at random from the school class list and measured their height to obtain an estimate of the average height of all sixth graders at the school. The average she calculated was a statistic since it provided information from a sample representing the population of sixth graders at the school.



1.04

A numerical measure that describes a sample characteristic is a ________.


1.5

A numerical measure that describes an entire group of people or things is a ________.


1.06

The average GPA of thirty students from a large community college having coffee at the student union is a ________.



Descriptive Statistics and Inferential Statistics



Descriptive statistics involve the organization, summarization, and display of data. They are used to present quantitative descriptions in a manageable form. A research study may involve lots of measures or, alternatively, may measure a large number of people or things on one particular measure of interest.

Descriptive statistics help us to simplify the resulting large amounts of data in a way that makes the data easier to grasp. Each descriptive statistic reduces lots of data into a simpler summary. For instance, consider a baseball player's batting average. It is figured by dividing the number of hits by the number of times at bat (reported to three significant digits). A batter who is hitting .333 is getting one hit in every three times at bat. This single number summarizes a large number of discrete events.

Consider many students’ least favorite statistic, the Grade Point Average (GPA). GPA can be used to summarize and describe a student's performance across a large number of classes.

There is also a downside to summarizing data with descriptive statistics. You would not know from the batting average whether the ball player has been hitting home runs or singles, or whether the player has been in a slump recently. The GPA doesn't tell you anything about the rigor of the courses the student has been taking, i.e., were they easy classes or more difficult classes? Even with these limitations, though, descriptive statistics provide a useful summary and can make it possible to draw comparisons across people or situations.

Inferential statistics is the science of drawing conclusions about a population based on data collected from a sample, i.e., conclusions that extend beyond the data that are "in hand." Inferential statistics are used to make generalizations about things we cannot directly measure, while descriptive statistics are used to describe a set of data that we have somehow collected.

Researchers use inferential statistics when it is not convenient, or possible, to examine each member of an entire population. For example, it is impractical to measure the response time of all humans to a particular stimulus, but a researcher could make an inference about response time based on the data from a sample of people.

Examples: Descriptive Statistics

The researcher who collected data about the average income of residents of an apartment project published her information in a professional journal that reported a summary of the data. This is an example of descriptive statistics since it involved the organization, summarization, and display of the data.

In a previous example, a statistics instructor scored all of the midterm exams for his statistics class and presented the data to the class during a review of the exam results. This is also an example of descriptive statistics, involving the organization, summarization, and display of the data.

Examples: Inferential Statistics

The pollster who collected data on responses to questions about the latest health care proposal reported that 60% of U.S. residents agree with the Amerimed health care proposal. The data came from a sample but the conclusion extends beyond that. This is an example of inferential statistics, where a conclusion is drawn (an inference is made) about a population based on data collected from a sample.

The school nurse who collected data on heights of sixth-graders used the data to make an estimate of the average height of all sixth grade students at the school. This is another example of inferential statistics, using data from a sample to reach a conclusion about the larger population of students.

1.07

Match the following to the correct response.

Premise
Response
1

Graduating high school seniors are ranked according to their grade averages.

A

Inferential

2

A survey concludes that most students favor using e-books for texts.

B

Descriptive

3

Batting averages of ballplayers are posted online.

C

Inferential

4

A researcher reports that 90% of college students text during class.

D

Descriptive


Quantitative and Qualitative Data

Quantitative data are information about quantities. They tell us how much or how many of something, and are presented as sets of numbers. The numbers can be either continuous or discrete.

  • Continuous data can (theoretically) take on any value within a specific range. A continuous variable can be measured. It tells us how much.
  • Discrete data can only take on certain values, and there are gaps between the values. A discrete variable can be counted. It tells us how many.

Qualitative data are presented as names or categories. They can be observed, but not measured. The data are descriptive only, and cannot be operated on mathematically.

Figure 1.1: Types of data​

Examples: Quantitative and Qualitative Data

Grade point average (GPA) given on the typical 4-point scale is quantitative and is continuous since it can take on any value from 0 to 4 (3.11, 3.12, 3.13, etc.).

On the other hand, grade point given as letter grades is qualitative, since it is represented as categories designated (usually) as A, B, C, etc.

The number of students in a statistics class is a quantitative measure. It is discrete since it can only assume certain values (21, 22, 23, etc.).

The section number of the statistics class is qualitative. It only serves to identify the section; it wouldn’t make sense to add or multiply class section numbers, for instance.

1.08

Select a match to classify each of the following data examples.

Premise
Response
1

Computer operating system designation

A

Quantitative discrete

2

Annual salary of professors at the university

B

Quantitative continuous

3

Number of correct answers on a chemistry exam

C

Qualitative


Levels of Measurement

Data can be classified according to the four levels of measurement, which then determine the statistical treatment that is appropriate.

Nominal data are qualitative only. They can only be used as categories based on names, labels or qualities. The categories are mutually exclusive, meaning all cases must be sorted into one or another category with no overlap. Numbers may be used as labels but have no additional meaning or operability. 

  • The names of the sub-scales of a psychological test are examples of nominal data.  They identify the scales and represent categories.
  •  A list of postal codes represents nominal data. The codes may be shown as numbers or combinations of letters and numbers but have no mathematical properties; they only serve as identifiers.

Ordinal data can be either quantitative or qualitative and can be ordered, or ranked, with cases sorted into discrete groups. The distance between the data points is not meaningful, however. The ordering can indicate higher, or lower, or more, or less with respect to some value or characteristic. 

  • Finishing position in a marathon race is an example of ordinal data.
  • Researchers may use a Likert Scale to have respondents rate their satisfaction level with a product or service or to indicate their level of agreement with a statement or position. The scale choices represent ordinal data.

Interval data are quantitative. The distance between the data points is meaningful, but a zero simply represents a point on the scale and is not an inherent zero. Since the distance between points is meaningful, values can be added and subtracted and averages can be calculated but not ratios, due to lack of an inherent, or absolute, zero point. 

  • Scores on a standardized IQ test are examples of interval data since the intervals are equal and have the same meaning, but there is no absolute zero.
  • Temperature shown on a Celsius or Fahrenheit scale is interval level since zero on either scale is not an absolute zero showing the total absence of heat. On either of those scales, a temperature of 30º is 15º more than a temperature if 15º but it does not represent twice as much heat.

Ratio data have meaningful intervals and have an inherent zero point. Since there is an inherent zero, the ratio of two data points can be meaningfully expressed as a multiple of another data point. Ratio scales provide a wealth of possibilities when it comes to statistical analysis. Ratio level variables can be meaningfully added, subtracted, multiplied, and divided (ratios) and can be operated on to determine a variety of statistical measures. 

  • Height and weight are examples of ratio data; they have absolute zero points.
  • Numerical scores on a statistics exam are ratio level data since there is an absolute zero point and scores can be used to form ratios (a score of 75 is 50% higher than a score of 50, etc).

Note: An inherent zero represents the total absence of something. In order to calculate ratios, the data set must have an inherent, or absolute, zero.

Figure 1.2: Levels of measurement​

Example: Measurement Levels

A school psychologist needed to gather data for a study of academic performance at the two high schools in her district. Her data gathering proceeded through several stages:

Stage One

She first identified students who had volunteered to participate according to which school they attended, Meriwether Lewis School or Mildred Johnson School. This classification was at the nominal level of measurement since it consisted of categories.

Stage Two

Following this step, the students were rated by their teachers as Low Performing, Medium Performing, or High Performing. This classification was qualitative, and at the ordinal level of measurement, since the rankings are ordered, but the distance between the rankings is not meaningful in a way that can be calculated.

Stage Three

She then arranged the data she had collected according to the students’ school grade as 10th grade, 11th grade, and 12th grade. This classification was at the interval level since the distance between scores is meaningful, but there is no inherent zero.

Stage Four

To conclude the study, the psychologist compared the numerical GPA of students in each performance level in each grade between schools in terms of the percent difference between them. For example, Lewis School High Performing 10th-graders averaged 12% higher GPA’s than High Performing 10th-graders at Johnson School. This measurement was at the ratio level since the data have an inherent zero.



1.09

Click on the data type of a license plate in ABC-123 format.


1.10

Click on the data type of average speed on a section of freeway.


1.11

Identify the data type and level: finishing position in a bicycle race.

A

Qualitative

B

Quantitative Discrete

C

Quantitative Continuous

D

Nominal

E

Ordinal

F

Interval

G

Ratio



Variables

A variable in mathematics is a symbol that can represent different values in an expression or equation. Its use in statistics is similar but also a bit different from that. In statistics, a variable is something that can change or be changed, such as attention span, memories of events, time to respond to a stimulus, etc. It is something that varies or has the potential to vary.

One way to think of a variable is that it can be divided into parts. Researchers often identify variables they are interested in studying and then measure them under different conditions. In effect, they are counting the parts under different conditions to see how the parts might change under these differing conditions.

Levels or Scores?

  • If there are not many parts, they are usually called levels.
  • If a variable can be divided into many parts, these parts are usually called scores or something similar.

Examples: Levels vs. Scores

  • Handedness can be considered as a variable with two levels, left and right.
  • Gender can be considered as a variable with two levels, male and female.
  • GPA could be a variable with five levels: A, B, C, D, and F.
  • Or GPA expressed on a four-point scale could be a variable with many parts, e.g., 3.00, 3.01, 3.02, etc.
  • Temperature is another variable with many possible scores or parts.


Design of Statistical Studies

In many ways, the design of a study is even more important than the analysis. A study with a flawed design cannot be saved, no matter what statistical analysis technique is used. A study that is designed well, but was analyzed inappropriately, can have its data re-analyzed and still provide meaningful results.

Prior to analyzing the results of a study, a researcher must determine that the study’s design was appropriate, so as to have confidence that the results will be both valid and reliable. A well-designed study should follow the steps below:

Figure 1.3: Steps in designing a study​​


Types of Studies

Statistical studies can be grouped according to the type of design. The four general types include those below. While it is not an exhaustive list, it will serve to illustrate some of the main differences.

Experimental Study

Experimental studies are designed to support claims of cause and effect. They typically involve a scenario in which a treatment, procedure, or program is intentionally introduced and a result or outcome is observed.

In its simplest form, the researcher applies a treatment to participants and records the effect of the treatment by comparing results between those who received the treatment (experimental group) and those who did not (control group).

A hallmark of the experimental study is this active manipulation of the treatment condition.

Observational Study

An observational study is a study where researchers simply collect data based on what is seen and heard and infer based on the data collected. The researcher observes and measures differences between individuals or groups without influencing or manipulating any part of the environment. Researchers should not interfere with the subjects or variables in any way. The treatment that each subject receives may be a pre-existing condition that is determined beyond the control of the investigator.

This is often described as a passive manipulation of conditions.

Jane Goodall's work with chimpanzees in the wild may be the best-known example of a series of observational studies conducted in a natural environment.

Simulation

Simulation involves modeling random events in such a way that the simulated outcomes closely match real-world outcomes. By observing these simulated outcomes, researchers gain insight into the real world of behaviors. The simulation may be role-played to reproduce, as accurately as possible, the conditions of a real-life situation. In some cases the researcher may use a mathematical or physical model, often a computerized program, to reproduce the conditions of a situation.

The Stanford University Prison Simulation Experiment conducted by Philip Zimbardo in 1971 is a famous (or infamous) example of a simulation experiment. It started out to be a simulation study using a mock prison environment but had to be curtailed early in the experiment. You'll see why in the accompanying video trailer.


Survey

The researcher investigates characteristics of a population by having participants respond to questions, usually by interview, telephone, or mail. Internet surveys have become popular, but may have low reliability where responses are voluntarily offered without there having been a random pre-selection of participants.   

A survey can be distinguished from an observational study in that a survey requires an interaction with the participant(s), while a true observational study does not.

An issue with surveys is that they are subject to bias as a result of the way questions are posed, and the way respondents are selected for participation. We'll have more to say about this later in the chapter when we discuss sampling methods.

Variables in Experimental Studies

In experiments, an independent variable is the variable being manipulated or changed in some way. This active manipulation is a defining characteristic of experiments.

  • In a study of the effect of sleep deprivation on reaction time, sleep deprivation would be the independent variable.
  • In a study of the effect of sunlight on plant growth, sunlight would be the independent variable. 

The dependent variable is the variable being studied and is expected to change in some way as a result of the manipulation of the independent variable. Its value is thought to be dependent on the independent variable.

  • In the sleep deprivation and reaction time study, reaction time would be the dependent variable.
  • In the sunlight and plant growth experiment, plant growth would be the dependent variable.
Figure 1.4: Independent and dependent variables

controlled variable is a potentially confounding variable that has been ruled out as a possible influence on the results.

  • In the reaction time study the confounding variable, time of day of the test, could be controlled by ensuring all participants participated in the reaction time test at the same time of day.
  • In the plant growth study, a potentially confounding variable might be differences in the soil composition the various plants were placed in. This could be controlled by ensuring all plants were grown in exactly the same soil mixture. 

Variables in Non-Experimental Studies

The chief difference between experimental and non-experimental studies is in the way the independent variable is identified and managed. In an experiment, the independent variable is actively manipulated by the researcher. In a non-experimental study, the independent variable usually has levels that are compared, but they are not actively manipulated. Instead, the levels of the variable are pre-existing, and the comparison is often described as being based on passive manipulation.



1.12

Identify the IV and DV in an experiment to see how stress level affects heart rate in humans.

Premise
Response
1

IV

A

Heart rate

2

DV

B

Gender

C

Stress level


1.13

Identify the IV and DV: A study of reaction time while texting and driving.

Premise
Response
1

IV

A

Driving a car

2

DV

B

Texting

C

Reaction time


1.14

Identify the IV and DV: A comparison of math SAT scores between male and female students.

Premise
Response
1

IV

A

SAT scores

2

DV

B

Gender

C

Student


1.15

Identify the IV and DV: A comparison of blood sugar levels before and after eating.

Premise
Response
1

IV

A

Blood sugar level

2

DV

B

After eating

C

Before eating

D

Food consumption


Question 1.16

1.16

In any of the previous questions about independent and dependent variables, what are some possible confounding variables that might need to be considered? How might they be controlled?

Click here to see the answer to Question 1.16.


Validity and Reliability

A well-designed study will be both valid and reliable. If not, any conclusions, or inferences, may be suspect.

Validity has to do with how accurate the study is. The question to be answered is "does it measure what it purports to measure?" To the extent that a study meets that criterion, it has experimental validity. Ask "is this measure relevant to the question being asked?"

Reliability has to do with consistency of the results over time and trials, which can be established by replicating the results. Replication of results is the gold standard for establishing reliability in social science research.

Replication of results involves repeating an experiment with other participants to confirm that the same results can be obtained in another setting.

Examples of Validity and Reliability

If a bathroom scale is reliable it tells you the same weight every time you step on it as long as your weight has not actually changed. However, if the scale is not working properly, this number may not be your actual weight. If that is the case, this is an example of a scale that is reliable, or consistent, but not valid. For the scale to be valid and reliable it not only needs to tell you the same weight every time you step on the scale, but it also has to measure your actual weight.

Figure 1.5: Reliability vs. validity​

Question 1.17

1.17

Standardized IQ tests have sometimes been criticized for perceived shortcomings in validity. What do you see as possible issues? How could these issues be addressed and resolved?

Click here to see the answer to Question 1.17.


Sampling Methods

Sampling refers to the method by which scores are obtained in order to make comparisons and/or draw conclusions. A sample is used to represent a population because it is difficult, or impossible, to measure every member of that population of interest. A sample is a subset of the population from which it is drawn, and of which it is representative. It is important to distinguish between taking a census of a population and sampling from the population.

Census: Counting or measuring some aspect of all members of a population.

Sampling: Gathering data from a group selected to represent a population.

In order to accurately represent the population it is drawn from, a sample must not be biased. It must be drawn in a way that ensures, to the maximum extent possible, that all sources of bias have been ruled out or controlled for. Generally, this is accomplished by using random sampling methods.

A biased sample is not representative of the population from which it is drawn. This bias introduces sampling error, defined as the difference between the sample results and the results that would have been obtained had the entire population been measured. To avoid bias, researchers use random selection.

Random selection of participants ensures that differences among them are distributed randomly, i.e., in a way that equalizes the effect across groups so that the effect of the difference is canceled out.

Selection Methods

Methods of random selection include using tables of random numbers to guide the selection or specially designed computer programs that can make selections from a database on a random basis.

The example often used for this is picking names from a (well-shuffled) hat. The idea is that all the names of members of a population are written on slips of paper and placed into a hat. The hat is shaken to shuffle the paper slips and then names are drawn, one at a time, and not replaced, until n names have been drawn.

​And FYI, using a table of random numbers or a random number generator is probably a more sound method than drawing paper slips from a hat.

Sample Size

In theory, we might have a sample of one or many, although extremely small sample sizes bring up issues of how well the sample represents the population as a whole and, as a result, the generalizability of the results to the population (validity and reliability). There will be more about this issue in a later chapter. 

The Science Behind the News video clip below points out the importance of ensuring that samples are selected in a way that ensures the results will, in fact, be representative of the population the results are intended to apply to. 



Random Sample

A random sample is generally defined as one in which each and every member of the population has an equal chance of being included. In practice, it is a generalized term referring to any method of sampling based on random selection. While the common denominator is always some form of randomization, in practice there are a number of "flavors" of random sampling.

Simple Random Sample (SRS)

A simple random sample of size n consists of n individuals from the population chosen in such a way that every set of n individuals has an equal probability of being selected.

  • "n" represents whatever number of objects or names the researcher has determined in advance to use as the sample size.

Key Properties of SRS

The key properties that define a sample as being a simple random sample are:

  • There is a population consisting of N objects or individuals.
  • A sample is selected consisting of n objects or individuals.
  • All possible samples of size n are equally likely to occur.

Note that capital N is used to represent the population and lower case n is used to represent the number of sample participants.

The basic procedure is to:

  • Identify the population of interest
  • Determine the sample size n
  • Randomly select n individuals so that every set of n individuals has an equal probability of being selected

Example: Simple Random Sampling

  • Population: All middle school students in the Maryvale school district (N=1874)
  • Method: Students randomly selected from a list of all middle school students in the district
  • Sample: 240 students

Stratified Sample

If a population is made up of identifiable subgroups, and the researcher wants to ensure these groups are equally represented in the sample, it can be stratified. The population is divided into groups, called strata, and a proportionate number of participants are randomly selected from each of these strata so that the number of participants in each stratum is determined by matching their proportion in the total sample size to their proportion in the population.

The basic procedure is to:

  • Divide the population into groups (strata)
  • Select a simple random sample from each group
  • Collect data from each sampling unit in each of the groups

Example: Stratified Sampling

  • Population: All middle school students in the Maryvale school district (N=1874)
  • Groups (strata): The 12 middle schools in the district
  • Method: 20 students randomly selected from each of the high schools
  • Sample: 12 × 20 = 240 students. 

A variation on stratified sampling is the disproportionate stratified sampling method, in which the different strata have different sampling fractions relative to the population. A caution here is that this method could tend to skew the results.

In stratified sampling, a proportionate number of the members of each subgroup are included in the sample.

Cluster Sample

A researcher may decide to use naturally occurring subgroups as the basis for a sample. The subgroups are often (but not always) based on some kind of geographic division. If each subgroup is likely to have the same general makeup relative to the purpose of the study, the researcher might elect to use one of these groups instead of sampling from the entire population. To the extent the selected group does not accurately represent the population, however, bias may be introduced. 

The basic procedure is to:

  • Divide the population into groups, or clusters
  • Obtain a random sample of clusters taken from the total clusters
  • Obtain data from every sampling unit in each of the randomly selected clusters

Example: Cluster Sampling

  • Population: All middle school students in the Maryvale school district (N=1874)
  • Groups (clusters): The 12 middle schools in the district
  • Method: 3 middle schools randomly selected from the total schools
  • Sample: Every student in the 3 selected middle schools

In cluster sampling, all members of one (or more) subgroup(s) are included in the sample.

Systematic Sample

In a systematic sample, members of the population are ordered in some way and assigned numbers. A starting point is randomly selected and every nth member selected. Alternatively, the researcher might divide the total population size by the desired sample size to obtain a sampling fraction, and then use that as the constant difference between participants. This has the advantage of avoiding any possibility of obtaining an unintentionally clustered sample using SRS.

The basic procedure is to:

  • Identify the population of interest
  • Determine the sample size n
  • Determine the algorithm to use, i.e., select a starting point and the constant difference between participants
  • Follow the algorithm to select n individuals

Example: Systematic Sampling

  • Population: All middle school students in the Maryvale school district (N=1874)
  • Method: Students selected from a list of all middle school students in the district following an algorithm giving the starting point and constant difference between selections
  • Sample: 240 students

In systematic sampling, participants are selected based on a predetermined algorithm.

This method is easy to use, and can often be done manually, but may introduce bias if there is any regularly occurring pattern in the data.

Convenience Sample

A convenience sample consists of whatever members of a population are readily available. Since no effort is made to control for confounding variables, this method often produces biased results. Convenience sampling is often used, for example, by university researchers who commonly select undergraduate students from their classes for participation in studies. This poses some obvious risks to the generalizability of the results.

The basic procedure is to:

  • Identify a readily available pool of participants
  • Assign participants from the pool to the sample

Example: Convenience Sampling

  • Population: Students or people in general
  • Method: Students from a history class assigned to participate in an experiment for extra credit
  • Sample: 24 students

In convenience sampling, participants are selected based on ready availability.

Convenience sampling may be most useful as a tool to be used in conjunction with exploratory studies; if the results indicate support for the research hypothesis, follow-up studies can be run to see if the results can be replicated with stricter controls.


1.18

Identify the type of sampling method that is used, or could best be used, in the studies below by matching the sampling methods to the appropriate scenario.

Premise
Response
1

A college professor assigns students from her Introduction to Psychology class to participate in a study of reaction time.

A

Simple random

2

A researcher wants to investigate how different age groups compare in terms of their ability to recall differing types of information.

B

Cluster

3

A researcher wants to spread participant selection across the entire population of interest in order to avoid any possibility of selecting too many participants from a "pocket" within the population.

C

Stratified

4

A survey is designed and presented to randomly selected participants from one zip code in a city because the survey budget is limited and this zip code appears to be a good representation of the overall makeup of the city's population.

D

Systematic

5

A college admissions officer uses a table of random numbers to select student IDs for a follow-up study to see if grades of graduating seniors actually correspond to pre-admittance entrance exam scores.

E

Convenience


A Day Without Statistics

Now, can you imagine a day without statistics? Or what that day might be like? The video here looks at the role of statistics in our everyday lives and poses some interesting questions. 


Case Study: Is Drinking Coffee Good for You? 

There have been on-and-off discussions in health-oriented publications about the effects, good or bad, of coffee consumption. Some point out downsides while some emphasize benefits.

Jee et al. (1999), conducted a quantitative review of clinical trials of the effect of long-term coffee consumption on blood pressure and reported that "coffee drinking was associated with a 2.4 mm Hg higher systolic blood pressure and 1.2 mm Hg higher diastolic blood pressure."

Meyers (2004) editorially reviewed several previously conducted studies and criticized them as providing "little evidence to suggest that habitual consumption . . . causes an increase in blood pressure of any clinical importance."

Beydoun et al. (2014) investigated the effects of several lifestyle factors on memory and cognition and reported "beneficial effects of caffeine intake . . . on domains of letter fluency, attention, and working memory."

Discussion Questions

Use the links in the References section below to locate the text of these studies (the Beydoun et al. (2014) study only has an abstract available) and respond to the discussion prompts below.

Case Study Question 1.01

Case Study Question 1.01

Identify the population of interest in these studies and, where possible, the sampling method that was, or might have been, used.

Click here to see the answer to Case Study Question 1.01.

Case Study Question 1.02

Case Study Question 1.02

Identify the independent and dependent variables and how they were measured.

Click here to see the answer to Case Study Discussion 1.02.

Case Study Question 1.03

Case Study Question 1.03

Identify potentially confounding variables that were, or should have been, controlled and how this might have been, or could be, accomplished.

Click here to see the answer to Case Study Question 1.03.

Case Study Question 1.04

Case Study Question 1.04

Identify the levels of measurement of the data collected and whether they are qualitative or quantitative.

Click here to see the answer to Case Study Question 1.04.

Case Study Question 1.05

Case Study Question 1.05

Discuss the inferences that were made, and suggest alternative explanations where appropriate.

Click here to see the answer to Case Study Question 1.05.

References

Jee, Sun Ha, He, Jang, Whelton, Paul, Suh, Il, & Klag, Micahel. The effect of Chronic Coffee Drinking on Blood Pressure. Hypertension, 1999; 33:647-652. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/10024321.

Myers, Martin G. Effect of Caffeine on Blood Pressure Beyond the Laboratory. Hypertension, 2004; 43:724-725. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/10024321.

Beydoun, May, Gamaldo, Alyssa, Beydoun, Hind, Tanaka, Toshiko, Tucker, Katherine, Talegawkar, Sameera, Ferrucci, Luigi, & Zonderman, Alan. Caffeine and Alcohol Intakes and Overall Nutrient Adequacy Are Associated with Longitudinal Cognitive Performance Among U.S. Adults. The Journal of Nutrition, 2014; 144;6, 890-901. Retrieved from http://jn.nutrition.org/content/144/6/890.abstract.


Pre-Class Discussion Questions

Class Discussion 1.01

Class Discussion 1.01

What is the key difference between a population and a sample?

Click here to see the answer to Class Discussion 1.01.

Class Discussion 1.02

Class Discussion 1.02

What is the difference between a parameter and a statistic?

Click here to see the answer to Class Discussion 1.02.

Class Discussion 1.03

Class Discussion 1.03

What is the difference between descriptive and inferential statistics?

Click here to see the answer to Class Discussion 1.03.

Class Discussion 1.04

Class Discussion 1.04

Distinguish between quantitative and qualitative data.

Click here to see the answer to Class Discussion 1.04.

Class Discussion 1.05

Class Discussion 1.05

Describe five types of samples, based on how the subjects' data were collected.

Click here to see the answer to Class Discussion 1.05.


Locked Content
This Content is Locked
Only a limited preview of this text is available. You'll need to sign up to Top Hat, and be a verified professor to have full access to view and teach with the content.


Answers to In-Chapter Questions

Answer to Question 1.16

A possible confounding variable would be the individual differences and variations in level of response to the stressor being imposed. This might be controlled by including a pre-test to quantify level of response.

Click here to return to Question 1.16.


Answer to Question 1.17

One possibility is the consideration of generalizability and whether results for, say, an African-American are appropriately interpreted in terms of norms that may or may not have been established on a like population.

Click here to return to Question 1.17.


Answers to Case Study Questions

Answer to Case Study Question 1.01

Sampling methods varied, as these studies were based on data drawn from previously conducted studies published in the literature. Populations of interest were coffee drinkers generally, with possible stratification within the population based on a variety of socio-economic and/or medical history conditions.

Click here to return to Case Study Question 1.01.


Answer to Case Study Question 1.02

IV = coffee consumption as measured by reported dosage (cups/day), usually by self-report.

DV = blood pressure as measured with typical blood pressure apparatus

Click here to return to Case Study Question 1.02.


Answer to Case Study Question 1.03

Possible confounding variables are virtually anything that could have a coincidental effect on blood pressure at the time of the trials. Stress level, environmental changes as a function of the setting, etc.

Click here to return to Case Study Question 1.03.


Answer to Case Study Question 1.04

Coffee consumption in cups = ratio level, quantitative

Blood pressure: there could be some discussion here about whether blood pressure is a true ratio level variable. 

Click here to return to Case Study Question 1.04.


Answer to Case Study Question 1.05

Based on correlational analysis, a relationship was generally found relating higher coffee consumption to higher blood pressure. Possible alternative explanations have to do with the possibility of outside factors influencing blood pressure readings and whether the blood pressure readings are transitory or represent a “real” effect related to the coffee dosing.

Click here to return to Case Study Question 1.05.



Answers to Pre-Class Discussion Questions

Answer to Class Discussion 1.01

A population describes an entire group, sometimes referred as the “population of interest,” while a sample is a subset of that entire group, selected in such a way as to be representative of that larger group.

Click here to return to Class Discussion 1.01.


Answer to Class Discussion 1.02

A parameter is a numerical description of some characteristic of a population, while a statistic is a numerical description of some characteristic of a sample. 

Click here to return to Class Discussion 1.02.


Answer to Class Discussion 1.03

Descriptive statistics are ways of organizing and presenting data, while inferential statistics are used to make generalizations about things we cannot measure directly.

Click here to return to Class Discussion 1.03.


Answer to Class Discussion 1.04

Quantitative data can be counted or measured and can be operated on mathematically, while qualitative data are descriptive but have no mathematical properties.

Click here to return to Class Discussion 1.04.


Answer to Class Discussion 1.05

Random samples involve selecting from a larger group in such a way that each and every sample of a given size has an equal chance of occurring.

Stratified sampling involves selecting so that a proportionate number of the members of each subgroup are included in the sample.

Cluster sampling involves selecting all members of one (or more) subgroup(s) for the sample.

Systematic sampling involves selecting participants based on a predetermined algorithm.

Convenience sampling involves selecting participants based on their ready availability

Click here to return to Class Discussion 1.05.



Image Credits

[1] Image courtesy of Martin Grandjean under CC BY-SA 3.0