Conversations about sampling methods and sampling bias often take place at 60,000 feet. That is, researchers like to talk about the theoretical implications of sampling bias and to point out the potential ways that bias can undermine a study’s conclusions. Although these conversations are important, it is good to occasionally talk about what sampling looks like on the ground. At a practical level, what methods do researchers use to sample people and what are the pros and cons of each?
Non-random sampling techniques lead researchers to gather what are commonly known as convenience samples. Convenience samples are often based on who it’s easy for the researchers to contact. However, most online research does not qualify as pure convenience sampling. Often, researchers use non-random convenience sampling methods but strive to control for potential sources of bias. Here are some different ways that researchers can sample:
Voluntary sampling occurs when researchers seek volunteers to participate in studies. Volunteers can be solicited in person, over the internet, via public postings, and a variety of other methods. A researcher using voluntary sampling typically makes little effort to control sample composition.
A common form of voluntary sampling is the customer satisfaction survey. After a business provides a service or good, they often ask customers to report on their satisfaction. Because the business is asking all customers to volunteer their thoughts, the sample is voluntary and susceptible to bias.
Within academia, researchers often seek volunteer samples by either asking students to participate in research or by looking for people in the community. Within industry, companies seek volunteer samples for a variety of research purposes. Because volunteer samples are inexpensive, researchers across industries use them for a variety of different types of research.
Snowball sampling begins when researchers contact a few people who meet a study’s criteria. After those people complete the study, the researchers ask each person to recommend a few others who also meet the study criteria. By building on each participant’s social network, the hope is that data collection will snowball until the researchers reach enough people for their study.
Snowball sampling is an effective way to find people who belong to groups that are difficult to locate. For example, psychologists may use snowball sampling to study members of marginalized groups, such as homeless people, closeted gay people, or people who belong to a support group, such as Alcoholics Anonymous. After gaining the trust of a few people, the researchers could ask the participants to recommend some other members of the group. By proceeding from one recommendation to the next, the researchers may be able to gain a large enough sample for their project.
Snowball sampling is most common among researchers who seek to conduct qualitative research with hard-to-reach groups. Academic researchers might use snowball sampling to study the members of a stigmatized group, while industry researchers might use snowball sampling to study customers who belong to elite groups, such as a private club.
When researchers engage in quota sampling, they identify subsets of the population that are important to represent and then sample participants within each subset.
If you wanted to study Americans’ beliefs about economic mobility, it would be important to sample people from different steps on the economic ladder. That is, you would want to make sure your sample included people who make a lot of money, people who make a moderate amount of money, and some people who make a little bit of money. To obtain this sample, you might set up quotas that are stratified by people’s income. That is what one researcher recently did using CloudResearch’s Prime Panels.
Quota sampling is extremely common in both academic and industry research. Sometimes, researchers set simple quotas to ensure there is an equal balance of men and women within a study. At other times, researchers want to represent several groups and, therefore, set up more extensive quotas that allow them to represent several important demographic groups within a sample.
Judgment sampling occurs when a researcher uses his or her own judgment to select participants from the population of interest. The researcher’s goal is to balance sampling people who are easy to find with obtaining a sample that represents the group of interest. Hence, when using judgment sampling, researchers exert some effort to ensure their sample represents the population being studied.
Imagine a research team that wants to know what it’s like to be a university president. Compared to the entire population, very few people are or have been employed as the president of a university. Rather than rely on other sampling techniques that have a low probability of contacting university presidents, the researchers may choose a list of university presidents to contact for their study. By using their judgment in who to contact, the researchers hope to save resources while still obtaining a sample that represents university presidents.
Researchers within industry and academia sometimes rely on judgment sampling. Whenever researchers choose to restrict their data collection to the members of some special group, they may be engaged in judgment sampling.
Random sampling techniques lead researchers to gather representative samples, which allow researchers to understand a larger population by studying just the people included in a sample. Although there are a number of variations to random sampling, researchers in academia and industry are more likely to rely on non-random samples than random samples.
Simple random sampling is the most basic form of probability sampling. In a simple random sample, every member of the population being studied has an equal chance of being selected into the study, and researchers use some random process to select participants.
Researchers who want to know what Americans think about a particular topic might use simple random sampling. The researchers could begin with a list of telephone numbers from a database of all cell phones and landlines in the U.S. Then, using a computer to randomly dial numbers, the researchers could sample a group of people, ensuring a simple random sample.
Simple random sampling is sometimes used by researchers across industry, academia and government. The Census Bureau uses random sampling to gather detailed information about the U.S. population. Organizations like Pew and Gallup routinely use simple random sampling to gauge public opinion, and academic researchers sometimes use simple random sampling for research projects. However, because simple random sampling is expensive and many projects can arrive at a reasonable answer to their question without using random sampling, simple random sampling is often not the sampling plan of choice for most researchers.
Systematic sampling is a version of random sampling in which every member of the population being studied is given a number. Then, researchers randomly select a number from the list as the first participant. After the first participant, the researchers choose an interval, say 10, and sample every tenth person on the list.
Colleges and universities sometimes conduct campus-wide surveys to gauge people’s attitudes toward things like campus climate. To conduct such a survey, a university could use systematic sampling. By starting with a list of all registered students, the university could randomly select a starting point and an interval to sample with. Contacting every student who falls along the interval would ensure a random sample of students.
Systematic sampling is a variant of simple random sampling, which means it is often employed by the same researchers who gather random samples. Researchers engaged in public polling and some government, industry or academic positions may use systematic sampling. But, much more often, researchers in these areas rely on non-random samples.
Cluster sampling occurs when researchers randomly sample people within groups or clusters the people already belong to.
Imagine that researchers want to know how many high school students in the state of Ohio drank alcohol last year. The researchers could study this issue by taking a list of all high schools in Ohio and randomly selecting a portion of schools (the clusters). Then, the researchers could sample the students within the selected schools, rather than sampling all students in the state. By randomly selecting from the clusters (i.e., schools), the researchers can be more efficient than sampling all students while still maintaining the ability to generalize from their sample to the population.
Researchers who study people within groups, such as students within a school or employees within an organization, often rely on cluster sampling. By randomly selecting clusters within an organization, researchers can maintain the ability to generalize their findings while sampling far fewer people than the organization as a whole.
Multistage sampling is a version of cluster sampling. Multistage sampling begins when researchers randomly select a set of clusters or groups from a larger population. Then, the researchers randomly select people within those clusters, rather than sampling everyone in the cluster.
Researchers who want to study work-life balance and employee satisfaction within a large organization might begin by randomly selecting departments or locations within the organization as their clusters. If each cluster is large enough, the researchers could then randomly sample people within each cluster, rather than collecting data from all the people within each cluster.
Similar to cluster sampling, researchers who study people within organizations or large groups often find multistage sampling useful. Multistage sampling maintains the researcher’s ability to generalize their findings to the entire population being studied while dramatically reducing the amount of resources needed to study a topic.
Stratified sampling is a version of multistage sampling, in which a researcher selects specific demographic categories, or strata, that are important to represent within the final sample. Once these categories are selected, the researcher randomly samples people within each category.
Researchers at the Pew Research Center regularly ask Americans questions about religious life. To ensure that members of each major religious group are adequately represented in their surveys, these researchers might use stratified sampling. In doing so, researchers would choose the major religious groups that it is important to represent in the study and then randomly sample people who belong to each group. By using this technique, the researchers can ensure that even small religious groups are adequately represented in the sample while maintaining the ability to generalize their results to the larger population.
Stratified sampling is common among researchers who study large populations and need to ensure that minority groups within the population are well-represented. For this reason, stratified sampling tends to be more common in government and industry research than within academic research.
CloudResearch connects researchers with a wide variety of participants. Using our Prime Panels platform, you can sample participants from hard-to-reach demographic groups, gather large samples of thousands of people, or set up quotas to ensure your sample matches the demographics of the U.S. When you use our MTurk Toolkit, you can target people based on several demographic or psychographic characteristics. In addition to these tools, we can provide expert advice to ensure you select a sampling approach fit for your research purposes. Contact us today to learn how we can connect you to the right sample for your research project.
