Sampling Bias
In the United States, and other Western countries, it is common to recruit university undergraduate students to participate in psychological research studies. Using samples of convenience from this very thin slice of humanity presents a problem when trying to generalize to the larger public and across cultures. Aside from being an over-representation of young, middle-class Caucasians, college students may also be more compliant and more susceptible to attitude change, have less stable personality traits and interpersonal relationships, and possess stronger cognitive skills than samples reflecting a wider range of age and experience (Peterson & Merunka, 2014; Visser, Krosnick, & Lavrakas, 2000).
Put simply, these traditional samples (college students) may not be sufficiently representative of the broader population. Furthermore, considering that 96% of participants in psychology studies come from western, educated, industrialized, rich, and democratic countries (so-called WEIRD cultures; Henrich, Heine, & Norenzayan, 2010), and that the majority of these are also psychology students, the question of non-representativeness becomes even more serious.
When studying a basic cognitive process (e.g., working memory) or an aspect of social behavior that appears to be fairly universal (e.g., cooperation), a non-representative sample may not be a big deal but over time research has repeatedly demonstrated the important role that individual differences (e.g., personality traits and cognitive abilities) and culture (e.g., individualism vs. collectivism) play in shaping social behavior.
For instance, even if we only consider a tiny sample of research on aggression, we know that narcissists are more likely to respond to criticism with aggression (Bushman & Baumeister, 1998); conservatives, who have a low tolerance for uncertainty, are more likely to prefer aggressive actions against those considered to be “outsiders” (de Zavala et al., 2010); countries where men hold the bulk of power in society have higher rates of physical aggression directed against female partners (Archer, 2006); and males from the southern part of the United States are more likely to react with aggression following an insult (Cohen et al., 1996).
When conducting research across cultures it is important to ensure that there is equivalence across samples from other cultures to maintain the internal consistency (validity) of the research study (Harzing, et al., 2013; Matsumoto and Juang, 2013). Asking middle-school students in the United States about their online shopping experiences may not be a representative sample for middle school students in Kenya, Africa. Even when trying to control for demographic differences there are some experiences that cannot be separated from culture (Matsumoto and Luang, 2013). For example, being Catholic in the United States does not have the same meaning as being Catholic in Japan or Brazil. Researchers must consider the experiences of the sample in addition to basic demographic information.
