1.4 – Common Research Designs

While our research questions determine the nature and size of the population, they also determine the ways we design our research studies.

In general, scientific research often is focused on exploring relationships between different concepts. For example, perhaps we want to know whether people who have more years of education tend to make more money, or whether people who get good grades in high school get good grades in college. However, some researchers are interested in whether one concept causes changes in another concept. For example, perhaps we want to know whether paying children to paint pictures reduces their intrinsic motivation to paint, or whether a new therapeutic intervention is effective in reducing anxiety. The difference between these two approaches comes down to whether or not there is an attempt to explore a causal relationship.

Most empirical research falls under these two general approaches, referred to as correlational or experimental research.

Correlational research refers to a research method that examines the relationship between concepts. Specifically, it looks at whether the two concepts “co-relate,” thus the word “correlation.”

It is important to recognize that correlational research cannot determine causal relationships. It only allows researchers to examine whether or not two variables are likely to be related to each other or not. In order to make any claims about causal relationships, a researcher must use an experimental research design.

Experimental research refers to a research method that examines whether one concept causes changes in another concept. This type of research methodology is more involved because it is tough to isolate a causal effect.

For example, let’s say that we want to determine if A causes B. In order to effectively establish a causal relationship, we would likely need to meet these three criteria:

1. A occurs before B
2. A and B are correlated to each other.
3. There are no other things besides A that might be affecting B.

In general, the first two criteria are often not too difficult to achieve, but it’s the third criterion, what is referred to by researchers as internal validity, that is the most problematic. And it is this criterion that drives the experimental method.

Let’s go back to our study on whether psychotherapy helps people with depression. Suppose researchers found a number of people who had been diagnosed with depression and asked them whether they were seeing a counselor for their depression or not. Then, ten weeks later, they measured each person’s level of depression and they find that those who had been seeing a counselor were less depressed. Can you conclude that the psychotherapy caused the improvement?

Let’s look at whether this study meets the three criteria. First, we seem to have established covariance; it does appear that there is a relationship between psychotherapy and depression because the people getting the psychotherapy were less depressed. Second, we seem to have established temporal precedence; the psychotherapy happened before they measured their depression level. However, we do not appear to have eliminated any other explanations for the reduction in depression levels. One possible explanation is that the participants who were seeing a counselor are more motivated, thus leading them to proactively do something to help with their depression. Motivated individuals are also likely to become less depressed on their own. Thus, we can’t say that psychotherapy caused the improvement in depression levels.

However, if we make some adjustments to the research study and apply the experimental method we would be able to examine a causal effect. To run an experiment, researchers manipulate one of the variables (A) and then measure the other variable (B). When we say manipulate, what we mean is that the researcher controls what happens to the participants. Typically a research experiment will involve different conditions that participants might experience. For example, in one condition, participants might get a pain relief drug, and in the other condition, participants might get a placebo (a pill that has no active drug in it). The researcher will then control the condition in which each participant is placed by randomly assigning them to a condition. In this example, the researcher “manipulated” the conditions in which the participants were placed. Now the researchers can examine whether the pain relief drug caused a reduction in the experience of pain by exploring differences in the the pain levels experienced in each condition.

In our psychotherapy example, the researchers would take a group of participants who were diagnosed with depression and then randomly assign half of the participants to receive ten weeks of counseling, while the other half would not receive counseling. By manipulating who receives counseling, you are preventing the personal characteristics of the participants (such as motivation) from influencing whether or not they receive counseling, thus removing those alternative explanations for any reduction in depression level.

If done well enough so that the only difference between the two groups of participants is that one group got counseling and the other did not, then when you measure the depression level at the end of the ten weeks, any difference has to be due to whether or not they received counseling. We have now established all three components of causality: covariance, temporal precedence, and no other explanations. Thus, you have established that psychotherapy causes a decrease in depression.

The Unique Case of Quasi-Experiments

One difficulty with experimental research involves the fact that in many situations researchers would like to look at a causal effect for a particular condition or experience, but it may be impossible to manipulate that condition. For example, suppose a researcher wanted to research if experiencing an earthquake causes people to feel more anxiety. To do this research and be able to establish that causal effect, the researcher would have to do an experiment, which means that they would have to manipulate which participants experience an earthquake and which ones don’t. In other words, they would have to randomly assign people to experience an earthquake. Obviously, this would be impossible because we can’t always predict when and where an earthquake will happen, and even if we could we certainly couldn’t force people to move to that area or move out of that area at the designated time.

Instead, the researcher could find people who just happened to have experienced an earthquake recently and then compare their anxiety levels to a group of people who had not experienced an earthquake recently. Thus, the conditions in this study are not manipulated or controlled by the researcher. This type of research study is called a quasi-experiment. The prefix “quasi-” can be thought of as meaning “sort of.” Thus, this type of research study is “sort of an experiment.” It has a lot of the characteristics of a “true experiment” (e.g., there are different conditions, and all participants are measured on the same thing) but it lacks the key feature of manipulating the conditions to which participants are assigned.

It is important to note that when a researcher uses a quasi-experiment, it is impossible to establish the third rule of causation that there are no other explanations for any changes in the measured outcome. Take our earthquake research, for example. Let’s say that we find that the earthquake group has more anxiety than the no earthquake group. Because we didn’t manipulate whether or not are participants experienced an earthquake, the differences in anxiety between the two conditions might be due to something else besides the experience of an earthquake. We might find that our participants who have experienced an earthquake recently all live in a particular region in California, while our participants who have not recently experienced an earthquake live somewhere else. And maybe we were doing the research during a time when there were also massive wildfires in California. It’s possible that the anxiety in the earthquake condition was more due to the wildfires than the earthquakes.

Ultimately, it is important that we are aware that the only time that researchers can make causal claims is when they’ve done a true experiment.