Introduction

Where deep ethnography (chapter 14) is a tradition that relies on naturalistic techniques of data collection, foregrounding the specificity of a particular culture and site, there are other times when researchers are looking for approaches that allow them to make use of some of the analytical techniques developed by statisticians and quantitative researchers to generalize the data they are collecting. Rather than push into a deeper understanding of a culture through thick interpretive descriptions, these researchers would rather abstract from a sufficiently large body of cases (or persons) to hazard predictions about a connection, relationship, or phenomenon. You may already have some experience learning basic statistical techniques for analyzing large data sets. In this chapter, we describe how some research harnesses those techniques to supplement or augment qualitative research, mixing methods for the purpose of building stronger claims and arguments. There are many ways this can be done, but perhaps the most common mixed methods research design involves the use of survey data (analyzed statistically via descriptive cross-tabs or fairly simple regression analyses of large number probability samples) plus semistructured interviews. This chapter will take a closer look at mixed methods approaches, explain why you might want to consider them (or not), and provide some guidance for successful mixed methods research designs.

What Is It? Triangulation, Multiple Methods, and Mixed Methods

First, a bit of nomenclature. Mixed methods can be understood as a path toward triangulation. Triangulation is a way of strengthening the validity of a study by employing multiple forms of data, multiple investigators, multiple theoretical perspectives, or multiple research methods. Let’s say that Anikit wants to know more about how first-year college students acclimate to college. He could talk to some college students (conduct interviews) and also observe their behavior (fieldwork). He is strengthening the validity of his study by including multiple forms of data. If both the interview and the observations indicate heavy reliance on peer networks, a reported finding about the importance of peers would be more credible than had he only interviewed students or only observed them. If he discovers that students say one thing but do another (which is pretty common, after all), then this, too, becomes an interesting finding (e.g., Why do they forget to talk about their peers when peers have so much observable influence?). In this case, we say that Anikit is employing multiple forms of data, or even that he relies on multiple methods. But he is not, strictly speaking, mixing data. Mixed methods refer specifically to the use of both quantitative and qualitative research methods. If Anikit were to supplement his interviews and/or observations with a random sample of one thousand college students, he would then be employing a mixed methods approach. Although he might not get the rich details of how friends matter in the survey, the large sample size allows statistical analyses of relationships among variables, perhaps showing which groups of students are more likely to benefit from strong peer networks. So to summarize, both multiple methods and mixed methods are forms of research triangulation,^[1] but mixed methods include mixing both qualitative and quantitative research elements.

Mixed methods techniques, then, are pretty unique. Where many qualitative researchers have little interest in statistical generalizability, and many quantitative researchers undervalue the importance of rich descriptions of singular cases, the mixed methods researcher has an open mind about both approaches simultaneously. And they use the power of both approaches to build stronger results:^[2]

Quantitative (mainly deductive) methods are ideal for measuring pervasiveness of “known” phenomena and central patterns of association, including inferences of causality. Qualitative (mainly inductive) methods allow for identification of previously unknown processes, explanations of why and how phenomena occur, and the range of their effects (Pasick et al. 2009). Mixed methods research, then, is more than simply collecting qualitative data from interviews, or collecting multiple forms of qualitative evidence (e.g., observations and interviews) or multiple types of quantitative evidence (e.g., surveys and diagnostic tests). It involves the intentional collection of both quantitative and qualitative data and the combination of the strengths of each to answer research questions. (Creswell et al. 2011:5; emphases added)

Why Use Mixed Methods?

As with all methodological choices, the answer depends on your underlying research questions and goals. Some research questions are better answered by the strengths of the mixed methods approach. Small (2011) discusses the use of mixed methods as a confirmation or complement of one set of findings from one method by another. Creswell and Clark (2017:8ff.) note the following situations as being particularly aided by combining qualitative and quantitative data collection and analysis: (1) when you need to obtain both more complete (need for qualitative) and more corroborated (need for quantitative) information; (2) when you need to explain (need for qualitative) initial results (quantitative); (3) when you need to do an exploratory study (need for qualitative) before you can really create and administer a survey or other instrument (quantitative); (4) when you need to describe and compare different types of cases to get a more holistic understanding of what is going on; (5) when you need (or very much want!) to include participants in the study, adding in qualitative elements as you build a quantitative design; (6) when you need all the tools at your disposal to develop, implement, and evaluate a program.

Please note what is not included in this list: because you can. Mixed methods research is not always preferable, even if in general it makes your study “stronger.” Strength is not the only criterion for quality or value. I have met many students in my career who assume that the mixed methods approach is optimal because it includes both qualitative and quantitative research. That is the wrong way of looking at things. Mixed methods are optimal when and only when they fit the necessities of your research question (e.g., How can I corroborate this interesting finding from my interviews so that proper solutions can be fashioned?) or underlying goal (e.g., How can I make sure to include the people in this program as participants of the study?).

If you are just starting out and learning your way through designing your first study, mixed methods are not default requirements. As you will see in the next section on design, mixed methods studies often happen sequentially rather than consecutively, so I recommend you start with the study that has the most meaning to you, the one that is the most compelling. Later on, if you want to add (mix) another approach for the sake of strength or validity or “corroboration” (if you are adding quantitative) or “explanation” (if you are adding qualitative), you can always do that then, after the completion of your first study.

Segue: Historical Interlude

For those interested in a little history, one could make the case that mixed methods research in the social sciences actually predates the development of either quantitative or qualitative research methods. The very first social scientists (what we call “social science” in the West, which is itself a historical construct, as many other peoples have been exploring meaning and interpretation of the social world for centuries if not millennia) often employed a mélange of methods to address their research questions. For example, the first sociologists in the US operating out of the “Chicago School” of the early twentieth century surveyed neighborhoods, interviewing people, observing demographic subcultures, and making tallies of everything from the numbers of persons in households to what languages were being spoken. They learned many of these techniques from early statisticians and demographers in Europe—people like Charles Booth (1902), who surveyed neighborhoods in London, and Frédéric Le Play, who spent decades examining the material conditions of the working classes across Europe, famously including family “budgets” along with interviews and observations (see C. B. Silver 1982). The renowned American sociologist W. E. B. Du Bois, who was the first Black man to earn a PhD from Harvard University, also conducted one of the very first mixed methods studies in the US, The Philadelphia Negro (1899). This work mapped every Black residence, church, and business in Philadelphia’s Seventh Ward and included observations and details on family structure and occupation (similar to Booth’s earlier work on London). Continuing through the 1930s and 1940s, “community studies” were conducted by teams of researchers who basically tallied everything they could find about the particular town or city they chose to work in and performed countless interviews, months and years of fieldwork, and detailed mappings of community relationships and power relations. One of the most famous of these studies includes the “Middletown” studies conducted by Robert and Helen Lynd (1929, 1937).

As statistical analysis progressed after World War II alongside the development of the technology that allowed for ever faster computations, quantitative research emerged as a separate field. There was a lot to learn about how to conduct statistical analyses, and there were more refinements in the creation of large survey instruments. Qualitative research—the observations and interviews at the heart of naturalistic inquiry—became a separate field for different kinds of researchers. One might even say qualitative research languished at the expense of new developments of quantitative analytical techniques until the 1970s, when feminist critiques of positivist social science emerged, casting doubt on the superiority of quantitative research methods. The rise of interdisciplinarity in recent decades combined with a lessening of the former harsh critique of quantitative research methods and the “paradigm wars” (Small 2011) has allowed for an efflorescence of mixed methods research, which is where we are today.

Mixed-Methods Research Designs

Returning from our historical interlude to the list of possible uses of mixed methods, we now confront the question of research design. If we are using more than one method, how exactly do we do this, and when? The how and the when will depend largely on why we are using mixed methods. For example, if we want to corroborate findings emerging from interviews, then we obviously begin with interviews and follow with, perhaps, a large survey. On the other hand, if we are seeking to explain findings generated from a survey, we begin with that survey and add interviews or observations or focus groups after its completion. And if we are seeking to include participants in the research design itself, we may want to work concurrently, interviewing and holding focus groups as surveys are administered. So it all depends on why we have chosen to use mixed methods.

We can think of our choices here in terms of three possibilities. The first, called sequential explanatory, begins with quantitative data (collection) and then follows with qualitative data (collection). After both are collected, interpretations are made. The second, called sequential exploratory, begins the other way around, with qualitative followed by quantitative. After both are collected, interpretations are made. The third, called concurrent triangulation, conceives of both quantitative and qualitative elements happening concurrently. In practice, one may still happen before the other, but one does not follow the other. The data then converge, and from that convergence, interpretations are made.

In sequential explanatory design (figure 15.1), we are asking ourselves, “In what ways do the qualitative findings explain the quantitative results?” (Creswell et al. 2017). This design thus gives some priority to the quantitative data. The qualitative data, collected after the quantitative data, is used to provide a better understanding of the research problem and then the quantitative data alone.

Often, this means providing some context or explaining meanings and motivations behind the correlations found in the quantitative data. For example, in my research on college students (Hurst 2019), I found a statistical correlation between upper-middle-class female students and study abroad. In other words, and stating this rather baldly, class*gender could be used to predict who studied abroad. But I couldn’t fully explain why, given the survey data I had collected.^[3] To answer these (and other) questions that the survey results raised, I began interviewing students and holding focus groups. And it was through these qualitative forms of data collection that I found a partial answer: upper-middle-class female students had been taught to see study abroad as a final “finishing” component of their education in a way that other students simply had not. They often had mothers who had done the same. And they clearly saw connections here to the kinds of well-traveled cosmopolitan adults they wanted to become.

In sequential exploratory design (figure 15.2), we are asking ourselves, “In what ways do the quantitative findings generalize (or confirm) the qualitative results?” (Creswell et al. 2018). This design thus gives some priority to the qualitative data. The quantitative data, collected after the qualitative data, is used to confirm the findings.

This approach is ideal for developing new instruments or when a researcher intends to generalize findings from a qualitative study to different groups or populations. The American Sociological Association (ASA) Task Force on First-Generation and Working-Class Persons wanted to understand how class background may have played a role in the success of sociology graduate students and faculty. Because this was a relatively new research question, the task force began by conducting several focus groups, asking general questions about how class might have affected careers in sociology. Based on several recurring findings (e.g., high debt burdens, mentorship, feelings of fit), the task force developed a survey instrument that it then administered to more than one thousand sociologists, thus generalizing the preliminary findings and providing corroboration of some of the key variables at play.

In concurrent triangulation design (figure 15.3), neither the quantitative nor the qualitative component takes precedence. Although in practice one might precede the other in time, neither is the tail that wags the dog, so to speak. They are both the dog. The general of this design is to better understand or deepen one’s understanding of the phenomenon under study. The goal is to obtain different but complementary data that strengthen (validate) the overall results.

These designs might be either nested or nonnested. In a nested design, a subsample of an original randomized sample is used for further interviews or observation. A common nested design form is where in-depth interviews are conducted with a subsample of those who filled out a survey. Nonnested designs occur when it is impractical or impossible to recruit the same individuals that took place in the survey. The research I conducted for my book Amplified Advantage (Hurst 2019) is an example of this. I supplemented a large national survey of college students and recent college graduates with interviews and focus groups of similar college students and graduates who were not participants in the study (or who may have been randomly selected as participants but without my knowledge or linking their data). Nonnested designs are much more flexible than nested designs, but they eliminate the possibility of linking data across methods.

As with all research design, it is important to think about how best to address your particular research question. There are strengths and weaknesses of each design. Sequential design allows for the collection and analysis of different methods separately, which can make the process more manageable. Sequential designs are relatively easy to implement, design, and report. Sequential exploratory designs allow you to contextualize and generalize qualitative findings to larger samples, while sequential explanatory designs enable you to gain a deeper understanding of findings revealed by quantitative data analysis. All sequential design takes a lot of time, however. You are essentially doubling your research. This is why I do not recommend these approaches to undergraduate students or graduate students in master’s programs. In contrast, concurrent designs, whose dual methods may be conducted simultaneously, may be conducted more quickly. However, as a practical matter, you will probably end up focusing first on one data collection method and then the other, so the time saved might be minimal.^[4] Concurrent design can also preclude following up on interesting findings that emerge from one side of the study, and the abbreviated form may prevent clarification of confusing issues that arise during analysis. If the results are contradictory or diverge, it may also be difficult to integrate the data. You might end up with more questions to pursue for further study and not much conclusive to say at the end of all your work.

Finally, there is what I will call here the recursive design model (figure 15.4), in which you combine both explanatory and exploratory sequential design.

This design is currently being used by the ASA task force mentioned above. The first stage of data collection involved several focus groups. From these focus groups, we constructed a survey that we administered to ASA members. The focus group survey could be viewed as an example of exploratory sequential design. As the surveys were being analyzed, we added a nested set of interviews with persons who had taken the survey and who indicated a willingness to participate in this later stage of data collection. These interviews then help explain some of the findings from the survey. The entire process takes several years, however, and involves multiple researchers!

Advanced: Crossover Design

Small’s (2011) review of the state of mixed methods research argues that mixed methods are being increasingly adopted in social science research. In addition to sequential and concurrent research designs, where quantitative and qualitative data work to either confirm or complement each other, he sets forth examples of innovative designs that go further toward truly blending the special techniques and strengths of both quantitative and qualitative methods.^[5] Written in 2011, I have seen scant evidence so far that these blended techniques are becoming well established, but they are promising. As new software programs for data analysis emerge, along with increased computing power, there will be greater opportunities for crossover work. Perhaps you can take up the charge and attempt one of these more innovative approaches yourself.

Here is Small’s (2011:73ff.) list of innovative crossover research design:

1. Network analyses of narrative textual data. Here, researchers use techniques of network analysis (typically quantitative) and apply them to narratives (qualitative), coding stories as separate “nodes” and then looking for connections between those nodes, as is done in network analysis.
2. Sequence analyses of narrative textual data. Here, techniques of event structure analysis and optimal matching (designed for analysis of quantitative data) are applied to narratives (qualitative data). The narratives are reconceived as a series of events, and then causal pathways between these events are mapped. This allows for identification of crucial turning points as well as “nonsignificant” events that just happened.
3. Quantitative analyses of semantic (meaning) elements of narrative textual data. The basic distinction between quantitative (data in the form of numbers) and qualitative (date in the form of words) gets blurred here, as words themselves and their meanings and contexts are coded numerically. I usually strongly advise beginning students to do this, as what often happens is that they begin to think quantitatively about the data, flattening it considerably. However, if done with full attention to meaning and context, the power of computing/analytical software may strengthen the coding process.
4. Narrative analyses of large-n survey data. In contrast to the first three designs listed above, where quantitative techniques were applied to qualitative data, we now come to a situation where the reverse takes place. Here we have a large data set, either coded numerically or “raw” with various choice options for each question posed. Rather than read the data set as a series of factors (variables) whose relationship one explores through statistical analyses, the researcher creates a narrative from the survey responses, contextualizing the answers rather than abstracting them.^[6]
5. Regression-based analyses of small-n or narrative textual data. This is by far the most common crossover method and the reverse of the fourth example. Many qualitative software analysis programs now include basic quantitative analytical functions. The researcher can code interview transcripts and fieldnotes in such a way that allows for basic cross-tabulations, simple frequency statistics, or even basic regression analyses. Transcripts and fieldnotes can generate “variables” for such analyses.

Despite the promise of blending methods in this way, the possibility of doing damage to one’s study by discounting the particular values of either quantitative or qualitative approaches is a real one. Unlike mixed methods, where the two approaches work separately (even when designed to concur in time), crossover research blends or muddies the two. Small (2011) warns, “At a minimum, the application of techniques should not be fundamentally contrary to the epistemological principles from which they are derived or to the technical problems for which they were intended” (76). When employing any of these designs or blending approaches, it is very important to explain clearly and fully what one’s aims are and how the analysis has proceeded, as this allows others to evaluate the appropriateness of the design for the questions posed.