Diving into the qual-quant decision
By Shannon Sharp
Evaluators and researchers have long debated the merits of quantitative and qualitative methods, using terms like “gold standard” credibility and rigor. But what do these categories mean? Is one better than the other? Do we have to choose? Spoiler alert: NO—You do not have to fall strictly on one side or the other. Let’s take a dive into the methods pool and use your evaluation questions to decide what methods best fit your needs. You may even find that you’re most comfortable floating in the middle, where quantitative and qualitative methods flow together.
The shallow end: Breadth
Going along with our pool analogy, wading in the shallow end would represent quantitative methods. These, like the shallow end of the pool, are more controlled and surface-deep, meaning they can encompass a larger population. Quantitative data are numbers-based and utilize standardized methodology. Some examples of quantitative data often used in evaluations include close-ended survey responses, test scores and various types of secondary data. These data are used to answer questions about what is happening and to what extent. These data are often analyzed with powerful statistical tests for generalization of larger populations. To go beyond the surface and answer questions about how and why things are happening, we look to qualitative methods.
The deep end: Depth
Qualitative methods, such as interviews, focus groups and open-ended survey items take us into the deep end of the pool by providing greater context and deeper meaning into complex, individual experiences. Qualitative data help us answer questions such as “How?” and “Why?” because they are adaptable and provide more individualized information. The drawback here is that, while themes may emerge, because information gleaned from qualitative data is inherently individualized, they cannot be generalized to the larger population like quantitative data can.
Getting your feet wet
Now that we’ve gone through the basics of quantitative and qualitative methodology, let’s think about how to select the most appropriate methods for your evaluation questions. As noted above, quantitative methods are best suited for evaluation questions that ask “What?” and “To what extent?”
For example, to evaluate the extent to which a teacher training program impacted student achievement, I analyzed the students’ standardized test scores. This provided statistical evidence of student achievement. On the other hand, I also wanted to understand how and why the training program impacted student learning. For this, I relied on qualitative data from interviews with the teachers about how they implemented what they learned in the trainings and how their students responded.
Floating in the middle
As you’ve probably realized by now, I think selecting only quantitative or qualitative methods for your evaluation will often leave you high and dry—wishing that your results either carried more power or meaning. That is why so many evaluations employ a mixed-methods approach, like in my example above, using qualitative methods to support and enhance quantitative findings.
