Jill Russell, RDS London’s Qualitative Research Methods Lead discusses how you can quantify your qualitative research.
When we think about qualitative research, we tend not to think so much about numbers, that’s the domain of quantitative studies, we assume. In line with this thinking, a common problem RDS advisers see when supporting researchers in developing their qualitative proposals is a vagueness about sample size and a lack of justification for sample selection.
We often find ourselves asking researchers questions such as: how many participants are you intending to interview? Or, if they have specified a number: Why have you chosen that particular number? Why not more or less? One NIHR funding panel provided this feedback to an unsuccessful applicant: “The panel noted that the description of the qualitative work lacked clarity and the justification for sample selection was inadequate”.
So, what advice can we give to researchers about quantifying their qualitative work?
First, quantification is important and needs to be done! True, qualitative research is not based on the probabilistic sampling of quantitative research, and the statistical generalisation that follows from that. But it’s essential that qualitative proposals are explicit and concrete about the basis on which a sample is to be selected. There are no hard and fast rules regarding what constitutes the right sample size for a qualitative study, but what’s important is that the researcher makes a coherent and defensible case. ‘Specify’ and ‘justify’ are two key words of advice we give to researchers.
It helps to bear in mind that what qualitative samples are invariably aiming for what Ritchie and Lewis refer to as ‘symbolic representation’, where units of a sample are chosen to both ‘represent’ and ‘symbolise’ features of relevance to the investigation (Ritchie and Lewis, 2003).
Another goal of qualitative sampling is to reflect diversity within the boundaries of the defined population. We often recommend to researchers that they include a sampling grid in their proposal, clearly mapping out the primary selection criteria, with an indication of numbers, or range of numbers, for each cell.
Criteria might include age, gender, regional location, or how regularly a participant attends a particular service, for example. While such a tool is likely to reassure funders and can certainly be a useful aid to thinking through sampling choices and decisions, it is important that the grid does not become an immutable template. Use it as a guide and a reflective prompt, not as something that sets your sample size in stone.
Second, make sure you understand the terms you use and use them appropriately. If you don’t properly understand a term, it’s best to avoid using it at all.
The most common culprit here that we see as advisers is the term ‘data saturation’. We see statements such as “we plan to recruit approximately 30 individuals or until data saturation is achieved”, or “qualitative interviews will be conducted on a subset of patients until data saturation”. Nor is this sort of unreflective use of the term restricted to novice researchers; a recent textbook on using qualitative research with randomized controlled trials states that “Data saturation is a good justification for final sample size in studies” (O’Cathain, 2018). It is not, necessarily.
Data (or theoretical) saturation is a complex concept, deriving from the theoretical sampling approach of a grounded theory research design advocated by Strauss and Corbin (Strauss & Corbin, 1998). In this approach, a researcher undertakes data analysis alongside data collection, developing theoretical ideas emerging from the data, and keeps sampling, and analysing, until no new ideas are coming out of the data. It implies that “the researcher has obtained sufficient depth and richness and has accounted for all variance required to justify theoretical claims about the full range of configurations within the social process in question” (Thorne, 2016).
While this can be a valid methodological approach, and well suited to some studies engaged in in-depth social theorising, the term ‘data saturation’ is frequently inappropriately used in qualitative proposals. Often proposals are using a purposive, not theoretical, sampling approach, and are undertaking data sampling, collection, and analysis in a linear sequence, rather than in an iterative back-and-forth fashion, and typically are using a thematic not grounded theory approach to analysis. Several qualitative methodologists have voiced concerns about the inappropriate use of the term ‘data saturation’ as a justification for determining sample size and concluding data collection.
Questions have even been raised about whether in fact, in qualitative research, there ever is or can be a point at which no new variation or ideas could emerge. At a more practical level, there is the argument that funding bodies may not be open to the sort of open-ended commitment to data collection that theoretical sampling implies.
Third, don’t shy away from being honest in identifying time and resources as a reasonable element in constraining your sample size. Qualitative research is highly intensive in terms of the research resources it requires. Being concrete and explicit about the numbers involved will enable you to work out, for example, how many hours of interview recordings you will collect, what resources you will need for transcribing and analysis, and so on.
Working through this level of detail at the proposal stage can often help refine the scope and focus of the study, ensuring a better fit between aspirations and resources.
References
Ritchie J and Lewis J (2003)
Qualitative Research Practice, London, Sage.
O’Cathain A (2018)
Using Qualitative Research with
Randomized Controlled Trials, Oxford, OUP.
Strauss A.L and Corbin J (1998)
Basics of Qualitative Research: Grounded Theory Procedures and Techniques, Second edition,
Thousand Oaks, Ca: Sage.
Thorne S (2016)
Interpretive Description, Qualitative Research for Applied Practice. Second edition, Abingdon, Oxon, Routledge.
Online resources
By the way, we have a range of resources about qualitative research in our resources section.
