Evaluating qualitative research

Even though qualitative research is often seen to be more loosely-defined, it is still possible to evaluate its quality, even though appropriate quality measures may differ slightly between qualitative and quantitative research. Let's take a look.

Reliability vs. Dependability

Reliability, or "repeatability," is important in quantitative research, hence the need for quantitative researchers to provide enough information so that their study can be replicated. Reliability, however, is not necessarily a good measure for evaluating fieldwork, as what people might say in an interview one day might be different to another day. Furthermore, you may get slightly different responses and results depending on the participants. In other words, you cannot step into the same stream twice. A slightly different, but more appropriate metric, is dependability, which represents the stability of the data over time.

Objectivity vs. Confirmability

As there is subjectivity required in processes such as coding, qualitative researchers cannot really strive for complete objectivity. Instead, confirmability may be a better metric. Confirmability assures that the data, interpretations and outcomes are rooted in the context and people studied, rather than in the personal biases of the researchers. In order to strive for confirmability, the decision-making trail, purposes and assumptions should all be made explicit. It may also be helpful to have a supportive but "critical friend" to be a sounding board as you interpret the data.

External validity vs. Generalisability

Generalisability is qualitative research's answer to external validity. Just like external validity, generalisability means that the findings should be applicable to other settings, and theories should be able to be expanded and generalised. Qualitative research also aims for generativity, which refers to the ways in which the work enables someone to see or act upon phenomena. Unlike in quantitative research, where external validity is usually judged by concrete factors such as sampling methods, the responsibility for judging the generalisability of qualitative studies rests on the readers of the study.

More about validity...

Validity refers to the degree to which the researchers' claims correspond to reality. Valid studies measure what they intend to measure. There are several different approaches towards the concept of validity in qualitative research.

• Parallel perspective / transactional notion of validity- This approach focuses on establishing a match between the realities of the researchers and the researched. Techniques used include triangulation and member checking (allowing the participants to read the transcripts and results and provide feedback).
• Diversification perspective / transformational approach- This approach focuses on "catalytic validity," or the degree to which the research process energised participants and gave them a better knowledge of reality. Ultimately, this approach focuses on whether the research made a difference.
• Letting go perspective- This perspective abandons the notion of validity completely and focuses instead on characterising traits of the research. A few sets of criteria for looking at these traits have been devised, though there is also the option of developing the criteria while engaging in inquiry.

References

Sparkes, AC & Smith, B 2013, Qualitative Research Methods in Sport, Exercise and Health: From Process to Product, Taylor & Francis Ltd, London.