Quality assurance of qualitative research: a suggested approach for assessing and strengthening qualitative research within global health trials

This article is part of the network’s archive of useful research information. This article is closed to new comments due to inactivity.  We welcome new content which can be done by submitting an article for review or take part in discussions in an open topic or submit a blog post to take your discussions online.

The concept of quality assurance of clinical trials is well established, as are the mechanisms and processes for enacting it, informed largely by the Good Clinical Practice (GCP) guidelines (European Medicines Agency 2002).  The GCP guidelines are designed to assure the protection of research participants, and the credibility and quality of trial data and results.  Questions have been raised about the suitability of GCP beyond drug registration trials, and the burden that adherence to the guidelines poses for researchers, particularly in low income countries (Lang, White et al. 2010).  However, GCP remains the universal standard for quality assurance of trials and has become a legal requirement for conducting clinical trials in a number of countries, leading to a culture of structured, standardised processes for maintaining and demonstrating quality in clinical research.

The picture is markedly different for qualitative research, however.  Social science research and qualitative methods are increasingly in demand in the field of ‘global health’ (Gilson, Hanson et al. 2011), including alongside, or embedded within, clinical trials to add deeper interpretations of the success – or otherwise – of interventions (Montgomery and Pool 2011).  This mounting appreciation of qualitative methods, and proximity to research with established quality assurance protocols, has driven demand for cohesive and universally-recognised guidance for assuring and improving the quality of qualitative research (Reynolds, Kizito et al. 2011), akin to that available for clinical trials research.    The principles of GCP reflect a positivist perspective of what quality is in research and how it can be assured, and this is at odds with the interpretive, subjective perspective underpinning the vast majority of qualitative research.  As such, the GCP guidance is neither appropriate nor easily applicable for use with qualitative research (ibid.), but the lack of suitable guidelines available for the qualitative researcher is unhelpful at best (Barbour 2001).

From our own experiences with conducting qualitative research within a group of projects investigating malaria diagnosis and treatment in nine countries in Africa and Asia (The ACT Consortium, www.actconsortium.org), we have needed to respond to demands for demonstrating the quality of our research processes and outcomes.   Alongside this, we have experienced challenges in assessing and maintaining the quality of day-to-day qualitative research activities when faced with spatial distances between senior investigators and field teams working in remote field sites.  Faced with an absence of cohesive guidance to support the strengthening of the quality or our work, we decided to explore further how to develop practical guidelines to this aim.  

Following a review of the literature around assuring quality of qualitative research (Reynolds, Kizito et al. 2011), we developed and piloted an approach through which qualitative research activities can be assessed and strengthened. We named this approach the ‘quality assessment and strengthening’ (QAS) approach.  This is based on a framework emerging from the literature review that presents quality assurance from a procedural perspective, underpinned by ‘key principles’ of good practice – for example, reflexivity, comprehensiveness, a systematic approach – rather than a checklist of specific techniques or methods.  QAS is a mechanism for generating feedback for research teams on their practice and how to improve, while data collection and analysis activities are ongoing.  We conceive of quality assurance as a set of practices that are undertaken through the lifetime of a study, from conception to interpretation and dissemination.  As such, we propose that QAS can address one stage of an ongoing quality assurance process. 

The aims of this approach are to provide qualitative research teams with an opportunity to:

1.receive assessment and constructive feedback on how their study is being conducted and ways in which they can improve; 

2.demonstrate to external audiences – eg funders, peer reviewers, policy makers - that their research has undergone a systematic assessment of quality.

This approach is informed by the practice of ‘monitoring’ in clinical trials research, which is commonly defined as  the act of overseeing the process and progress of a trial through a series of checks to ensure it is being conducted, recorded and reported in line with the protocol, SOPs and relevant regulatory requirements (Williams 2006).  Audits are commonly conducted as part of monitoring processes in clinical research, and our QAS approach also draws on this, recommending that research teams engage with social scientists external to the study to conduct the assessment of quality.  However, we have identified several differences between traditional ‘monitoring’ and our proposed QAS approach, to align the latter more closely with key principles of qualitative research:   

•Rather than assessing whether the qualitative research activities being conducted are ‘right’ or ‘wrong’, the QAS approach adopts an interpretive perspective to assessing quality, based on criteria appropriate to the specific research study conducted.

•The QAS approach presents a more collaborative method of assessing quality, whereby the priorities for assessment are decided through discussion between the research team and assessors, and the indicators for quality are agreed in advance by the research team and assessors, based on what is most appropriate for the specific study being assessed.

•In contrast with the ‘fault-finding’ approach commonly associated with audits of clinical research, the QAS approach seeks to offer qualitative research teams an opportunity to reflect upon their activities and the decisions they have made, in relation to the aims of the study, to strengthen their reflexive perspective.

•Rather than relying on study documents as the chief focus of quality assessment, the QAS approach recommends a combination of documentary, observational and discursive methods to assess to what extent the qualitative research activities reflect the quality criteria, and what improvements can be made.

We present here a sample protocol which outlines the QAS approach, consisting of an assessment visit conducted by independent assessors who feed back their interpretations and recommendations to the research team, who are expected to take responsibility for considering and acting upon the recommendations as part of a strategy to strengthen the quality of their research.  In summary, the protocol addresses: the methodological approach for assessing and strengthening quality; planning and prioritisation of research activities to be assessed; methods and indicators of quality; reporting; and developing an action plan for strengthening quality.  The protocol presents explanations for each section plus sample text that can be tailored to be relevant to specific studies and applications of the QAS approach.  It also includes sample tools for use by the assessors and research team, which can also be tailored to fit the particular study being assessed.  This protocol and its appendices were developed and piloted between 2011 and 2012, within the ACT Consortium by a team of social scientists who conducted a quality assessment of a qualitative study in Uganda.  A detailed and systematic evaluation of the pilot was undertaken, and subsequent revisions were made to the approach and documentation of the QAS approach.  

It is anticipated that investigators will recognise the value of QAS for their study as part of a broader strategy for quality assurance of the research process, and that they will approach external social scientists to conduct the assessment, guided by the sample QAS protocol and documentation we have developed.   As such, the QAS exercise would be suitable for a reciprocal quality assurance arrangement, similar to the reciprocal monitoring models that have been proposed in recognition of the need to build capacity for quality assurance of clinical research, particularly in low-income settings (Chilengi, Ogetii et al. 2010; Ogetii 2010).  In our pilot, two Ugandan social scientists who were part of the ACT Consortium carried out the QAS activities, visiting colleagues from a different ACT Consortium project elsewhere in Uganda.  We found this model effective and therefore suggest that groups of social scientists with experience in qualitative research could form a reciprocal arrangement for assessing each others’ studies.  In addition to building a collaborative relationship through the QAS process and potentially beyond, this would also build capacity for quality assurance of qualitative research in the future.

This QAS approach is intended to address only one stage of quality assurance of qualitative research, which we consider to be a researcher-led process that should underpin the entire course of research, from the early planning stages to the final writing up and dissemination.  This QAS protocol should not be considered a substitute for establishing an ongoing, robust quality assurance strategy for the whole qualitative research process.  We recognise that guidance on establishing such a strategy remains lacking but hope that the approach presented here can be a springboard for further debate about how to develop guidelines for quality assurance.  We welcome feedback and comments on the QAS presented here so that we may continue to revise and expand the resources available to qualitative researchers for assuring the quality of their research.