Health Technology Assessment 1998 Vol. 2: No. 16. (Executive Summary)
Executive summary
View/Download full
monograph in Adobe Acrobat format (1036 kbytes)
View/Download this
4-page summary in Adobe Acrobat format (suitable for printing)
Qualitative research methods in health technology assessment: a review of the literature
E Murphy
R Dingwall
D Greatbatch
S Parker
P Watson
School of Sociology and Social Policy, University of Nottingham, Nottingham, UK
Background
Qualitative research involves the collection, analysis and interpretation of data that are not easily reduced to numbers. These data relate to the social world and the concepts and behaviours of people within it. Qualitative research can be found in all social sciences and in the applied fields that derive from them, for example, research in health services, nursing and pharmacy. These research methods are not a recent invention but their application in health technology assessment (HTA) or health services research may be novel. In order for commissioners and researchers to utilise these methods and gain valuable knowledge from the results obtained, it is important that they understand the principles of qualitative methods and the way that they may be used to set benchmark standards.
Objectives
The objective of this review was to examine the nature and status of qualitative methods in relation to their potential uses in HTA.
Methods
The search tools often used for systematic reviews were not appropriate for this review as
it would be necessary to cover the equivalents of MEDLINE in a range of disciplines and
applied fields, many of which do not have databases of comparable coverage. In addition,
important methodological writing in the field of social science started long before
indexing for computer databases, and much of the most significant work has been published
in books rather than journals.
Having set the boundaries and organised the categories for this review, therefore, the authors read as widely and as comprehensively as was feasible in the time available. The authors have compared different researchers' approaches to the same issue and examined the work of authors who offer different perspectives.
Perspective
Idealists versus realists
Qualitative work is often identified with idealism while quantitative work is identified with realism. However, most qualitative researchers accept that there is an objective, material world, as do realists, but question our ability to know this directly. In the social sciences, what people perceive or believe is the basis of their actions rather than what an impartial observer might think was actually true.
Qualitative versus quantitative methods
The goal of all research in HTA should be to establish knowledge about which we can be reasonably confident, and to provide findings that are relevant to policy makers and practitioners. Therefore, decisions about whether qualitative or quantitative methods (or a combination of both) are most appropriate to a particular research problem should be made on the basis of which approach is likely to answer the question most effectively and efficiently.
Qualitative methods are useful in the exploratory stages of a research project, where they will often help to clarify or even set the research question, aid conceptualisation and generate hypotheses for later research. Qualitative methods may also be used to interpret, qualify or illuminate the findings of quantitative research and to test hypotheses.
Qualitative research is particularly useful to policy makers and planners by providing descriptive information and understanding of the context in which policies will be implemented.
Sampling and generalising
In sampling decisions in qualitative research, pragmatic considerations should be integrated with sampling in a systematic way just as in quantitative research; opportunistic sampling should be avoided if possible. The emergent nature of qualitative research means that sampling decisions need to be made throughout the study; such decisions should again be systematic and principled.
Where the aim is to build or develop theory, subjects should be selected accordingly; such theoretical sampling makes use of existing theory to make predictions, and then seeks subjects who allow the researcher to test the robustness of such predictions under different conditions.
Methods of qualitative research
Participant observation
Participant observation can be used to study the impact of technologies upon the routine functioning of the setting in which they are to be implemented. Participant observation may also be used to review health technologies currently in practice, and has the potential for uncovering the process through which professional inputs are transformed into patient/ client outcomes thereby identifying opportunities for modifying current practice to improve outcomes.
Interviews
Qualitative interview techniques are used, particularly in exploratory research, to study
the range and complexity of ideas and definitions employed by individuals and groups
involved in the implementation of health technologies. Both qualitative and quantitative
interviewing share the same fundamental problem, however, in that they rely upon
interviewees' reports and such reports are necessarily constrained by the context in which
they are collected.
Written records
The analysis of written records has an important contribution to make to our understanding
of the processes and consequences associated with new technologies. In addition, documents
such as health diaries may provide important data on the implementation of health
technologies.
Conversation analysis
The techniques of conversation analysis can provide detailed data on the impact of new
technologies upon healthcare settings, the organisation of professional work and the
interactions between health professionals and patients.
Research ethics
The same ethical principles apply to qualitative and quantitative research in HTA. The
mechanical application of ethical codes developed in the context of biomedicine may be
unduly constraining in qualitative research and may distract from those ethical risks
which are specific to qualitative research. Covert research will rarely, if ever, be
justified in HTA. Such research is likely to be a betrayal of trust and a gross invasion
of privacy.
Assessment of qualitative research
The same assessment criteria of validity and relevance are appropriate for both qualitative and quantitative research in HTA.
Relevance
The relevance of HTA research is related to its potential generalisability to groups or
settings beyond those studied. Given that most qualitative research is based on a single
case or only a small number of subjects, the generalisability of qualitative research is
achieved by the generation of theoretical statements, which may guide policy makers but
remain to be tested through application in other contexts.
Data handling
HTA commissioners should look for evidence that applicants intend to use systematic
methods for coding and handling their qualitative data and that methods proposed for
analysing such data are appropriate to the research objective.
Computerised analysis packages for qualitative data offer an efficient way of handling qualitative data sets and may improve the rigour of the analysis by facilitating searches for falsifying evidence. However, such programs should be used only as a means of facilitating the analysis process rather than carrying out the analysis, which depends upon the theoretical sensitivity of the analyst.
Judgements about the validity of research depend upon being able to form a judgement of the research process. Researchers therefore need to provide a detailed record of their methods. Given the non-standardised nature of qualitative research, such records are likely to be more elaborate than in reports of quantitative research.
The trustworthiness of data analyses is enhanced where researchers can demonstrate that they have considered alternative plausible explanations for their data. The validity of research findings is enhanced where the researchers increase our understanding of all members in a setting and do not present one-sided accounts. Confidence in the validity of findings is increased where there is evidence of researcher sensitivity to the ways in which the data have been shaped by the researcher's presence.
While the practices of respondent validation and triangulation may increase the comprehensiveness of a study, neither can be treated as tests of the validity findings.
Conclusions
There are strengths and limitations to qualitative approaches as there are to quantitative methods. However, where qualitative research is conducted properly and data analysed thoroughly, this approach can provide valuable information on the implementation and impact of health technologies on both health professionals and patients.
Publication
Murphy E, Dingwall R, Greatbatch D, Parker S, Watson P. Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assessment 1998; 2(16).
NHS R&D HTA Programme
The overall aim of the NHS R&D Health Technology Assessment (HTA)
programme is to ensure that high quality research information on the costs, effectiveness
and broader impact of health technologies is produced in the most efficient way for those
who use, manage and work in the NHS. Research is undertaken in those areas where the
evidence will lead to the greatest benefits to patients, either through improved patient
outcomes or the most efficient use of NHS resources.
The Standing Group on Health Technology advises on national priorities
for health technology assessment. Six advisory panels assist the Standing Group in
identifying and prioritising projects. These priorities are then considered by the HTA
Commissioning Board supported by the National Coordinating Centre for HTA.
This report is one of a series covering acute care, diagnostics and
imaging, methodology, pharmaceuticals, population screening, and primary and community
care. It was identified as a priority by the Methodology Panel and funded as project
number 93/44/04.
The views expressed in this publication are those of the authors and
not necessarily those of the Standing Group, the Commissioning Board, the Panel members or
the Department of Health. The editors wish to emphasize that funding and publication of
this research by the NHS should not be taken as implicit support for the recommendations
for policy contained herein. In particular, policy options in the area of screening will,
in England, be considered by the National Screening Committee. This Committee, chaired by
the Chief Medical Officer, will take into account the views expressed here, further
available evidence and other relevant considerations.
Reviews in Health Technology Assessment are termed 'systematic' when
the account of the search, appraisal and synthesis methods (to minimise biases and random
errors) would, in theory, permit the replication of the review by others.
Series Editors:
Andrew Stevens, Ruairidh Milne, Ken Stein
Assistant Editor:
Melanie Corris
©1998 Crown Copyright