Effectiveness guidance documents (EGDs) provide specific recommendations about the design of prospective studies that will inform decisions by patients, clinicians and payers. The recommendations are targeted to clinical researchers conducting studies of specific types of interventions or clinical conditions. EGDs are intended to be analogous to Food and Drug Administration (FDA) guidance documents in the US, but are focused on design elements intended to support clinical and health policy decision-making. The recommended methods aim to achieve a balance between internal validity, relevance and feasibility. These documents are developed through an extensive consultative process involving a broad range of experts and stakeholders.
Comparative Effectiveness Research (CER) is the generation and synthesis of evidence that compares the benefits and harms of different treatment options to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels. In addition, the Institute of Medicine (IOM) used six characteristics to describe CER: 1) Informing Informs a specific clinical decision from the patient perspective or a health policy decision from the population perspective, 2) compares at least two alternative interventions, each with the potential to be “best practice”, 3) describing results at the population and subgroup levels, 4) measures outcomes—both benefits and harms—that are important to patients, 5) employs methods and data sources appropriate for the decision of interest, 6) is conducted in settings that are similar to those in which the intervention will be used in practice.
There is a strong need for additional CER for disorders that are common and costly to society and that have a great degree of variation in their treatment. Acupuncture is one of the complementary and alternative therapies used for patients suffering from chronic symptoms, such as pain. Guidelines for designing and reporting CER involving acupuncture are needed since acupuncture treatment varies among different types of providers, states and countries, and the design and results of previous studies have been highly variable.
“Effectiveness” is a measure of the extent to which an intervention, when deployed in the field in routine circumstances, does what it is intended to do for a specific population and therefore can be of high relevance to policy evaluation and the health care decisions of providers and patients. In contrast, “efficacy” refers to the extent to which a specific intervention is beneficial under ideal conditions. It is important to note that there is often no sharp distinction between efficacy and effectiveness trials. Rather, different dimensions of trial design may be distributed at varying sites across the continuum. For randomized trials, the distinction between explanatory and practical or pragmatic is also often made[8–10]. However, within this document we will use the more general terms “efficacy” and “effectiveness” to label both ends of the continuum and a trial that is more on the effectiveness side would be a pragmatic trial.
Traditional medical systems such as Chinese medicine frequently use multiple interventions, many of which are individually complex, e.g. an herbal formula or an acupuncture point combination. Such complex interventions, whose components may act additively or synergistically, are difficult to evaluate. Yet, the aspect of CER that addresses effectiveness in a more everyday practice setting suggests that studies can be designed to compare real-world whole systems of health care. The Medical Research Council guidance in the UK provides the following advice for developing and evaluating complex interventions:
A good theoretical understanding is necessary to understand how the intervention causes change, so that weak links in the causal chain can be identified and strengthened.
Lack of effect may reflect implementation failure rather than genuine ineffectiveness; a thorough process evaluation is needed to identify implementation problems.
Variability in individual level outcomes may reflect higher level processes; sample sizes may need to be larger to take account of the extra variability. Cluster randomized designs may also be considered.
A single primary outcome may not make best use of the data; a range of measures will be needed and unintended consequences accounted for where possible.
Ensuring strict standardization may be inappropriate; if a specified degree of adaptation to local settings is allowed for in the protocol, the intervention may work better.
The aim of the present EGD is to provide comprehensive, consensus-based guidance for future acupuncture studies by taking existing guidelines into account. This document should be seen as part of a larger research framework that would include translational research strategies to enhance the likelihood that mechanistic studies, clinical studies and clinical practice will each inform and be informed by one another.