Systematic reviews of complementary therapies - an annotated bibliography. Part 1: Acupuncture

Background Complementary therapies are widespread but controversial. We aim to provide a comprehensive collection and a summary of systematic reviews of clinical trials in three major complementary therapies (acupuncture, herbal medicine, homeopathy). This article is dealing with acupuncture. Potentially relevant reviews were searched through the register of the Cochrane Complementary Medicine Field, the Cochrane Library, Medline, and bibliographies of articles and books. To be included articles had to review prospective clinical trials of acupuncture; had to describe review methods explicitly; had to be published; and had to focus on treatment effects. Information on conditions, interventions, methods, results and conclusions was extracted using a pretested form and summarized descriptively. Results From a total of 48 potentially relevant reviews preselected in a screeening process 39 met the inclusion criteria. 22 were on various pain syndromes or rheumatic diseases. Other topics addressed by more than one review were addiction, nausea, asthma and tinnitus. Almost unanimously the reviews state that acupuncture trials include too few patients. Often included trials are heterogeneous regarding patients, interventions and outcome measures, are considered to have insufficient quality and contradictory results. Convincing evidence is available only for postoperative nausea, for which acupuncture appears to be of benefit, and smoking cessation, where acupuncture is no more effective than sham acupuncture. Conclusions A large number of systematic reviews on acupuncture exists. What is most obvious from these reviews is the need for (the funding of) well-designed, larger clinical trials.

compared to their relevance in actual care, the total number of clinical trials in many areas is considerable [4]. There is a need to know what evidence is available and which questions have not yet been addressed. Systematic reviews are considered to be the best available method to this end [5]. In recent years an increasing number of such reviews has been performed in a variety of complementary therapies. The objective of our series of reports is to provide a comprehensive collection and transparent summary of the available systematic reviews of clinical trials in three major complementary therapies (acupuncture, herbal medicine, and homeopathy). It was not our primary objective to assess efficacy as we do not consider a review of reviews of a large number of interventions an appropriate tool for this purpose. However, when summarizing the results of systematic reviews it is unavoidable to cite their conclusions on efficacy. Our results are presented in three consecutive articles. This first article deals with acupuncture.
Acupuncture is a therapy that involves the stimulation of defined points on the skin typically by inserting needles; however, related techniques such as manual (acupressure), electrical or laser stimulation of acupuncture points are also often summarised under this term [6,7]. Acupuncture is, together with the use of herbal medicines and other treatment modalities, a part of traditional Chinese medicine. In the West it is often used as a single therapy. The basic idea underlying acupuncture is that disorders related to the flow of Chi (the traditional Chinese concept translated as vital force or energy) can be prevented or treated by stimulating the relevant points on the body surface.

Methods
To be included in this overview reviews had to meet the following criteria: 1) Report reviews prospective (not necessarily controlled) clinical trials of acupuncture or related methods (such as acupressure) in humans. 2) Reports explicitly describe, at least, one of the following issues: a) methods for searching primary studies and eligibility criteria for primary studies; b) methods to assess quality aspects; c) methods to summarise the results of the primary studies. 3) Reports are published in journals, books, theses, or the internet. Reviews published before 1989 and as abstracts only were not included. 4) The primary focus of the report is on treatment effects (not diagnosis, side effects, risks, etc.). There were no language restrictions. Disease-oriented reviews including a variety of interventions were included only if they reviewed at least 4 acupuncture trials.
The primary source for identification of systematic reviews was the register of the Cochrane Complementary Medicine Field. For the compilation of this register a va-riety of databases including Medline, Embase, CISCOM, AMED and other sources have been searched. In addition, we searched 1) Medline 1989 to July 2000 using a standard strategy to identify systematic reviews [8] (see strategy list) combined with the terms acupuncture or acupressure; and 2) the Cochrane Library (last check in issue 2000, 3). Bibliographies of articles obtained and relevant textbooks were screened for further potentially relevant articles. The literature list from the Complementary Medicine Field register was screened in a first step independently by two reviewers who excluded all references for which they were sure that the papers were not systematic reviews. Abstracts of the publications identified by other means were screened by one reviewer. Full copies were obtained for all potentially relevant papers. One (in 46% of papers), two (53%) or three (1%) reviewers checked eligibility and extracted information (bibliographic details, topic, intervention, inclusion criteria, methodological issues, studies and number of patients included, results, and conclusions) from included reviews using pretested forms. For this report the included reviews were summarised in a tabular format giving basic information on the conditions, interventions, comparisons, number of studies reviewed, methodological features, results, and conclusions drawn by the reviewers (if possible in the original wording). We assessed the following methodological features: Comprehensiveness of the literature search (scored if in addition to Medline other databases and non-electronic sources were searched), whether inclusion and exclusion criteria were explicitly listed, whether the quality of primary studies was assessed using fomal methods (such as scores or checklists), whether a summary of results was provided for each included study, and whether a quantitative meta-analysis was performed.
If several review publications by the same team of reviewers with the same focus and published within a time span of 3 years were available these were considered as updates unless inclusion criteria for the two versions were clearly different.

Results
From a total of 48 potentially relevant reviews preselected through the screening process, 39 (published in 45 papers) met the inclusion criteria . Five reviews were excluded as they were not truly systematic reviews (not meeting inclusion criterion 2) [54][55][56][57][58] and 4 as they did not focus on treatment effects but on methodological issues [59][60][61][62].
As expected, many reviews addressed pain (see table 1). Three reviews (published 1989 [10,12], 1990 [11] and 2000 [9]) focused on chronic pain. The study samples (total number of trials covered in any of the reviews 72) and review methods differed considerably among the reviews but all agreed that the available evidence is inconclusive. Back and neck pain were addressed in six reviews (total number of trials covered for each back and neck pain 11). Conclusions regarding back pain were contradictory [13,[15][16][17][18][19] while the evidence regarding neck pain was considered inconclusive [13,14]. Six reviews addressed various types of headaches (total number of trials covered 25). The largest review which focused on migraine and tension-type headache [21] drew tentatively positive conclusions. The remaining reviews included fewer studies and the evidence was considered inconclusive [20,[22-25]].
The available trials suggest that acupuncture is effective in postoperative pain after dental treatment and temporomandibular dysfunction although further research is still considered necessary [26,27,53].    Systematic reviews of acupuncture and acupressure for rheumatic diseases, addiction, nausea and asthma are summarized in table 2. A variety of rheumatic conditions has been addressed in five reviews. The limited evidence available so far suggests that acupuncture may have favorable effects in fibromyalgia [28]. The majority of the available trials on osteoarthritis report improvement with both sham and true acupuncture but no significant differences between the two [29]. For a variety of other rheumatoid diseases the evidence was considered inconclusive [30,31,32].
The trials on smoking cessation show in a fairly consistent manner that acupuncture does not seem to have lasting effects over sham acupuncture [33][34][35][36][37][38][39] (total number of trials covered 22). The only identified review on alcohol and heroin addiction which met our inclusion criteria was published in 1989 [39]; several narrative re-views including new trials on the topic are available (for example [55]).
Trials on nausea are unique in acupuncture research as they focus almost completely on acupuncture or acupressure at a single point (P6). There is good evidence that both acupuncture and acupressure are effective in postoperative nausea (total number of trials covered 24) while the results are equivocal for early morning sickness [40][41][42][43] (total number of trials covered 8). Most of the trials report positive results but a recent high quality trial could not replicate the findings [63]. Several studies suggest that stimulation of P6 is also effective in treating chemotherapy-induced nausea [43].
Results on asthma are contradictory and all available trials are very small [44][45][46][47] (total number of trials covered 15).  Systematic reviews of acupuncture and acupressure for a variety of additional conditions are summarized in table 3. While the findings of the available trials on tinnitus and weight reduction do not suggest important effects [48,49,50] a review on acupuncture as an adjunct treatment in stroke rehabilitation found promising results [51].
Finally, there is one review summarizing research on acupressure for various conditions (acceptable evidence only available for nausea [52]) and another on acupuncture in various conditions related to dentistry (reporting promising evidence for temporomandibular dysfunction [53]).

Discussion
Although a considerable number of clinical trials on acupuncture is available the evidence so far is very often inconclusive. Apart from postoperative nausea (positive) and smoking cessation (negative) the reviewers apparently felt unable to make clear conclusions whether acupuncture was effective or not. This finding is somewhat frustrating. The primary problem in acupuncture studies seems to be sample size (most studies are very small). For example, median sample sizes of trials in low back pain, headache and osteoarthirits were 50 [16], 37 [21] and 31 [29], respectively. A second relevant problem is methodological quality. There was an almost uniform call for large, well-designed studies. Why are so few such studies available?
One reason is probably the lack of funding for such studies. There is little industrial interest in acupuncture, so major funding has to come mainly from public resources.
In  The value of acupuncture as an analgesic must be questioned due to problems in the trials. But the effect in temporomandibular dysfunction and facial pain seems real legend see table 1 therapies like physiotherapy) is a term for describing a group of quite heterogeneous interventions. Some providers clearly find it misleading to include techniques without needling into systematic reviews of acupuncture while those who apply acupressure or laser acupuncture often hold the opinion that the crucial issue is the stimulation of the correct point no matter by which way. Acupuncture strategies for the treatment of a specific condition can be highly variable. Some acupuncturists use similar approaches in all patients with a given Western diagnosis while others consider this as inadequate and claim that the treatment has to be "individualized." This is often misunderstood: Several reviews described a negative correlation between study quality and outcome (better studies were more often negative; for example [9,11,13]). This finding has to be interpreted with caution. Many reviews included trials with very different control groups: waiting list or no treatment, the various "placebo" techniques listed above, and a variety of active controls. If acupuncture has at least some placebo effect, attributable to the psychological effects of being administered a novel therapy, one would expect placebo-controlled trials to show smaller differences between groups than trials without placebo control. Acupuncture trials without placebo control cannot be blinded and therefore typically score lower on quality scales. Given that links between study quality and outcome are confounded by the issues of non-specific effects, and given that trials with different control groups answer different questions, analyses checking the influence of quality aspects on outcomes should only be performed within groups of trials with comparable controls.
This annotated bibliography of systematic reviews should also be interpreted with great caution. The risk of oversimplification in a systematic review is great. In a review of reviews it is extreme. We summarise the conclu-sion of a systematic review in a single phrase. Clinical decisions for treatment of individual patients should not be based on our work. For this, patients and health care professionals have to turn to the original reviews. Our aim was to provide a clear summary of what is available and where further information can be found. We tried to be as comprehensive as possible in our search but cannot exclude that we have overlooked eligible work, particularly if this was not published in a journal.
Our findings are generally in accordance with those of a panel of the National Institutes of Health reporting on acupuncture in 1998 [67]. However, the conclusions of the panel that the evidence on acupuncture is promising for a variety of conditions but not conclusive seems slightly more optimistic than the picture which emerges from the available systematic reviews.
We did not systematically search the literature on systematic reviews of side effects of acupuncture. We came across one such review which had collected case reports [68]. However, for a reliable assessment of safety large scale prospective studies or effective surveillance systems are needed in addition.
In conclusion, while in some areas only older reviews are available and some minor topics are not reviewed at all it is obvious that what is needed is new primary research and not new systematic reviews. Future trials should have larger sample sizes, rigorous methods, and should reflect principles and practice of acupuncture as applied in actual practice. Until conclusive data becomes available we must be aware that most decisions in health care regarding acupuncture are based -as in many other areas of medicine -on partial evidence about which reasonable people can disagree.

Competing interest
KL, DM, GtR, BB and AV have been involved in some of the reviews analyzed. These were extracted and assessed by other members of the team.