Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

The quantity and quality of complementary and alternative medicine clinical practice guidelines on herbal medicines, acupuncture and spinal manipulation: systematic review and assessment using AGREE II

BMC Complementary and Alternative MedicineBMC series – open, inclusive and trusted201616:425

https://doi.org/10.1186/s12906-016-1410-8

Received: 6 July 2016

Accepted: 20 October 2016

Published: 29 October 2016

Abstract

Background

Complementary and alternative medicine (CAM) use is often not disclosed by patients, and can be unfamiliar to health care professionals. This may lead to underuse of beneficial CAM therapies, and overuse of other CAM therapies with little proven benefit or known contraindications. No prior research has thoroughly evaluated the credibility of knowledge-based resources. The purpose of this research was to assess the quantity and quality of CAM guidelines.

Methods

A systematic review was conducted to identify CAM guidelines. MEDLINE, EMBASE and CINAHL were searched in January 2016 from 2003 to 2015. The National Guideline Clearinghouse, National Center for Complementary and Integrative Health web site, and two CAM journals were also searched. Eligible guidelines published in English language by non-profit agencies on herbal medicine, acupuncture, or spinal manipulation for adults with any condition were assessed with the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument.

Results

From 3,126 unique search results, 17 guidelines (two herbal medicine, three acupuncture, four spinal manipulation, eight mixed CAM therapies) published in 2003 or later and relevant to several clinical conditions were eligible. Scaled domain percentages from highest to lowest were clarity of presentation (85.3 %), scope and purpose (83.3 %), rigour of development (61.2 %), editorial independence (60.1 %), stakeholder involvement (52.0 %) and applicability (20.7 %). Quality varied within and across guidelines. None of the 17 guidelines were recommended by both appraisers; 14 were recommended as Yes or Yes with modifications.

Conclusions

Guidelines that scored well could be used by patients and health care professionals as the basis for discussion about the use of these CAM therapies. In future updates, guidelines that achieved variable or lower scores could be improved according to specifications in the AGREE II instrument, and with insight from a large number of resources that are available to support guideline development and implementation. Future research should identify CAM therapies other than those reviewed here for which guidelines are available. Research is also needed on the safety and effectiveness of CAM therapies.

Keywords

Complementary and alternative medicine Integrative medicine Systematic review AGREE II Clinical practice guideline

Background

It is currently estimated that more than 70 % of North Americans have tried at least one form of complementary and alternative medicine (CAM), [13] collectively spending billions of dollars annually on these therapies [4, 5]. CAM has been defined as “a group of diverse medical and health care interventions, practices, products or disciplines that are not generally considered part of conventional medicine” [6]. The National Center for Complementary and Integrative Health (NCCIH) further defines a non-mainstream practice used together with conventional medicine as “complementary”, a non-mainstream practice used in place of conventional medicine as “alternative”, and the coordinated delivery or use of conventional and complementary approaches as “integrative” [6]. This study henceforth refers to therapies that fall into all of these categories as CAM.

The past several decades have seen a sharp increase in research on CAM given the strong patient-driven market [7]. Examples of well-studied CAM therapies that show potential benefit include chiropractic spinal manipulation for low back pain and headaches [812], and acupuncture for different types of pain [1318]. Recognizing such benefits, academic institutions are increasingly incorporating CAM into medical education, research and practice [11]. However, a variety of factors appear to influence whether and how CAM is used. Patients may not discuss their use of CAM with health care professionals out of fear of being judged or not seeing this as important to disclose, potentially leading to contraindications with other treatment [1922]. Many health care professionals were not exposed to CAM in their medical training [23], are unfamiliar with CAM therapies, and find it challenging to discuss use or disuse of CAM with their patients [24, 25]. This is exacerbated by the fact that CAM is comprised of many different and unrelated types of therapies and schools of thought about their use [26]; and the reliability of evidence about safety and effectiveness varies between CAM therapies [2729]. Given all of these factors, concerns have been raised about legal and ethical issues pertaining to the recommendations that health care professionals offer their patients about using or not using CAM therapies [24, 30]. Hence, patients and health care professionals may benefit from credible, knowledge-based resources upon which to base discussions and decisions about use of CAM.

Health care professionals often rely on evidence-informed clinical practice guidelines to understand whether use of a given therapy is recommended, and as a basis for informed and shared decision-making with patients about associated risks and benefits [31]. Research on a variety of clinical topics has identified that overuse, underuse or misuse of therapies may be associated with guidelines that are of poor quality [32], and the quality of guidelines has been proven to vary considerably [33]. Few studies have examined CAM guidelines. Content analysis of 10 guidelines on cardiovascular disease and type II diabetes revealed that CAM-relevant information was brief, in some cases unclear, inconclusive and lacking in direction for health care professionals [34]. Analysis of 65 National Institute for Health and Clinical Excellence guidelines available in 2009 found that, among 17 guidelines that mentioned CAM, it was not clinically relevant to most; in 14 of 48 guidelines that did not mention CAM, available evidence on the safety and effectiveness of relevant CAM therapies had not been included [35]. Therefore, no research has thoroughly evaluated the credibility of CAM guidelines. An understanding of the nature of CAM guidelines available to support informed and shared decision-making among patients and providers would help to identify whether such resources are absent and thus needed, or how they could be improved, thereby guiding future guideline development and associated research. The purpose of this study was to assess the quantity and quality of CAM guidelines.

Methods

Approach

A systematic review was conducted to identify CAM guidelines using standard methods [36] and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria [37]. A protocol was not registered. Eligible guidelines were assessed with the widely used and validated Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument [38]. AGREE II is a tool that assesses the methodological rigour and transparency in which a guideline is developed, and is the international “gold standard” for the assessment of guidelines. Detailed information is available on the AGREE web site [www.agreetrust.org]. It consists of 23 items grouped in six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence.

Eligibility criteria

Eligibility criteria for CAM guidelines were based on the Population, Intervention, Comparison and Outcomes framework. Eligible populations were adults aged 19 years and older with any diseases or conditions. With respect to interventions, guidelines were more likely to have been published on CAM interventions for which evidence has accumulated. We referred to a bibliometric and content analysis of CAM trials in the Cochrane Library by Wieland et al. [39] which found that the CAM therapies most commonly evaluated in trials included herbal supplements (non-vitamin, non-mineral dietary supplements or Chinese herbal medicine), acupuncture, and chiropractic or osteopathic manipulation [39]. For this study, guidelines were eligible if they specifically focused on any of these CAM therapies (category 1 – CAM-specific), or were general CAM guidelines that included at least one recommendation (for or against) at least of these CAM therapies (category 2 – CAM-general). We excluded general guidelines, which includes many hundreds and perhaps thousands, as it would have been challenging to search for and screen them for potential mention of CAM. Comparisons pertained to the assessed quality of CAM guidelines. Outcomes were AGREE II scores which reflect guideline content and format. The following conditions were also applied to define eligible guidelines: developed by non-profit organizations including academic institutions, government agencies, disease-specific foundations, or professional associations or societies; published in 2003 or later, which corresponds to the publication of AGREE II which provides developers with criteria for developing high-quality guidelines; English language; and either publicly available or could be ordered through our library system. Publications in the form of consensus statements, protocols, abstracts, conference proceedings, letters or editorials; based on primary studies that evaluated CAM therapies; or focused on CAM curriculum, education, training, research, professional certification or performance were not eligible.

Searching and screening

MEDLINE, EMBASE and CINAHL were searched on January 28, 2016 from 2003 to 2015 inclusive. The search strategy (Additional file 1) included Medical Subject Headings and keywords that reflect terms commonly used in the literature to refer to CAM [7]. We also searched the National Guideline Clearinghouse, a publicly available repository of guidelines [http://www.guideline.gov/] using keyword searches restricted based on the eligibility criteria including “acupressure”, “acupuncture”, “Chinese medicine”, “chiropractic”, “chiropractor”, “herbal medicine”, “herbal supplement”, “herbal therapy”, “osteopath”, “phytotherapy”, “plant extract” and “spinal manipulation”. Next, we searched the NCCIH web site which contained a single list of CAM guidelines [https://nccih.nih.gov/health/providers/clinicalpractice.htm]; and the tables of contents of two CAM journals with the highest impact factors: BMC Complementary and Alternative Medicine [https://bmccomplementalternmed.biomedcentral.com/] and the Journal of Complementary and Alternative Medicine [http://www.liebertpub.com/overview/journal-of-alternative-and-complementary-medicine-the/26/] from January 2011 and December 2015. All three authors independently screened the titles and abstracts recovered from MEDLINE to standardize screening by discussing and resolving selection differences. Following this, JYN and LL screened titles and abstracts from all other sources. JYN and ARG screened full-text items to confirm eligibility.

Data extraction and analysis

The following data were extracted from each guideline and summarized: date of publication, country of first author; type of organization that published the guideline (academic institutions, government agencies, disease-specific foundations, or professional associations or societies); topic category 1 (CAM-specific) or category 2 (CAM-general); and guideline topic including type of CAM therapy and disease or condition. Most data were available in the guideline; to assess applicability, the web site of each developer was browsed and searched for any associated knowledge-based resources in support of implementation.

Guideline quality assessment

The extraction and analysis of data from eligible guidelines followed standardized methods for applying the AGREE II instrument [38]. To do this we used the instructional manual provided by AGREE for this purpose. This is a 60-page document that first describes the AGREE instrument, provides instructions on how to apply the instrument then, for each domain, provides detailed guidance on where to look in the guideline for relevant content to judge that domain and how to rate each item in that domain. First a pilot test of the AGREE II instrument was conducted with two guidelines during which all three authors independently assessed both guidelines with the AGREE II instrument. Discrepancies were discussed and resolved. JYN and LL then independently assessed all eligible guidelines for 23 items across six domains using a seven-point Likert scale from strongly disagree (1) to strongly agree (7) that the item is met; rated the overall quality of each guideline (1 to 7); and used that information to recommend for or against use of each guideline. ARG resolved differences. Average appraisal scores were calculated by taking the average rating for all 23 items of a single appraiser of a single guideline, followed by taking the average of this value for both appraisers. Average overall assessments were calculated as the average of both appraisers’ “overall guideline assessment” scores for each guideline. Scaled domain percentages were generated for inter-domain comparison, and were calculated by adding both appraisers’ ratings of items within each domain, and scaling by maximum and minimum possible domain scores, before converting this into a percentage. Average appraisal scores, average overall assessments and scaled domain percentages for each guideline was tabulated for comparison.

Results

Search results (Fig. 1)

Searches retrieved 3,350 items, 3,126 were unique, and 3,095 titles and abstracts were eliminated, leaving 31 full-text guidelines that were considered. Of those, 14 were not eligible, primarily because they were not focused on CAM (7), they could not be retrieved (3), or did not meet other eligibility criteria (4), leaving 17 guidelines eligible for review.
Fig. 1

PRISMA diagram

Guideline characteristics (Table 1)

Eligible guidelines were published in 2003 or later in Canada, the United States, United Kingdom, China, and Australia [4056]. The guidelines were funded and/or developed by professional associations or societies (13), academic (3), and an international agency (1). Nine guidelines were CAM-specific (2 herbal medicine, three acupuncture, four spinal manipulation) and 8 were CAM-general. Clinical topics included anorexia nervosa, breast cancer, cancer (general) diabetes, headache, herpes zoster, low back pain, lung cancer, major depressive disorder, migraine, multiple sclerosis, neck pain, and Parkinson’s disease.
Table 1

Characteristics of eligible guidelines

Guideline

Country (First Author)

Developer

CAM category

Guideline topic

Fogarty 2015 [40]

Australia

Unclear

Acupuncture

Acupuncture for Anorexia Nervosa

Bryans 2014 [41]

Canada

Canadian Chiropractic Association

Spinal manipulation

Chiropractic Treatment for Neck Pain

Greenlee 2014 [42]

United States

Society for Integrative Oncology

General CAM

Integrative Therapies as Supportive Care in Breast Cancer Patients

Yadav 2014 [43]

United States

American Academy of Neurology

General CAM

Complementary and Alternative Medicine in Multiple Sclerosis

Deng 2013 [44]

United States

American College of Chest Physicians

General CAM

Complementary Therapies and Integrative Medicine in Lung Cancer

Liu 2013 [45]

China

Unclear; Sponsored by World Health Organization

Acupuncture

Acupuncture for Herpes Zoster

Nahas 2013 [46]

Canada

Canadian Diabetes Association

Herbals

Natural Health Products for Diabetes

Holland 2012 [47]

United States

American Academy of Neurology

General CAM

NSAIDs and Complementary Treatments for Episodic Migraine Prevention

Bryans 2011 [48]

Canada

Canadian Chiropractic Association

Spinal manipulation

Chiropractic Treatment of Headache

Seffinger 2010 [49]

United States

American Osteopathic Association

Spinal manipulation

Osteopathic Manipulative Treatment for Low Back Pain

Deng 2009 [50]

United States

Society for Integrative Oncology

General CAM

Complementary Therapies and Botanicals for Integrative Oncology

Ravindran 2009 [51]

Canada

Canadian Psychiatric Association, Canadian Network for Mood and Anxiety Treatments

General CAM

Complementary and Alternative Medicine for the Management of Major Depressive Disorder

Filshie 2006 [52]

United Kingdom

Unclear

Acupuncture

Providing Acupuncture for Cancer Patients

Suchowersky 2006 [53]

Canada

American Academy of Neurology

General CAM

Neuroprotective Strategies and Alternative Therapies for Parkinson Disease

Anderson-Peacock 2005 [54]

Canada

Canadian Chiropractic Association, Canadian Federation of Chiropractic Regulatory Boards

Spinal manipulation

Chiropractic Treatment for Neck Pain

Werneke 2005 [55]

United Kingdom

Unclear

General CAM

Complementary Therapies for Cancer

Mechanick 2003 [56]

United States

American Association of Clinical Endocrinologists

Herbals

Clinical Use of Dietary Supplements and Nutraceuticals

Table 2

Overall recommendations for use of appraised guidelines

Guideline

Appraiser 1

Appraiser 2

Fogarty 2015 [40]

Yes with Modifications

Yes with Modifications

Bryans 2014 [41]

Yes with Modifications

Yes with Modifications

Greenlee 2014 [42]

Yes with Modifications

Yes with Modifications

Yadav 2014 [43]

Yes with Modifications

Yes with Modifications

Deng 2013 [44]

No

Yes with Modifications

Liu 2013 [45]

No

Yes with Modifications

Nahas 2013 [46]

Yes with Modifications

Yes

Holland 2012 [47]

Yes with Modifications

Yes with Modifications

Bryans 2011 [48]

Yes with Modifications

Yes with Modifications

Seffinger 2010 [49]

Yes with Modifications

Yes with Modifications

Deng 2009 [50]

No

Yes with Modifications

Ravindran 2009 [51]

Yes with Modifications

Yes with Modifications

Filshie 2006 [52]

No

No

Suchowersky 2006 [53]

Yes with Modifications

Yes with Modifications

Anderson-Peacock 2005 [54]

Yes with Modifications

Yes with Modifications

Werneke 2005 [55]

No

No

Mechanick 2003 [56]

Yes with Modifications

Yes with Modifications

Table 3

Scaled domain percentages for appraisers of each guideline

Guideline

Domain score (%)

 

Scope and purpose

Stakeholder involvement

Rigour of development

Clarity of presentation

Applicability

Editorial Independence

Fogarty 2015 [40]

94.4

47.2

65.6

75.0

94.4

47.2

Bryans 2014 [41]

88.9

47.2

75.0

80.6

88.9

47.2

Greenlee 2014 [42]

100.0

72.2

80.2

97.2

100.0

72.2

Yadav 2014 [43]

97.2

47.2

77.1

77.8

97.2

47.2

Deng 2013 [44]

83.3

58.3

60.4

91.7

83.3

58.3

Liu 2013 [45]

52.8

11.1

32.3

88.9

52.8

11.1

Nahas 2013 [46]

75.0

86.1

92.7

91.7

75.0

86.1

Holland 2012 [47]

88.9

30.6

57.3

86.1

88.9

30.6

Bryans 2011 [48]

69.4

41.7

74.0

80.6

69.4

41.7

Seffinger 2010 [49]

97.2

66.7

69.8

91.7

97.2

66.7

Deng 2009 [50]

58.3

69.4

51.0

97.2

58.3

69.4

Ravindran 2009 [51]

94.4

30.6

60.4

88.9

94.4

30.6

Filshie 2006 [52]

83.3

38.9

14.6

72.2

83.3

38.9

Suchowersky 2006 [53]

94.4

72.2

66.7

80.6

94.4

72.2

Anderson-Peacock 2005 [54]

97.2

47.2

77.1

77.8

97.2

47.2

Werneke 2005 [55]

69.4

50.0

27.1

69.4

69.4

50.0

Mechanick 2003 [56]

77.8

72.2

71.9

94.4

77.8

72.2

Average appraisal scores, average overall assessments and recommendations regarding use of guidelines

Average appraisal scores, average overall assessments, and recommendation regarding use for each guideline are shown in Additional file 2. The average appraisal scores for each of the 17 guidelines ranged from 3.3 to 5.5 on the seven-point Likert scale (where seven equals strongly agree that the item is met); 14 guidelines achieved or exceeded an average appraisal score of 4.0, and seven guidelines achieved or exceeded an average appraisal score of 5.0. Average overall assessments for the 17 guidelines ranged between 3.0 (lowest) and 5.5 (highest), including 14 guidelines equalling or exceeding a score of 4.0, and 7 guidelines equalling or exceeding a score of 5.0.

Overall recommendations (Table 2)

None of the 17 guidelines were recommended by both appraisers. Appraisers agreed in their overall recommendation for 13 of 17 guidelines including 2 No [52, 55], and 11 Yes with modifications [4043, 4749, 51, 53, 54, 56]. Of the remaining four guidelines, three were rated by the two appraisers as No and Yes with modifications [44, 45, 50], while 1 guideline was rated at Yes and Yes with modifications [46].

Scaled domain percentage quality assessment (Table 3)

With regards to scaled domain percentages, scope and purpose scores were 52.8 to 100.0 %, stakeholder involvement scores were 11.1 to 86.1 %, rigor-of-development scores were 14.6 to 92.7 %, clarity-of-presentation scores ranged from 69.4 to 97.2 %, applicability scores were 0.00 to 60.42 %, and editorial independence scores ranged from 0.0 to 95.8 %.

Scope and purpose

The overall objectives and health questions were generally well-defined in all but one guideline [45]. Authors provided the goal of the guideline, the types of CAM they sought to assess, and the disease or condition that was the target of CAM therapy or therapies. The population to whom the guideline was meant to apply was sometimes less detailed. For example, two guidelines referred to the intended population as “patients” [46, 48].

Stakeholder involvement

Most guidelines thoroughly in detailed the characteristics of the members of the guideline development group, typically including degrees held by, and institutional affiliation of each member, in addition to some of the following: subject discipline, geographical location, and description of member’s role in the group [4144, 4649, 5356]. Some guidelines detailed the views and preferences of the target population [44, 46, 53, 56] while most did not [4043, 45, 4752, 54, 55]. Target users of the guideline were typically inconsistently defined. Some guidelines offered clear descriptions, for example, type of practitioner, specialty [40, 46, 49, 50, 52, 53, 56], while other guidelines offered few details about target users [43, 45, 47, 50, 54, 55].

Rigor of development

Systematic methods were almost always used to search for evidence and the criteria for selecting the evidence were almost always clearly described [4044, 4651, 53, 54, 56], with the exception of a few guidelines [45, 52, 55]. The strengths and limitations of the body of evidence were clearly described in all guidelines except for one [45]. The methods for formulating the recommendations varied; while some guidelines provided a fair amount of detail on how consensus was reached [40, 42, 46, 48, 49, 53, 54, 56], other guidelines provided minimal information if not none at all [40, 41, 45, 47, 51, 55]. All authors considered some health benefits, side effects, and/or risks in formulating their recommendations, with the exception of one [52]. Nearly all guidelines provided an explicit link between their recommendations and the supporting evidence with the exception of two guidelines in which this was inconsistent [49, 52]. While some guidelines explicitly stated that they were externally reviewed by experts prior to publication [41, 46, 54, 56], many did not [4244, 47, 48, 52, 55]. Some guidelines failed to mention the purpose and intent for, or the methods employed for the external review [40, 45, 4951, 53]. Most guidelines did not include a procedure for updating the guideline [42, 44, 45, 47, 48, 5056] and, among those that did, one guideline provided a detailed methodology [46].

Clarity of presentation

Generally, all guidelines offered specific and unambiguous recommendations. However, many typically lacked one or more of the following details: identification of the intent/purpose, relevant population, or caveats. All 17 guidelines scored highly in presenting different options for the management of the condition or health issue, thus contributing to this high scaled domain percentage [4056]. Key recommendations were also generally very easily identifiable.

Applicability

One guideline discussed facilitators and barriers to implementation of the recommendations [49]. Three guidelines included advice and/or tools to support implementation of the recommendations [49, 54, 56]. No guidelines addressed the resource implications of implementing the recommendations. Two guidelines provided monitoring and auditing criteria, while 14 guidelines contained little to no such information.

Editorial independence

Guidelines varied in reporting of the funding source or competing interests of the members of the guideline development panel. Several guidelines that declared a funding source did not state whether funding source influenced the content of the guideline [41, 42, 48, 53, 54, 56].

No guidelines explicitly stated that no funding supported their development. Guidelines also varied in reporting of competing interests. Several guidelines did not address competing interests [45, 51, 52, 55, 56]. While remaining guidelines did so, two did not specify how potential competing interests were identified or considered, or how they may have influenced the guideline development process or issuing of recommendations [50, 53].

Discussion

To identify credible, knowledge-based resources upon which patients and health care professionals can base discussions and decisions about use of CAM, the purpose of this research was to assess the quantity and quality of CAM guidelines. This study identified 17 guidelines (nine specific CAM therapy, eight mixed CAM therapies) published in 2003 or later that were relevant to a variety of conditions and diseases. Quality as assessed by the 23-item AGREE II instrument varied widely across guidelines overall and by domain; two guidelines scored 5.0 or higher in both average appraisal score and average overall assessment [46, 49], and three guidelines scored 3.5 or lower in both of these metrics [45, 52, 55] (1 = strongly disagree; 7 = strongly agree that criteria are met).

To our knowledge, no previous studies have assessed the quantity and quality of guidelines on CAM therapies. Thus, we believe that this is the first study to assess the credibility and nature of CAM guidelines. The findings are similar to those of guidelines on other clinical topics. In this study of CAM guidelines, the scaled domain percentages from highest to lowest were clarity of presentation (85.3 %), scope and purpose (83.3 %), rigour of development (61.2 %), editorial independence (60.1 %), stakeholder involvement (52.0 %) and applicability (20.7 %). In a previous study we found that, among 137 guidelines on a wide variety of clinical topics published from 2008 to 2013, the scaled domain percentages were ordered in similar fashion from highest (clarity of presentation 76.3 %) to lowest (applicability 43.6 %) [33]. Previous studies that examined a total of 654 guidelines published from 1980 to 2007 [57, 58], and 1,046 guidelines produced between 2005 and 2013 by 130 Australian guideline developers [59] also reported similar findings. Therefore the variable and sub-optimal quality of guidelines is not a unique phenomenon.

Notable strengths of this study included the use of a comprehensive systematic review to identify eligible CAM guidelines and the use of the validated AGREE II instrument by which to assess their quality, which is the internationally-accepted gold standard for appraising guidelines [38]. The interpretation of these findings may be limited by the fact that guidelines were independently assessed by two appraisers instead of four as recommended by the AGREE II instrument to optimize reliability. To mitigate this and standardize scoring, ARG, JYN and LL conducted an initial pilot-test during which they independently appraised the same two guidelines, then discussed the results and achieved consensus on how to apply the AGREE II instrument. Following appraisal of the 17 guidelines, ARG met with JYN and LL to discuss and resolve any uncertainties without unduly modifying legitimate discrepancies. This review does not address all CAM therapies; three therapies were chosen (herbal medicine, acupuncture, chiropractic or osteopathic manipulation) because they were identified as having the largest evidence base, and were therefore considered more likely to be the subject of guidelines [39]. We may not have identified all guidelines that included these three types of CAM therapy because, to establish a feasible scope, we did not search for guidelines on specific clinical topics and then peruse them for CAM-related content, and we did not search all CAM journals or the Guidelines International Network guideline library. We included CAM topics for which there was likely to be available evidence such as guidelines. Many patients use CAM lacking supporting evidence, therefore, it may be useful to examine guidelines on a broader range of CAM topics to evaluate the basis for recommending those therapies.

By describing the quantity and quality of CAM guidelines, this study revealed that few CAM guidelines are available to support informed and shared decision-making among patients and health care professionals. This likely reflects the lack of research on CAM therapies. Others have identified numerous factors that challenge CAM research including negative attitudes about CAM therapies [6065] and a lack of targeted funding [6669]. However, this is expected to change given that CAM therapies continue to be used by more than 40 % of the population in some regions of the world [70, 71]; and patients continue to use CAM despite documented risks associated with some CAM therapies [22, 7075]. As research emerges, so too will guidelines that focus on CAM therapies [10].

This study also revealed that the quality of CAM guidelines varied across domains within individual guidelines, and across different guidelines. This finding is relevant to those who will produce CAM guidelines in the future, and to developers of existing CAM guidelines that, when updated, could be improved. Apart from the AGREE II instrument, numerous principles, frameworks, criteria and checklists are available to help guideline developers, including CAM guideline developers, to generate the highest-quality products [7681].

Conclusions

This study identified 17 guidelines published since 2003 on CAM therapies including herbal medicines, acupuncture, and chiropractic or osteopathic manipulation. Appraisal of these guidelines with the AGREE II instrument revealed that quality varied within and across guidelines. Some of these guidelines that achieved higher AGREE II scores and favourable overall recommendations could be used by patients and health care professionals as the basis for discussion about the use of these CAM therapies. In future updates, guidelines that achieved variable or lower scaled domain percentage and overall recommendations could be improved according to specifications in the AGREE II instrument, and with insight from a large number of resources that are available to support guideline development and implementation [7580]. However, the fact that few CAM guidelines are available to support informed and shared decision-making between patients and health care professionals may continue to foster underuse of beneficial CAM therapies, and overuse or contraindicated use of other CAM for which there is no proven benefit or potential associated risks. This finding justifies the need for greater research on the safety and effectiveness of CAM therapies. Future research should also identify CAM therapies other than those reviewed here which are supported by sufficient evidence to serve as the basis for guideline development.

Abbreviations

AGREE II: 

Appraisal of guidelines for research & evaluation II

BMC CAM: 

BioMedCentral complementary and alternative medicine

CAM: 

Complementary and alternative medicine

JACM: 

Journal of complementary and alternative medicine

NCCIH: 

National Center for Complementary and Integrative Health

NGC: 

National Guideline Clearinghouse

PICO: 

Patients, intervention, comparison and outcomes

PRISMA: 

Preferred reporting items for systematic reviews and meta-analyses

Declarations

Acknowledgements

Not applicable.

Funding

Not applicable.

Availability of data and materials

All relevant data are included in this manuscript.

Authors’ contributions

JYN: made contributions to the design of the study, collected and analysed data, drafted the manuscript, and gave final approval of the version to be published. LL: contributed to study design and planning, assisted with the collection and analysis of data, and gave final approval of the version to be published. ARG: made substantial contributions to the design of the study, the collection of data as well as interpretation and analysis of the data, revised the manuscript critically, and gave final approval of the version to be published.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

All authors consent to this manuscript’s publication.

Ethics approval and consent to participate

This study involved a systematic review of peer-reviewed literature only; it did not require ethics approval or consent to participate.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Toronto General Hospital Research Institute, University Health Network

References

  1. Public Health Agency of Canada. Complementary and alternative health. http://www.phac-aspc.gc.ca/chn-rcs/cah-acps-eng.php. (2008). Accessed 28 Jan 2016.
  2. Barnes PM, Powell-Griner E, McFann K, Nahin RL. Complementary and alternative medicine use among adults: United States, 2002. Semin Integr Med. 2004;2(2):54–71.View ArticleGoogle Scholar
  3. Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, Kessler RC. Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey. JAMA. 1998;280(18):1569–75.View ArticlePubMedGoogle Scholar
  4. Nahin RL, Barnes PM, Stussman BJ, Bloom B. Costs of complementary and alternative medicine (CAM) and frequency of visits to CAM practitioners: US 2007. Hyattsville: Diane Publishing; 2010.Google Scholar
  5. Esmail N. Complementary and alternative medicine in Canada: trends in use and public attitudes, 1997-2006. Vancouver: Fraser Institute; 2007.Google Scholar
  6. National Institutes of Health, National Centre for Complementary and Integrative Health (NCCIH). Complementary, alternative, or integrative health: What’s in a name? https://nccih.nih.gov/health/integrative-health (2016). Accessed 28 Jan 2016.
  7. Ng JY, Boon HS, Thompson AK, Whitehead CR. Making sense of “alternative”, “complementary”, “unconventional” and “integrative” medicine: exploring the terms and meanings through a textual analysis. BMC Complement Altern Med. 2016;16(134):1.Google Scholar
  8. Chaibi A, Tuchin PJ, Russell MB. Manual therapies for migraine: a systematic review. J Headache Pain. 2011;12(2):127–33.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Walker BF, French SD, Grant W, Green S. A Cochrane review of combined chiropractic interventions for low-back pain. Spine J. 2011;36(3):230–42.View ArticleGoogle Scholar
  10. Bronfort G, Haas M, Evans R, Leininger B, Triano J. Effectiveness of manual therapies: the UK evidence report. Chiropr Man Therap. 2010;18(1):3.Google Scholar
  11. Bronfort G, Assendelft WJ, Evans R, Haas M, Bouter L. Efficacy of spinal manipulation for chronic headache: a systematic review. J Manipulative Physiol Ther. 2001;24(7):457–66.View ArticlePubMedGoogle Scholar
  12. Cherkin DC, Mootz RD. Chiropractic in the United States: training, practice, and research. Rockville: AHCPR Publication; 1997.Google Scholar
  13. Liu L, Skinner M, McDonough S, Mabire L, Baxter GD. Acupuncture for low back pain: an overview of systematic reviews. Evid Based Complement Alternat Med. 2015;2015:328196.PubMedPubMed CentralGoogle Scholar
  14. Manyanga T, Froese M, Zarychanski R, Abou-Setta A, Friesen C, Tennenhouse M, Shay BL. Pain management with acupuncture in osteoarthritis: a systematic review and meta-analysis. BMC Complement Altern Med. 2014;14(1):312.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, Sherman KJ, Witt CM, Linde K. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012;172(19):1444–53.View ArticlePubMedPubMed CentralGoogle Scholar
  16. Cherkin DC, Sherman KJ, Avins AL, Erro JH, Ichikawa L, Barlow WE, Delaney K, Hawkes R, Hamilton L, Pressman A, Khalsa PS. A randomized trial comparing acupuncture, simulated acupuncture, and usual care for chronic low back pain. Arch Intern Med. 2009;169(9):858–66.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Yuan J, Purepong N, Kerr DP, Park J, Bradbury I, McDonough S. Effectiveness of acupuncture for low back pain: a systematic review. Spine J. 2008;33(23):E887–900.View ArticleGoogle Scholar
  18. Witt CM, Jena S, Brinkhaus B, Liecker B, Wegscheider K, Willich SN. Acupuncture for patients with chronic neck pain. Pain. 2006;125(1):98–106.View ArticlePubMedGoogle Scholar
  19. Ventola CL. Current issues regarding complementary and alternative medicine (CAM) in the United States: Part 1: The widespread use of CAM and the need for better-informed health care professionals to provide patient counseling. Pharmacol Ther. 2010;35(8):461.Google Scholar
  20. Barraco D, Valencia G, Riba AL, Nareddy S, Draus CB, Schwartz SM. Complementary and alternative medicine (CAM) use patterns and disclosure to physicians in acute coronary syndromes patients. Complement Ther Med. 2005;13(1):34–40.View ArticlePubMedGoogle Scholar
  21. Coulter ID, Willis EM. The rise and rise of complementary and alternative medicine: a sociological perspective. Med J Aust. 2004;180(11):587.PubMedGoogle Scholar
  22. Tsai HH, Lin HW, Simon Pickard A, Tsai HY, Mahady GB. Evaluation of documented drug interactions and contraindications associated with herbs and dietary supplements: A systematic literature review. Int J Clin Pract. 2012;66(11):1056–78.View ArticlePubMedGoogle Scholar
  23. Cowen VS, Cyr V. Complementary and alternative medicine in US medical schools. Adv Med Educ Pract. 2015;6:113–7.View ArticlePubMedPubMed CentralGoogle Scholar
  24. Adams KE, Cohen MH, Eisenberg D, Jonsen AR. Ethical considerations of complementary and alternative medical therapies in conventional medical settings. Ann Intern Med. 2002;137(8):660–4.View ArticlePubMedGoogle Scholar
  25. Wilkinson S, Gomella LG, Smith JA, Brawer MK, Dawson NA, Wajsman Z, Dai L, Chodak GW. Attitudes and use of complementary medicine in men with prostate cancer. J Urol. 2002;168(6):2505–9.View ArticlePubMedGoogle Scholar
  26. Fisher P, Ward A. Complementary medicine in Europe. BMJ. 1994;309(6947):107.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Pearson NJ, Chesney MA. The CAM education program of the national center for complementary and alternative medicine: an overview. Acad Med. 2007;82(10):921–6.View ArticlePubMedGoogle Scholar
  28. Kroll DJ. ASHP statement on the use of dietary supplements. Am J Health Syst Pharm. 2004;61(16):1707–11.PubMedGoogle Scholar
  29. Halcón LL, Chlan LL, Kreitzer MJ, Leonard BJ. Complementary therapies and healing practices: faculty/student beliefs and attitudes and the implications for nursing education. J Prof Nurs. 2003;19(6):387–97.View ArticlePubMedGoogle Scholar
  30. Weir M. Legal issues for medical doctors in the provision of complementary and alternative medicine. Med Law. 2007;26(4):817–28.PubMedGoogle Scholar
  31. Shekelle P, Woolf S, Grimshaw JM, Schunemann H, Eccles MP. Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012;7:62.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Mickan S, Burls A, Glasziou P. Patterns of “leakage” in the utilization of clinical guidelines: a systematic review. Postgrad Med J. 2011;87(1032):670–9.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Gagliardi AR, Brouwers MC. Do guidelines offer implementation advice to target users? A systematic review of guideline applicability. BMJ Open. 2015;5(2):e007047.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Team V, Canaway R, Manderson L. Integration of complementary and alternative medicine information and advice in chronic disease management guidelines. Aust J Prim Health. 2011;17(2):142–9.View ArticlePubMedGoogle Scholar
  35. Ernst E. Assessments of complementary and alternative medicine: the clinical guidelines from NICE. Int J Clin Pract. 2010;64(10):1350–8.View ArticlePubMedGoogle Scholar
  36. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. London: The Cochrane Collaboration; 2011.Google Scholar
  37. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.View ArticlePubMedGoogle Scholar
  38. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P. AGREE II: advancing guideline development, reporting and evaluation in health care. Can Med Assoc J. 2010;182(18):E839–42.View ArticleGoogle Scholar
  39. Wieland LS, Manheimer E, Sampson M, Barnabas JP, Bouter LM, Cho K, Lee MS, Li X, Liu J, Moher D, Okabe T. Bibliometric and content analysis of the Cochrane complementary medicine field specialized register of controlled trials. Syst Rev. 2013;2:51.View ArticlePubMedPubMed CentralGoogle Scholar
  40. Fogarty S, Ramjan LM. Practice guidelines for acupuncturists using acupuncture as an adjunctive treatment for anorexia nervosa. Complement Ther Med. 2015;23(1):14–22.View ArticlePubMedGoogle Scholar
  41. Greenlee H, Balneaves LG, Carlson LE, Cohen M, Deng G, Hershman D, Mumber M, Perlmutter J, Seely D, Sen A, Zick SM. Clinical practice guidelines on the use of integrative therapies as supportive care in patients treated for breast cancer. J Natl Cancer Inst Monogr. 2014;50(2014):346–58.View ArticleGoogle Scholar
  42. Bryans R, Decina P, Descarreaux M, Duranleau M, Marcoux H, Potter B, Ruegg RP, Shaw L, Watkin R, White E. Evidence-based guidelines for the chiropractic treatment of adults with neck pain. J Manipulative Physiol Ther. 2014;37(1):42–63.View ArticlePubMedGoogle Scholar
  43. Yadav V, Bever C, Bowen J, Bowling A, Weinstock-Guttman B, Cameron M, Bourdette D, Gronseth GS, Narayanaswami P. Summary of evidence-based guideline: complementary and alternative medicine in multiple sclerosis. J Neurol. 2014;82(12):1083–92.View ArticleGoogle Scholar
  44. Deng GE, Rausch SM, Jones LW, Gulati A, Kumar NB, Greenlee H, Pietanza MC, Cassileth BR. Complementary therapies and integrative medicine in lung cancer: diagnosis and management of lung cancer: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2013;143 Suppl 5:e420S–36.View ArticlePubMedGoogle Scholar
  45. Liu ZS, Peng WN, Liu BY, Wang J, Wang Y, Mao M, Deng YH, Yu JN, Liaw Y, Mu Y, Luo Y. Clinical practice guideline of acupuncture for herpes zoster. Chin J Integr Med. 2013;19(1):58–67.View ArticlePubMedGoogle Scholar
  46. Nahas R, Goguen J. Natural health products. Can J Diabetes. 2013;37 Suppl 1:S97–9.View ArticlePubMedGoogle Scholar
  47. Holland S, Silberstein SD, Freitag F, Dodick DW, Argoff C, Ashman E. Evidence-based guideline update: NSAIDs and other complementary treatments for episodic migraine prevention in adults Report of the Quality Standards Subcommittee of the American Academy of Neurology and the American Headache Society. Neurology. 2012;78(17):1346–53.View ArticlePubMedPubMed CentralGoogle Scholar
  48. Bryans R, Descarreaux M, Duranleau M, Marcoux H, Potter B, Ruegg R, Shaw L, Watkin R, White E. Evidence-based guidelines for the chiropractic treatment of adults with headache. J Manipulative Physiol Ther. 2011;34(5):274–89.View ArticlePubMedGoogle Scholar
  49. Seffinger MA, Buser BR, Licciardone JC. American Osteopathic Association guidelines for osteopathic manipulative treatment (OMT) for patients with low back pain. Clinical guideline subcommittee on low back pain. J Am Osteopath Assoc. 2010;110(11):653–66.Google Scholar
  50. Ravindran AV, Lam RW, Filteau MJ, Lespérance F, Kennedy SH, Parikh SV, Patten SB. Canadian Network for Mood and Anxiety Treatments (CANMAT) Clinical guidelines for the management of major depressive disorder in adults.: V. Complementary and alternative medicine treatments. J Affect Disord. 2009;117 Suppl 1:S54–64.View ArticlePubMedGoogle Scholar
  51. Deng GE, Frenkel M, Cohen L, Cassileth BR, Abrams DI, Capodice JL, Courneya KS, Dryden T, Hanser S, Kumar N, Labriola D. Evidence-based clinical practice guidelines for integrative oncology: complementary therapies and botanicals. J Soc Integr Oncol. 2009;7(3):85.PubMedGoogle Scholar
  52. Filshie J, Hester J. Guidelines for providing acupuncture treatment for cancer patients–a peer-reviewed sample policy document. Acupunct Med. 2006;24(4):172–82.View ArticlePubMedGoogle Scholar
  53. Suchowersky O, Gronseth G, Perlmutter J, Reich S, Zesiewicz T, Weiner WJ. Practice Parameter: Neuroprotective strategies and alternative therapies for Parkinson disease (an evidence-based review) Report of the quality standards subcommittee of the American Academy of Neurology. J Neurol. 2006;66(7):976–82.View ArticleGoogle Scholar
  54. Anderson-Peacock E, Blouin JS, Bryans R, Danis N. Chiropractic clinical practice guideline: evidence-based treatment of adult neck pain not due to whiplash. J Can Chiropr Assoc. 2005;49(3):158.PubMedPubMed CentralGoogle Scholar
  55. Werneke U. A guide to using complementary alternative medicines in cancer. Drugs. 2005;101(5):1403–11.Google Scholar
  56. Mechanick JI, Brett EM, Chausmer AB, Dickey RA, Wallach S, Bergman DA, Garber JR, Hamilton CR, Handelsman Y, Holdy KE, Kukora JS. American Association of Clinical Endocrinologists medical guidelines for the clinical use of dietary supplements and nutraceuticals. Endocr Pract. 2003;9(5):417–70.View ArticlePubMedGoogle Scholar
  57. Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J, Schunemann H. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):1–7.Google Scholar
  58. Knai C, Brusamento S, Legido-Quigley H, Saliba V, Panteli D, Turk E, Car J, McKee M, Busse R. Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. Health Policy. 2012;107(2):157–67.View ArticlePubMedGoogle Scholar
  59. National Health and Medical Research Council (NHMRC). Annual report on Australian clinical practice guidelines. Canberra: National Health and Medical Research Council; 2014.Google Scholar
  60. Weisleder P. Unethical prescriptions: alternative therapies for children with cerebral palsy. Clin Pediatr. 2010;49(1):7–11.View ArticleGoogle Scholar
  61. Singh S, Ernst E. Trick or treatment: the undeniable facts about alternative medicine. New York: WW Norton & Company; 2008.Google Scholar
  62. Chatfield K, Partington H, Duckworth J. The place of the university in the provision of CAM education. Aust J Homeopathic Med. 2012;24(1):16–20.Google Scholar
  63. Colquhoun D. Science degrees without the science. Nature. 2007;446(7134):373–4.View ArticlePubMedGoogle Scholar
  64. Offit PA. Studying complementary and alternative therapies. JAMA. 2012;307(17):1803–4.View ArticlePubMedGoogle Scholar
  65. Colquhoun D. Should NICE evaluate complementary and alternative medicines. BMJ. 2007;334:506–7.View ArticleGoogle Scholar
  66. Fischer FH, Lewith G, Witt CM, Linde K, von Ammon K, Cardini F, Falkenberg T, Fønnebø V, Johannessen H, Reiter B, Uehleke B. High prevalence but limited evidence in complementary and alternative medicine: guidelines for future research. BMC Complement Altern Med. 2014;14(1):1.View ArticleGoogle Scholar
  67. Ernst E, Cohen MH, Stone J. Ethical problems arising in evidence based complementary and alternative medicine. J Med Ethics. 2004;30(2):156–9.View ArticlePubMedPubMed CentralGoogle Scholar
  68. Ernst E. Obstacles to research in complementary and alternative medicine. Med J Aust. 2003;179(6):279–80.PubMedGoogle Scholar
  69. Nissen N, Manderson L. Researching alternative and complementary therapies: mapping the field. Med Anthropol. 2013;32(1):1–7.View ArticlePubMedGoogle Scholar
  70. Clarke TC, Black LI, Stussman BJ, Barnes PM, Nahin RL. Trends in the use of complementary health approaches among adults: United States, 2002–2012. Natl Health Stat Report. 2015;10(79):1–16.Google Scholar
  71. Harris PE, Cooper KL, Relton C, Thomas KJ. Prevalence of complementary and alternative medicine (CAM) use by the general population: a systematic review and update. Int J Clin Pract. 2012;66(10):924–39.View ArticlePubMedGoogle Scholar
  72. Werneke U, Earl J, Seydel C, Horn O, Crichton P, Fannon D. Potential health risks of complementary alternative medicines in cancer patients. Br J Cancer. 2004;90(2):408–13.View ArticlePubMedPubMed CentralGoogle Scholar
  73. Ernst E. Harmless herbs? A review of the recent literature. Am J Med. 1998;104(2):170–8.View ArticlePubMedGoogle Scholar
  74. Patel DN, Low WL, Tan LL, Tan MM, Zhang Q, Low MY, Chan CL, Koh HL. Adverse events associated with the use of complementary medicine and health supplements: An analysis of reports in the Singapore Pharmacovigilance database from 1998 to 2009. Clin Toxicol. 2012;50(6):481–9.View ArticleGoogle Scholar
  75. Angell M, Kassirer JP. Alternative medicine-the risks of untested and unregulated remedies. N Engl J Med. 1998;339(12):839–40.View ArticlePubMedGoogle Scholar
  76. Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Bossuyt P, Chang S, Muti P, Jaeschke R, Guyatt GH. GRADE: assessing the quality of evidence for diagnostic recommendations. Evid Based Med. 2008;13(6):162–3.View ArticlePubMedGoogle Scholar
  77. Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O’Connell R. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005;5(1):23.View ArticlePubMedPubMed CentralGoogle Scholar
  78. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J. 2014;186(3):E123–42.View ArticleGoogle Scholar
  79. Gagliardi AR, Brouwers MC, Bhattacharyya O. A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools. Implement Sci. 2014;9:98.View ArticlePubMedPubMed CentralGoogle Scholar
  80. Gagliardi AR, Brouwers MC, Bhattacharyya OK. The development of guideline implementation tools: a qualitative study. CMAJ Open. 2015;3(1):e127–33.View ArticlePubMedPubMed CentralGoogle Scholar
  81. Gagliardi AR, Marshall C, Huckson S, James R, Moore V. Developing a checklist for guideline implementation planning: A review and synthesis of guideline development and implementation advice. Implement Sci. 2015;10:19.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2016

Advertisement