- Research article
- Open Access
- Open Peer Review
A survey of evidence users about the information need of acupuncture clinical evidence
BMC Complementary and Alternative Medicinevolume 16, Article number: 455 (2016)
The PRISMA statement was rarely used in the field of acupuncture, possibly because of knowledge gaps and the lack of items tailored for characteristics of acupuncture. And with an increasing number of systematic reviews in acupuncture, it is necessary to develop an extension of PRISMA for acupuncture. And this study was the first step of our project, of which the aim was to investigate the need for information of clinical evidence on acupuncture from the perspectives of evidence users.
We designed a questionnaire based on a pilot survey and a literature review of acupuncture systematic review or meta-analysis(SR/MA). Participants from five cities (Lanzhou, Chengdu, Shanghai, Nanjing and Beijing) representing the different regions of China, including clinicians, researchers and postgraduates in their second year of Master studies or higher level, were surveyed.
A total of 269 questionnaires were collected in 18 hospitals, medical universities and research agencies, and 251 (93 %) with complete data were used for analysis. The average age of respondents was 33 years (SD 8.959, range 25–58) with male 43 % and female 57 %. Most respondents had less than 5 years of working experience on acupuncture, and read only one to five articles per month. Electronic databases, search engines and academic conferences were the most common sources for obtaining information. Fifty-six percent of the respondents expressed low satisfaction of the completeness of information from the literature. The eight items proposed for acupuncture SR/MAs received all high scores, and five of the items scored higher than eight on a scale zero to ten. The differences for the scores of most items between postgraduates and non-postgraduates were not statistically significant.
The majority of the respondents were not very satisfied with the information provided in acupuncture SRs. Most of the items proposed in this questionnaire received high scores, and opinions from postgraduates and non-postgraduates tended to agree on most items. Comments from the respondents can promote future work.
Reporting guidelines promote transparent and rigorous reporting. In 1996, several experts from the United States, United Kingdom and Canada worked together and published the Consolidated Standards of Reporting Trials (CONSORT) statement in Journal of the American Medical Association (JAMA), which started the rapid development of reporting guidelines in medical research . With the evolving methodology of randomized controlled trials (RCT) and evidence-based medicine (EBM), CONSORT has been updated in 2001 and 2010 [2, 3], CONSORT 2010 being the latest version. CONSORT has been endorsed and uptaken after its release by several medical journals , and reporting guidelines have improved the quality of both the reporting and the methodology [5, 6]. Because different fields of medicine and different types of research data differ in characteristics, several extensions have been developed. The STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) was officially published in 2001, and was updated along with CONSORT in 2010 [7, 8]. Authors of acupuncture trials and systematic reviews have expressed their belief that STRICTA substantially contributes to the reporting of acupuncture interventions  and has significantly improved the reporting quality .
High-quality systematic reviews (SRs) of acupuncture form the best evidence to inform guidelines and clinical practice. Accurate and complete reporting enables readers to determine the internal and external validity of the research result, and editors and reviewers to make comprehensive and objective judgement effectively. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed for systematic reviews and meta-analyses (SR/MAs) . Although PRISMA promotes the quality of the reporting and methodology of SR/MAs, in some specific areas, it cannot meet all the needs. The extensions PRISMA-Equity , PRISMA-Abstract , PRISMA-Protocol  and PRISMA-Network Meta-Analysis  were developed in 2012, 2013, 2015 and 2015, respectively. In 2012, the assessment of the published 476 acupuncture SRs/MAs with PRISMA and their included RCTs showed low quality in general. Information from those studies could not meet the needs of acupuncture practitioners, and therefore was not usable for implementation . Furthermore, the results from a survey of the application status of the PRISMA statement indicated that it was rarely used in the field of acupuncture, possibly because of knowledge gaps among researchers. Another probable reason was lack of items tailored for the characteristics of acupuncture . On the other hand, research in acupuncture develops rapidly with an increasing number of studies published every year . Thus, it is necessary to develop an extension of PRISMA for acupuncture, and implement it along with PRISMA.
Our project aims to develop a reporting guideline for acupuncture SR/MAs. This project consists of three phases: the investigation about the requirements of evidence users for the information needs of reporting of acupuncture SR/MAs; three rounds of Delphi process; and a face-to-face meeting for reaching consensus. This paper described the first phrase, where we aimed to conduct a survey among the evidence users about their information needs, to directly capture their opinions about the current reporting of evidence and the further needs on information reported in acupuncture SR/MAs. Before this survey, we conducted a pilot study in Lanzhou on the questionnaire, and found that most of the questions and items were highly valued by the participants. We also investigated the reporting rates of the items in acupuncture SR/MAs, and found a range of 6.3 to 73.7 % . In this survey, with comments from users and evidence from the literature review, we aimed to design a rigorous and valid questionnaire, and to conduct a survey among acupuncture clinicians, researchers, teachers, and postgraduates across the whole country. Based on this survey, we will draft the items for acupuncture SR/MAs reporting for the Delphi process in next period.
Design of the questionnaire
We designed a questionnaire containing three main parts: 1) demographic information of the respondent, including sex, age, occupation, education, and professional title; 2) the experience, awareness and knowledge on the clinical evidence of acupuncture, including the duration of career, working fields, and reading behavior, 3) the importance of the proposed items from the perspectives of evidence users, including background information about acupuncture, diagnostic criteria of Chinese medicine, types and details of acupuncture, outcome measures of the effects, experience of the operators, and the rationale of the follow-up time. Furthermore, respondents were invited to provide any item they thought important but were missing in our list. The initial items in part three were based on the analysis of existing reporting guidelines, such as STRICTA and PRISMA, and a systematic review of 476 SR/MAs on acupuncture  (see Additional file 1 for a copy of the questionnaire).
The questions were designed to be easy to complete, with several pre-defined options from which the respondents could simply choose their answer. An open-ended response marked “other” was added into some questions if necessary. Respondents were asked to score the reporting items for the necessity and feasibility of inclusion in SR/MAs of acupuncture using the Likert scale  ranging from zero (not necessary) to ten (essential). For the eight items need to be rated, each question included a space for “explanation” in the end for respondents who were willing to provide more details.
Method to conduct survey
We conducted a pilot survey by two trained investigators (XQW, DW) in Lanzhou to find out necessary adjustment and to evaluate the feasibility in an attempt to predict an appropriate sample size and improve the study design prior to performing the formal one. Then, the two trained investigators completed the formal survey between April and June, 2014 by distributing and collecting the questionnaires in person, which could prevent the low response rate of electronic questionnaire. Our investigators were responsible only for distributing the material, interpreting the background, and checking with the respondents to ensure the completeness of the questionnaires. For subjective questions, the respondents were asked to fill it themselves without our investigators’ interfering.
In the formal survey, we visited Traditional Chinese Medicine agencies, hospitals and colleges in five cities (Lanzhou, Chengdu, Shanghai, Nanjing and Beijing). With the inclusion criterion that respondents must have acupuncture clinical experience more than 1 year, we surveyed clinicians, researchers, postgraduates in their second year or higher, and doctors majoring in acupuncture. We surveyed at least 50 individuals in each city, with about 40 % of respondents being students. Acupuncture students, practitioners and researchers have a critical role in bridging evidence and practice, and their opinions are mainly based on practical work which can provide valuable experience on what is the most urgently needed information in acupuncture SR/MAs. Before we distributed the questionnaire, we identified a liaison person in every city, and collected information about the respondents in advance. Then, the investigator visited the respondents one by one. This investigation was approved by the Research Ethics Committee of the First Hospital of Lanzhou University, Lanzhou, China (approval number: LDYYL2013-0007). All the participants were required to sign informed consent. We marked student or non-student on the left top of each questionnaire to avoid errors.
Data were processed using Epidata 3.1 and analyzed with software Excel 2013. For the close-ended question with given options, we used frequencies and percentages to summarize the results. For evaluating the importance of the proposed items, we used mean value of the score, and analyzed the difference between postgraduates and non-postgraduates (including clinicians and researchers) with independent-Sample T-test with the SPSS Statistics 19 software. Additional items provided by respondents beyond this questionnaire were listed in a table. We abstracted all the questionnaire independently and grouped them according to the type of suggestion or opinion on reporting. All discrepancies were discussed and agreed on for final interpretation.
A total of 269 questionnaires were collected, of which 251 (93 %) were used for analysis and 18 (7 %) were excluded because of missing data (Fig. 1). Of the included questionnaires, 52 (21 %) were from Lanzhou, 50 (20 %) from Chengdu, 48 (19 %) from Nanjing, 55 (22 %) from Shanghai, and 46 (18 %) from Beijing. The demographic and socio-economic characteristics of the respondents are shown in Table 1. The average age was 33 years (SD = 9.0, range 25–58).
Table 2 showed the findings on the respondents’ experience and knowledge on clinical evidence of acupuncture. More than half of the respondents had work experience of less than 5 years. The main work of most respondents was clinical treatment with acupuncture. On the other hand, a notable percentage of the participants were involved in more than three types of work. Most of the respondents read one to five articles each month. According to the response, electronic databases, printed professional journal, academic conferences, search engines, and ancient literature were all commonly used to obtain information. Forty-three of the respondents chose at least three or more approaches. The most popular type of literature read by the respondents was RCT, and for the satisfaction of the completeness of information in acupuncture literature, only 2 % were very satisfied.
For the reporting items in our questionnaire, mean scores were all above five. Table 3 presents the respondents’ ranking of each item. The additional suggestions proposed by the respondents about what should be presented in acupuncture SR/MAs were shown in Table 4.
Our survey and questions were designed for acupuncture evidence users with the hope of collecting reporting items for acupuncture SR/MAs, which was different from STRICA for reporting interventions in clinical trials of acupuncture. To ensure the representativeness, we discussed how to select the participants considering various aspects, including geographic location, gender, and work experience. We visited participants from five cities located in south, north, north-west and south-west China, and included about 40 % students. We surveyed students who were at least in the second year of their Master studies, and having completed an internship on acupuncture for at least one year, to avoid surveying participants whose knowledge of acupuncture is too limited to provide practical response. We recruited the participants from universities conducting research and teaching in TCM and their affiliated hospitals, research institutes for acupuncture, and TCM hospitals, to be able to fully reflect the need of scientific research among the various groups of evidence users.
Our findings were not optimistic in terms of the reading behavior of the respondents. Most of the respondents read one to five relevant articles per month, and surprisingly, we found also respondents who never read the literature. Yet most of the participants could give advice about the information they need in an article, based on their clinical experience of acupuncture. Forty-three percent of the respondents were not satisfied with the adequacy of information reported in SR/MAs, which corresponds to the poor reporting quality of acupuncture SR/MAs .
Acupuncture is used for various kinds of diseases  which differ from each other in terms of theoretical and evidence basis. The authors should clearly describe the theoretical knowledge on the use of acupuncture for this disease to promote the rapid understanding for readers. Acupuncture is characterized by a broad diversity of styles and approaches in both East Asia and Western countries , and the ways of conducting acupuncture and the theories behind these different styles also differ . Therefore the appropriate background information on the style, and theoretical basis of acupuncture for the specific disease should be reported.
In general, there are two different sets of diagnostic criteria: syndrome differentiation used in Traditional Chinese Medicine, and diagnostic criteria of diseases used in Western medicine . These criteria should be interpreted in completely different ways, and it is important to indicate which set of criteria, or both, should be used. There are diverse types of needles, and nine classical acupuncture needles are always mentioned, which include filiform needle, shear needle, round-pointed needle, spoon needle, lance needle, round-sharp needle, stiletto needle, long needle and big needle . With different functions , it is necessary to state which type of needles the study can be applied to. At the same time, the details of the acupuncture intervention including numbers of needles, names of acupoints, depths of puncture, de-qi or not, and times for needle retention, will all influence the treatment effect. As for judgement on effect, there are many indicators used only in acupuncture, such as acupuncture-tailored symptomatic relief and adverse effects like bleeding. Between 2000 and 2009, 95 cases of serious adverse events were reported . However, many such events have not been reported among adequately-trained acupuncturists, and they should therefore not be considered inherent to acupuncture, but instead being due to malpractice of acupuncturists . Therefore, a qualified acupuncturist is needed to ensure the effectiveness and safety of the treatment. The necessary follow-up time differs greatly between different conditions, and the patients treated with acupuncture also need to be followed up for possible side effects.
Our results showed a high acceptability and recognition for most items among the respondents (Table 3). The mean scores of the third, fourth, fifth, sixth and eighth item were higher than eight, the first item was 7.2, while items two and seven were lower than six. But we did not exclude or include any items in this period. Instead, we would like to integrate these items with additional comments from the respondents in order to form the checklist of items for the following Delphi process. We also compared the mean scores between postgraduates and NPs (including methodologists, acupuncture clinicians, researchers and teachers), using the independent-sample T-test. The results of the T-test showed that the difference of the mean scores between postgraduates and NPs were non-significant (p value ≥0.05) for six of the eight items, and the p-value was close to 0.05 also for the two remaining ones where the difference was found significant (7.5 and 7.7). This indicated that the experienced acupuncture workers and the less experienced postgraduates tend to put similar attention on the most items, that is, both groups demanded more complete information in acupuncture SR/MAs. For details of acupuncture interventions and qualification of acupuncture clinicians in methods, postgraduates gave higher scores compared with non-postgraduates, which might be related with their experience.
For additional comments in Table 4, the top five mentioned items (mentioned by seven or more respondents) were carefully analyzed. The 7.9.1 and 7.9.2 were not tailored for acupuncture, and they have already been mentioned in PRISMA. As for the 7.9.3 and 7.9.5, they were similar or close to 7.1 and 7.4 respectively. And 7.9.4 will be integrated in the item checklist in the future period.
Our study has both strengths and limitations. We designed the questionnaire based on a pilot study and a literature review, combining the two key factors in evidence based medicine—— evidence and practitioners’ experience . We designated two researchers to distribute and collect printed questionnaires and some explanatory materials without interfering the answer process, which helped us improve the response rate [30, 31]. We proposed eight items collected from current literature in our questionnaire, which probably does not cover all relevant issues for reporting SR/MAs of acupuncture. However, the open ended comments from the respondents gave us useful suggestions for further improvement. We limited our survey to five cities, but these cities represent in the different regions of China. We also considered the representative from the aspects of gender, geography, clinical experience, and experience in literature. The participants are representative of typical users of systematic reviews on acupuncture. Acupuncture is mainly practiced in specialized and general hospitals, where the health care workers are in general well educated and involved in research. In China, acupuncture is not commonly practiced in local health centers or private clinics, where the health care workers tend to be less well educated [32, 33]. In the next period, relevant experts will be identified and surveyed with an online questionnaire, and the results of this survey, including the comments and additional items suggested by the respondents, will be properly considered and integrated.
Most clinicians, researchers and students involved in acupuncture were not satisfied with the information provided in acupuncture SRs. This indicates the need for promoting the more complete and critical information reporting of acupuncture SR/MAs. Most of the items proposed in the questionnaire scored highly, and there were only small, mostly non-significant, differences between the opinions from postgraduates and more experienced NPs. The open-ended questions for respondents for collecting comments and additional items provided us with a lot information, which would promote our work in next step.
Consolidated Standards of Reporting Trials
Journal of the American Medical Association
Preferred Reporting Items for Systematic reviews and Meta-Analyses
Randomized controlled trial
Systematic review or meta-analysis
STandards for Reporting Interventions in Clinical Trials of Acupuncture
Traditional Chinese medicine
Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: CRD; 2009. Available at http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf. Accessed 16 Oct 2014.
Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357(9263):1191–4.
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010;8(1):18.
Endorsers. http://www.consort-statement.org/about-consort/endorsers. Accessed 21 Jan 2015.
Mannocci A, Saulle R, Colamesta V, D'Aguanno S, Giraldi G, Maffongelli E, et al. What is the impact of reporting guidelines on Public Health journals in Europe? The case of STROBE, CONSORT and PRISMA. J Public Health (Oxf). 2014;37(4):737–40.
Panic N, Leoncini E, de Belvis G, Ricciardi W, Boccia S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement on the quality of published systematic review and meta-analyses. PLoS ONE. 2013;8(12):e83138.
Wu TX, representative. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. Chin J Evid-based Med. 2010;10(10):1228–39.
Extensions. http://www.consort-statement.org/extensions? ContentWidgetId = 559. Accessed 21 Jan 2015.
Prady SL, MacPHERSON H. Assessing the utility of the standards for reporting trials of acupuncture (STRICTA): a survey of authors. J Altern Complement Med. 2007;13(9):939–43.
Hammerschlag R, Milley R, Colbert A, Weih J, Yohalem-Ilsley B, Mist S, Aickin M. Randomized controlled trials of acupuncture (1997–2007): an assessment of reporting quality with a CONSORT-and STRICTA-based instrument. Evid Based Complement Alternat Med. 2011;2011:25. Article ID 183910. doi:10.1155/2011/183910.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9.
Welch V, Petticrew M, Tugwell P, Moher D, O'Neill J, Waters E, et al. PRISMA-Equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333.
Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, et al. PRISMA for Abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med. 2013;10(4):e1001419.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777–84.
Liu Y, Zhang R, Huang J, Zhao X, Liu D, Sun W, et al. Reporting Quality of Systematic Reviews/Meta-Analyses of Acupuncture. PLoS ONE. 2014;9(11):e113172.
Wang XQ, Wei D, Liu YL, Wu CL, Ji K, Wei JQ, et al. A survey of application status of the PRISMA statement. Chin J Evid-based Med. 2014;14(9):1160–4 [Article in Chinese].
Wang C, He WJ, Guo Y. Analysis on acupuncture related articles published in periodicals in Science Citation Index(SCI)in 2008. Zhongguo Zhen Jiu. 2010;30(9):755–8.
Liu YL. Research on the Quality of Systematic Reviews and Randomized Controlled Trials of Acupuncture and Cognition of Reporting Guideline. Lanzhou University, 2012. [Article in Chinese]
Dawes JG. Do data characteristics change according to the number of scale points used? An experiment using 5 point, 7 point and 10 point scales. Int J Mark Res. 2008;51(1).
Acupuncture. Available at: https://en.wikipedia.org/wiki/Acupuncture. Accessed 6 Jan 2016.
Birch S, Felt R. Understanding acupuncture. Edinburgh: Churchill Livingstone; 1999.
Western medical acupuncture: a definition. Available at: http://your-gp.com/western-acupuncture-vs-traditional-acupuncture/. Accessed 6 Jan 2016.
Jiang M, Lu C, Zhang C, Yang J, Tan Y, Lu A, et al. Syndrome differentiation in modern research of traditional Chinese medicine. J Ethnopharmacol. 2012;140(3):634–42.
http://www.acupuncturemoxibustion.com/acupuncture/nine-classical-needles/. Accessed 6 Jan 2016.
Zhang TS, Jin CN, Guan F, et al. The history and development of new nine-needle. Chin Acupunct Moxibustion. 2009;07:591–4 [Article in Chinese].
Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. PAIN®. 2011;152(4):755–64.
Xu S, Wang L, Cooper E, Zhang M, Manheimer E, Berman B, et al. Adverse events of acupuncture: a systematic review of case reports. Evid Based Complement Alternat Med. 2013;2013.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71.
Nulty DD. The adequacy of response rates to online and paper surveys: what can be done? Assess Eval High Educ. 2008;33(3):301–14.
Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50(10):1129–36.
Lin ML. Comparison of the Present Situations of Acupuncture and Moxibustion among China Mainland and Taiwan Region. Nanjing University of Chinese Medicine. 2007. [Article in Chinese]
Li ZC. Analysis on Resource Allocation and Utilization in Township Hospitals Under Health Reform in China. Chinese Health Econ. 2009;28(5):31–3 [Article in Chinese].
The authors wish to thank Yajun Wang, Huijuan Tan, Guanghui An and Hao Chen with their help with the planning of survey, and Janne Estill for polishing the paper.
This study is supported by the National Natural Science Foundation of China (General Program, ID: 81373882), and the funding had no role in study design, collection, analysis and interpretation of data, writing of the report, and decision to submit the article for publication.
Availability of data and materials
All data generated or analysed during this study are included in this published article and its Additional file 1.
XES, XQW and KHY contributed to the conception and design, as well as acquisition of data of this study. XQW and DW performed the statistical analysis and drafted the manuscript. XZ and JG participated in the whole survey and helped to revise the manuscript. All authors read and approved the final manuscript.
The authors declare that there is no conflict of interest regarding the publication of this paper.
Consent for publication
Ethical approval and consent to participate
This investigation was approved by the Research Ethics Committee of the first hospital of Lanzhou University, Lanzhou, China. All the participants signed the informed consent.
The copy of our questionnaire. (DOC 44 kb)