Quality of reporting on randomized controlled trials of acupuncture for stroke rehabilitation

Background Results from clinical studies on acupuncture for stroke rehabilitation are contradictory. The reason for the inconsistent findings especially lie in the transparency and accuracy of randomized controlled trials (RCTs) reports. This study aims to analyze the quality of reporting and its correlates in RCTs on acupuncture for stroke rehabilitation. Methods Quality of reporting for included papers was assessed against a subset of criteria adapted from the CONSORT 2010 statement and STRICTA. An overall quality score (OQS) and a combined key methodological index score (MIS) was calculated for each trial. Then, factors associated with OQS and MIS were identified. Results A total of 15 RCTs were included in full text. The median OQS based on the CONSORT statement and STRICTA was 8 and 12, respectively. The significant predictors for CONSORT OQS was funding source, for STRICTA was year of publication. With regard to the MIS, no variable was associated with improved methodological quality. Conclusions Our study found that the overall quality of reporting on RCTs of acupuncture for stroke rehabilitation was general or good. But some items’ reporting was found where information was insufficient or inadequate in most studies which needed substantial improvement.


Background
Stroke is a leading cause of death in both middle-and high-income countries, responsible for 22% of all deaths worldwide [1]. The risk factors leading to increasing stroke incidence are an aging population, dietary changes, and work-related stress, etc. In China, stroke is the second most common cause of death in cities and the third most common cause of death in rural areas [2]. Also, stroke is a major cause of disability and dependency. Poststroke disability with a high incidence (>50%) brings a heavy burden to patients and their caregivers [3]. In the United States, the total societal and healthcare costs have risen from $53.6 billion in 2004 [4] to $68.9 billion in 2009 [3]. Clearly, patients with stroke urgently need safe and effective treatments for alleviating the burden, which drives people to search for conventional treatment to improve the effect of stroke rehabilitation, such as acupuncture.
For more than 3000 years, practitioners in China have used acupuncture to treat various diseases, including the sequelae of stroke. Acupuncture is widely used to improve motor, sensation, speech and other neurological functions in patients with stroke. Compared to other conventional interventions, acupuncture is relatively simple, inexpensive and safe which has been well accepted by Chinese patients and also increasingly practiced in some Western countries [5]. Results from clinical studies on acupuncture for stroke rehabilitation, however, are contradictory. Although some reviews indicate that too little evidence exists to prove the efficacy of acupuncture for stroke patients [6][7][8][9], a series of studies suggest that this modality is an effective method for improving disabilities due to stroke [10][11][12][13]. Researchers have called the quality of these meta-analyses into question, however, and the likely potential for publication bias suggests a need for additional research on the subject. Currently, it is necessary to evaluate the quality of reporting on randomized controlled trials (RCTs) of acupuncture for stroke rehabilitation, for quality of reporting is essential for guiding journal peerreview decisions, for experts' recommendations, and to conduct unbiased meta-analysis, as it influences our interpretation of evidence.
The Consolidated Standards for Reporting Trials (CONSORT) statement developed in 1996 [14] aimed to facilitate critical appraisal and interpretation of RCTs by authors, providing them guidance about how to improve the reporting of their trials, as well as by peer reviewers and editors, helping them identify reports with potentially biased results. The CONSORT statement has been further revised and is published as the CONSORT 2010 statement. Combined with the CONSORT statement, the acupuncture-specific Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) were designed to help authors fully report the acupuncture intervention in clinical trials [15,16]. The combination of both tools helps us evaluate the completeness and transparency of RCTs reports on acupuncture for stroke rehabilitation according to their items.
However, to the best of our knowledge, there are no data about the quality of RCTs reporting in acupuncture for stroke rehabilitation. The present study aims to (i) describe the characteristics of published RCTs reports on acupuncture for stroke rehabilitation; (ii) assess the overall quality and key methodological items of published articles of RCTS on acupuncture for stroke rehabilitation; and (iii) determine factors associated with better reporting quality.

Search strategy
We searched the Cochrane Stroke Group Trials Register

Inclusion & exclusion criteria
Types of studies RCTs comparing acupuncture with at least one control group that used placebo, sham treatment or conventional treatment in patients with subacute (one to three months since onset) or chronic stroke (over three months since onset) were included in this research. RCTs were included if invasive acupuncture was used as the sole intervention or as an adjunct to another standard treatment for stroke rehabilitation and if the control group received the same concomitant treatments as the acupuncture group. A randomization sequence generated from a random number table, calculator or computerized random number generator was considered authentic.

Types of participants
Patients of any gender, sex or ethnicity with ischemic or hemorrhagic stroke in the subacute or chronic phases were eligible. Stroke must be diagnosed according to the World Health Organization definition (rapidly developed clinical signs of focal (or global) disturbances of cerebral function, lasting more than 24 hours or leading to death, with no other apparent cause than of vascular origin [17]; or magnetic resonance imaging (MRI) or confirmed by computerized tomography (CT ).

Types of interventions
Acupuncture therapy will be defined as body acupuncture, scalp acupuncture, electro-acupuncture, warm needling, or ear acupuncture, etc. Studies that compare different types of acupuncture were excluded. Placebo acupuncture refers to a needle attached to the skin surface (not penetrating the skin but at the same acupoints) [18]. Sham acupuncture refers to: 1) a needle placed in an area close to but not in the acupuncture points [18]; 2) subliminal skin electro stimulation via electrodes attached to the skin [19].

Rating of overall reporting quality
An overall quality score (OQS) with 15 items from the CONSORT 2010 statement [20] and 17 items from the STRICTA [21,22] were used (Table 1). Each item was scored 1 if it was reported and 0 if it was not clearly, or definitely not stated. The CONSORT discussion section items were excluded because we considered them too subjective to evaluate.

Rating of key methodological items
Concealment of allocation, appropriate blinding (either practitioner, participant or assessor), and analysis according to intention-to-treat (ITT) principle are highly related to potential sources of systematic bias and thus distortion in the estimation of the effect. Consequently, these three important key methodological items from the CONSORT 2010 Statement have been assessed separately. We then developed three "yes"/"no" items, so that emphasis was placed on quality of reporting rather than adequacy of trial design. A combined key methodological index score (MIS) was calculated for each trial by combining the scores of these three factors (range, 0 to 3).

Data extraction & analysis
Each article was reviewed by two independent investigators. They extracted useful information by modified CON-SORT and STRICTA checklists (Table 1, 2 and 3). After both raters finished their review, we calculated Cohen's к-statistics to assess agreement between two reviewers. Agreement was judged as poor if к ≤ 0.20; fair if 0.20 lower than к ≤ 0.40; moderate if 0.40 lower than к ≤ 0.60; substantial if 0.60 lower than к ≤ 0.80; good if к higher than 0.80; and perfect if к = 1. Discrepancies were reviewed in detail and subsequently settled by consensus.
To identify factors associated with the overall quality of publications, we used the OQS as the outcome variable which was modeled using linear regression. Only variables that were significant at P ≤ 0.10 in the univariate models were used in a multivariate regression model for selecting significant variables (P ≤ 0.05). To identify factors associated with methodological quality, we used the MIS as the outcome variable in the regression analyses. As the outcome variable can be considered as a count, we relied on ordinal regression model and adjusted the variance empirically.
Linear and ordinal regression analysis was performed using SPSS version 20.0. Database of RCTs in acupuncture for stroke rehabilitation are provided in Additional file 1 and 2.

Results
The RCTs selection process is outlined in Figure 1. After identification, screening, eligibility determining of the literatures, a total of 15 relevant RCTs were included in the final analysis.

Characteristics of the included trials Year distribution of publication
Counting the number of articles, frequency, which refers to acupuncture for stroke rehabilitation in RCTs, was

Publication language
Of the 15 RCT papers, 10 (66.67%) were written in English. The remaining 5 papers (33.33%) were published in Chinese.

Nationality of authors
A total of 7 (46.67%) included papers were written by authors in Chinese research institutes; 7 (46.67%) were reported by international researchers; the remaining one paper (6.66%) was collaborations of Chinese and international researchers.

Funding source
Seven articles (46.67%) reported their sources of funding. Funding was obtained from provincial/municipal or international institutes.

Choice of comparator interventions
Interventions of nine trials (60.0%) comprised of sole acupuncture, followed by acupuncture plus other therapies (6 reports; 40.0%).

Quality of reporting
Rating of overall reporting quality The ratings of overall quality of reporting based on the CONSORT statement were listed in Table 1. When the 15 RCTs were considered, the median OQS was 8, with a minimum of 1 and a maximum of 15. Poor reporting existed in terms of "outcomes", "sample size", "outcomes and estimation", "ancillary analyses" with positive rate of less than 30% ( Table 1). All of the items had a substantial, good or perfect agreement ( Table 1). The ratings of overall quality of reporting based on STRICTA were listed in Table 2. When the 15 RCTs were considered, the median OQS was 12, with a minimum of 3 and a maximum of 16. Poor reporting existed in terms of "4a: Details of other interventions" and "4b: Setting and context of treatment" with positive rate of 20% and 33% ( Table 2). All of the items had a substantial or good agreement ( Table 2).

Rating of key methodological items
Allocation concealment, blinding, and analysis by intent to treat were reported in 5 (33%), 9 (60%), and 5 (33%) of the 15 RCTs, respectively ( Table 3). The median MIS was 1 with a minimum of 0 and a maximum of 3 (Table 3). Among the 15 studies, 5 (33%) did not report any of the three key methodological items. All of the items had a substantial or good agreement (Table 3).

Exploratory analysis: factors associated with better reporting quality
In univariate analyses, nationality of authors and funding source were associated with an increased OQS based on No. of participants in each group included in each analysis and whether it was done by "intention to treat" 5 33 6 to 60 0.82 0.68 to 1.00 Figure 1 Diagram flow of the randomized controlled trials articles selection process.
the CONSORT statement. After adjustment, the multivariate linear regression model suggested that funding source remained independent and significant predictors of overall quality. Specifically, the mean OQS based on the CONSORT statement increased by about 4.05 for manuscripts with funding source (95% CI, 0.44 to 7.67; P < 0.05) ( Table 4).
In univariate analyses, only year of publication was associated with an increased OQS based on STRICTA. Specifically, the mean OQS based on STRICTA increased by about 0.48 for manuscripts published in the period of one year increment (95%CI, 0.00~0.96; P ≤ 0.05) ( Table 5).
With regard to the MIS, in ordinal regression analyses, no variable was associated with improved methodological quality (P > 0.05).

Discussion
To the best of our knowledge, this study is the first investigation on the quality of reporting on RCTs of acupuncture for stroke rehabilitation according to the revised CONSORT and STRICTA guidelines. We found evidence that quality of reporting of the overall CONSORT items was general with median OQS 8 which was more than half of its total score 15; quality of reporting of the STRICTA items was good with median OQS 12 which was more than 70% of its total score 17; however, it is poor quality for the reporting of key methodological items with median MIS 1 which was 33% of its total score. Similar results were reported in some published studies [23,24]. Some studies found that the quality of reporting in RCTs on Chinese medicine remained poor (mainly in key methodological items), but had improved over time [25,26]. None of the variables we looked at in our ordinal regression model were significant predictors of improved methodological quality. We noted that the overall OQS based on the CONSORT Statement was correlated with funding source and STRICTA score was correlated with year of publication.
The purpose of reporting guidelines such as CON-SORT and STRICTA is to increase the transparency of study methods and ultimately improve the overall quality of research. However, poorly reported trials make it ambiguous for other readers to assess the validity of the results and may mislead medical policy-makers in their decisions. Our study found that the overall quality of reporting was general or good between 1997 to March 2013. There might be multiple reasons. First, most of included reports are published after 2001 when STRICTA and revised CONSORT statement was published, especially, STRICTA score was correlated with year of publication. In the evaluation of acupuncture trials performed by Prady et al. [27], the authors concluded that reporting of CONSORT items had improved after the introduction of CONSORT but that the introduction of STRICTA did not improve the reporting of STRICTA items. However, another evaluation of auriculotherapy trials [23] came to similar conclusions with ours. Maybe the introduction of STRICTA and CONSORT statement improved the overall quality of reporting in our research. Second, more and more clinicians were training in study design and reporting of RCTs, and some better quality research papers had been submitted and published in international journals which had a more rigorous peer-review process. Our finding that the overall OQS based on the CON-SORT statement was correlated with funding source, also suggested clinical trials with funding have more capacity to provide assurance for the better quality of study design and reporting of RCTs.
However, some items' reporting was found where information was insufficient or inadequate in most studies. These areas are as follows: "allocation concealment", "analysis by intent to treat", "outcomes", "sample size", "outcomes and estimation" and "ancillary analyses" based on the CONSORT statement; "details of other interventions" and "setting and context of treatment" based on STRICTA.
A generated allocation schedule should be implemented by using allocation concealment, a critical mechanism that prevents foreknowledge of treatment assignment and thus shields those who enroll participants from being influenced by this knowledge [20]. Trials in which the allocation sequence had been inadequately or unclearly concealed yielded larger estimates of treatment effects than did trials in which authors reported adequate allocation concealment. Intention to treat was defined as the inclusion of all patients randomly assigned in the analysis, regardless of whether they actually satisfied the entry criteria, the treatment actually received, and subsequent withdrawal or protocol deviations [25,27]. ITT is generally favored because it avoids bias associated with non-random loss of participants. Unfortunately, we found only 33% of the trial reports provided a description of ITT or allocation  concealment, making it very difficult to judge the validity of their findings.
Authors should indicate how the sample size and primary outcome was determined. Theoretically, clinical significant difference between interventions was detected by a high power if a trial had enough number of subjects. The researcher should note the primary and second outcome on which the calculation was based. For each primary and secondary outcome, results for each group, the estimated effect size and its precision (e.g. 95% confidence interval) presentation were recommended. However, no more than 30% of included reports presented the primary and second outcome, sample size calculation and outcome estimation, etc.
As for STRICTA, "details of other interventions" referred to the auxiliary techniques, prescribed self-treatment and lifestyle advice provided by the practitioner. All additional components, whether carried out by the practitioner or patient and whether integral or adjunctive to the acupuncture needling, should be described clearly [21]. "Setting and context of treatment" could also provide important additional components to treatment. For patients, the context factor includes some information given by practitioners might modify the trial's outcome; for practitioners, it includes some instructions given by patients after prescribing or proscribing explanations to them might modify practitioners' normal practice [21]. But the number of reports provided this information was small. As different acupuncturists providing treatment to different treatment arms will influence generalisability of the trial results, the details of other interventions and the background of both groups should be explained.
One limitation we should point out is that, due to language barrier, we didn't search for any manuscripts published in non-Chinese or English journals. This is an area for worthwhile future study. It remains unknown whether searching in other language journals would have altered the constitution of our sample or results.