Getting inside acupuncture trials - Exploring intervention theory and rationale

Abstract

Background

Acupuncture can be described as a complex intervention. In reports of clinical trials the mechanism of acupuncture (that is, the process by which change is effected) is often left unstated or not known. This is problematic in assisting understanding of how acupuncture might work and in drawing together evidence on the potential benefits of acupuncture. Our aim was to aid the identification of the assumed mechanisms underlying the acupuncture interventions in clinical trials by developing an analytical framework to differentiate two contrasting approaches to acupuncture (traditional acupuncture and Western medical acupuncture).

Methods

Based on the principles of realist review, an analytical framework to differentiate these two contrasting approaches was developed. In order to see how useful the framework was in uncovering the theoretical rationale, it was applied to a set of trials of acupuncture for fatigue and vasomotor symptoms, identified from a wider literature review of acupuncture and early stage breast cancer.

Results

When examined for the degree to which a study demonstrated adherence to a theoretical model, two of the fourteen selected studies could be considered TA, five MA, with the remaining seven not fitting into any recognisable model. When examined by symptom, five of the nine vasomotor studies, all from one group of researchers, are arguably in the MA category, and two a TA model; in contrast, none of the five fatigue studies could be classed as either MA or TA and all studies had a weak rationale for the chosen treatment for fatigue.

Conclusion

Our application of the framework to the selected studies suggests that it is a useful tool to help uncover the therapeutic rationale of acupuncture interventions in clinical trials, for distinguishing between TA and MA approaches and for exploring issues of model validity. English language acupuncture trials frequently fail to report enough detail relating to the intervention. We advocate using this framework to aid reporting, along with further testing and refinement of the framework.

Background

Acupuncture explored in clinical trials is often reported as if it were a clearly defined intervention, which can be taken at face value. In contrast, the theory and practice of acupuncture suggests acupuncture is more accurately described as a complex intervention, with multiple variations in its form of practice and considerable variation in the knowledge and skills of its practitioners and in the theory of healing underpinning them [1]. Moreover, unless researched within the context of placebo controlled trial, the specific and non-specific effects of acupuncture are difficult to pick apart [2]. For example, in attempting to clarify the nature and effects of complex interventions such as acupuncture, Paterson and Dieppe [3] usefully differentiate characteristic from incidental (placebo) effects; characteristic effects of an intervention are defined as being 'theoretically derived, unique to a specific treatment, and believed to be causally responsible for the outcome' (p1202). What is defined as a characteristic and incidental effect will depend on the underlying therapeutic framework that informs the work, for example in the context of acupuncture, a biomedical or Chinese medicine framework. As Paterson and Dieppe report, the biomedical explanation of the symptom (diagnosis) may not drive the rationale for the acupuncture treatment administered in the trial if using a Chinese medicine framework.

This potentially poses considerable challenges in building up an evidence base on the effectiveness of acupuncture. Acupuncture trials produce variable and sometimes inconclusive results [4–6]. With possibly limited detail on the nature of the acupuncture studied within the trial and the underlying therapeutic framework, the results may be difficult to apply to, and to reproduce within, practice. In particular, the mechanism of acupuncture (the process by which the desired change is effected) is often unclear, or may vary depending on the theory driving it, raising questions over the validity of the acupuncture intervention itself [7].

Previous work to address this issue is most notable in the published guidelines on standards for reporting acupuncture trials (STRICTA), first published in 2001 [8] and revised in 2010 [9]. These are intended to improve the completeness and transparency of reporting with a view to enable better interpretation and to allow for replication. Our own interest, and the focus of this paper, lies on the first item within the STRICTA checklist, 'acupuncture rationale.' This draws attention in particular to the need for information on the style of acupuncture and the reasoning for the provided treatment and treatment procedures [9].

Another set of literature raises the connected but wider notion of model validity [10–12]. This draws attention to the need for research to 'fit the paradigm' [10] of the whole system of the particular complementary and alternative therapy under investigation. It asks questions about whether or not the research design takes heed of its underlying philosophy and theory of healing, and thus is assessed within its own explanatory model. This relates inter alia to the practice context (is the studied intervention consistent with current practice?), the credibility of the intervention (is it credible that an intervention based on this explanatory model could lead to the intended effects?), treatment procedures (are these congruent with the underlying healing system?)[12, 13], and understanding of health-related changes arising from the intervention (for example, are the outcomes to be assessed sufficiently broad to encompass the effects of the therapy?)[14–16].

In her recent editorial, Cassidy [11] redraws attention to the complexity of acupuncture. This relates to its multiple styles, approach to needling, attention to the patient within the treatment session and reassessment at a subsequent session. Later, she asks, but 'what do we mean by acupuncture?'[12], and in contrast to biomedicine, what is its explanatory model? The revised Medical Research Council (MRC) guidelines for the conduct and reporting of trials of complex interventions also emphasise the early need for a theoretical understanding of how the intervention might bring about change [17]. Ensuring model validity thus enables clarity in assessing whether the intervention has had the expected effect in a clinical trial.

Aim

Our aim was to aid the identification of the explanatory model underlying acupuncture interventions and the assumed mechanisms within clinical trials by developing an analytical framework to differentiate two contrasting approaches to acupuncture (traditional acupuncture and Western medical acupuncture).

Methods

Uncovering the Underlying Theory

The development of the analytical framework was grounded within the principles of realist review [18]. Interest lies in making explicit how and why an intervention may result in particular outcomes. Realist review applies the principles of realist evaluation to the process of research synthesis. Its first step is to uncover or identify the essential or implicit theory or theories that underlie an intervention, that is, how the intervention is thought or meant to work and its expected impacts. In the current context, this is akin to clarifying the underlying therapeutic rationale that guides and lies implicit within the acupuncture intervention [19].

Developing the Analytical Framework

Within the context of Western professional acupuncture practice and research, two contrasting approaches to acupuncture are evident: traditional acupuncture (TA) grounded within Chinese or Japanese traditional medicine theory, and medical acupuncture (MA) based on a biomedical approach. Both are also differentiated in terms of the nature and length of education and training required for their practice [20].

Five key components of potential difference can be identified: 1) theory (for aetiology, diagnosis and treatment); 2) the rationale offered for how the treatment is expected to work; 3) the practice of the intervention; 4) the implied or assumed causal pathway for change; 5) the desired outcome of treatment. While these five components are interrelated, the underlying theory of acupuncture underpins them all. For example, the theory of Chinese medicine talks in terms of the balance of Qi, embracing how to diagnose, and how to formulate a treatment strategy. The practice should embody the theory; for instance, as Chinese medicine holds to the idea of holism, then its practice must reflect the understanding that mind, body and spirit are integral.

To develop the framework reported here, the educational content and guidelines of two UK professional bodies for acupuncture were examined to identify the core features of each approach. For TA, material from the British Acupuncture Council was examined; their members predominantly subscribe to a 'traditional acupuncture' perspective drawing on Chinese medicine theory. For MA, material from the British Medical Acupuncture Society was explored; their members are medically trained and thus can be characterised as subscribing to a 'medical acupuncture' approach. This material was supplemented by a priori knowledge of one of the authors (SP) [21], a practising acupuncturist-researcher with 25 years practice experience.

Based on these materials one can infer that acupuncturists that practise TA use Chinese and Japanese traditional medicine theory, which for the purposes of this paper we will shorten to Chinese Medicine (CM). Its mechanism, that is, the way in which change comes into being, is essentially individualistic, iterative and interactive with degrees of uncertainty at each stage of treatment, that is, the causal pathway is non-linear [22]. In contrast, MA uses 'neuro-physiological principles' based on evidence that acupuncture alters brain function through stimulating nerve pathways [23]. This is suggestive of a mechanism that has a linear pathway from acupuncture to symptom.

The resulting framework, presented in Table 1, displays a data extraction tool for categorising the acupuncture intervention provided. This framework makes explicit the differentiating features of a TA or MA model. Features of both systems that were about values rather than principles of theory were not included. Two points are noteworthy. Firstly, the heading 'aetiology of the symptom as integral to the treatment' is considered exclusively a TA feature; how a symptom starts provides diagnostic information for this type of acupuncture. Secondly, the heading 'responsiveness' is important as patient responses and changes provide new diagnostic information that may modify the treatment approach (iterative treatment). This is a feature that orientates the TA model to the individual over and above the symptom. In contrast, the MA model focus stays fixed on the symptom.

Table 1 Differentiation of theory, rationale, practice and outcome for two models of acupuncture (see text for more detailed definitions)

Methods for Applying the Framework

This framework was then applied to a set of clinical trials to explore its usefulness in uncovering the theory behind the treatment approach and choice of acupuncture points and thus its ability to expose the stated or assumed underlying causal process of change. In order to do this we applied the conceptual framework to one body of research - published trials of acupuncture for fatigue or vasomotor symptoms. The vast majority of acupuncture research has been conducted with pain for which the suggested causal pathway is simple - a release of beta-endorphins [24]. Choosing two symptoms that are commonly experienced and researched but that might represent greater complexity in terms of mechanism allows a more in-depth exploration of the value of the framework.

The studies were selected from a separate, wider systematic literature review being undertaken within a PhD study of acupuncture and early stage breast cancer. The inclusion criteria for this separate systematic review were:

• Study type: randomised and non-randomised studies, with a comparison/control group; uncontrolled studies published in key journals and/or cited by subsequent studies;

• Outcome measures: validated measures for either fatigue or vasomotor symptoms;

• Acupuncture: to include needling;

Only studies reported in the English language were included, all other studies were excluded, including where acupuncture was used as part of a pregnancy or fertility trial, or acupuncture took the form of auricular acupuncture, laser therapy, acupressure or TENS or used healthy subjects. We systematically searched nine databases from 1986 to 2008: (1) the major biomedical databases of Medline, Embase, CINAHL, PsychInfo and the Cochrane database of systematic reviews; and (2) the specialist databases of AMED, CISCOM, Acubriefs, and Cochrane Complementary Medicine Field Registry. A hand search of English language Chinese medicine journals (Journal of Chinese Medicine, European Journal of Oriental Medicine) was also undertaken. Search terms were 'acupuncture' or 'acupuncture therapy' combined with the above symptoms.

This resulted in a set of fourteen studies. Following the realist evaluation approach, interest lay in uncovering implicit theory that the studies' authors might suggest about how the intervention was thought or meant to work and its expected effects. Focus thus lay on examining the authors' statements about aims, hypotheses and underlying rationale which might be evident within either the introduction or discussion sections of the article. The framework was applied systematically to the selected papers, firstly to assess the degree to which information could be identified demonstrating adherence to a particular theoretical model and secondly to explore any differences in model adherence by symptom.

Results

Applying the Framework

Adherence to a Theoretical Model

The results suggest that studies could fit into one of three models or categories, TA, MA, and 'unclassifiable', that is, where there is insufficient information to identify a fit to either model. Table 2 provides detail for all fourteen studies including the rationale for the categorisation of each study.

Table 2 All included studies

For seven studies [25–31] information is either missing completely or is justified by suggesting that 'experts' chose the points. No clear rationale links the intervention to the outcome, and there is no discussion on any potential mechanisms at play. These studies may imply that acupuncture is a fixed, homogenous intervention and there is no reference to the complex mechanisms at play in the symptoms or what might be changed by the acupuncture. It is thus hard to ascertain what the researchers think might be 'working'. One common problem with these studies is that some suggest by description that they are using a TA model while they are actually using a fixed-point prescription, a common feature of an MA model. Use of a fixed-point prescription invalidates a description of the study as adhering to a TA model. It is, however, possible that authors have described a needle technique using the language of a TA model and omitted to provide a rationale for their point selection. It may be that their approach was founded within a MA model. If so, it is essential to offer a rationale linking point selection to the biomedical mechanism.

Two studies are characterised by the TA model [32, 33]. They use defining aspects as described on the left-hand side of Table 1. Although Dong et al [32] also offers a medical rationale, in addition to a TA one, as to the causal link between acupuncture and vasomotor symptoms this does not dictate how the treatment is carried out. Huang et al's [33] study used a 'placebo needle', which involved plastic rings and tape. This procedure was repeated for the 'real' acupuncture group and practitioners complained of it altering their practice.

Five of the studies use the MA model [34–38]; they are all vasomotor studies and are discussed in more depth below.

Exploration by Symptom

Examining the studies by symptom provides additional insight into how certain approaches have evolved.

Vasomotor Symptoms

Five of the nine vasomotor studies are arguably in the MA category and four come from one group of Swedish researchers [35–38]. Their description of a biomedical mechanism as justifying the use of acupuncture and hence the theory underpinning the intervention was rooted in biomedical science. However there are possible problems in characterising these studies due to the fact that they use acu-points for a symptom other than the outcome measure. The rationale for point selection is reported in Wyon et al [35] and similar language is used in the other four papers:

"As there was very sparse experience of acupuncture therapy for climacteric symptoms, the choice of acupuncture points was a modification of points used in previous studies on acupuncture treatment of dysmenorrhoea."

The authors describe the effects of acupuncture in biomedical terms and match these to a possible mechanism underlying the symptom:

"acupuncture could decrease hot flushes by regulating temperature control through increasing beta-endorphin levels and subsequent inhibition of GnRH."

This begs the question of how specific does the understanding of the biomedical mechanism of the symptom have to be to justify an effect and how strong does the link need to be between theory, anticipated mechanism and actual point selection to ensure a robust model. In addition there is the problem of the hypothesized mechanism of hot flushes and Deng et al [25] are upfront about this:

"it has been hypothesized that acupuncture regulates neurotransmitters involved in thermoregulation. Few data currently support this contention."

Of the other four studies, two follow a TA model [32, 33] and two are unclassifiable [25, 31]. Both Dong et al [32] and Huang et al [33] describe some of the defining aspects for TA displayed in Table 1 but some are assumed; for instance, offering individualised CM treatment is likely to include a differential diagnosis. Deng et al [25] could not be defined as either TA or MA and Cohen et al [34] apply virtually the same fixed-point prescription as the Swedish studies but use some CM language not linked to the point selection.

Fatigue

None of the five fatigue studies could be classed as either MA or TA [26–30] and all studies had a weak rationale for the chosen treatment. Some studies allude to a more 'traditional' model. Vickers et al [27] describe their selection of acu-points as:

"these points typically are used in Chinese medicine to treat fatigue,"

citing in support a widely used textbook [39] that is more focussed on point location than theory of CM. An examination of the referenced textbook for one of the chosen points (Spleen 9) does not refer to tiredness or fatigue but rather to

"abdominal distension, cold and pain of the abdomen, cutting pain in the middle of the intestines, no desire to eat... ". [p194]

In addition there is no explanation as to why the authors modified the points used during the study. The absence or weakness of a rationale for the chosen treatment in all five studies may be partly due to the possibility that no biomedical mechanism associated with fatigue has yet been discovered. It is acknowledged by Vickers et al [27] that fatigue is certainly a multi-factorial problem.

Discussion

Does the Framework Help to Identify the Explanatory Model?

The framework aided identification of large gaps in how the interventions were described as used for these particular symptoms; for instance, in the five MA model studies there was no explicit explanation of why the particular acu-points chosen had a particular action. Journal restrictions on word count may have contributed to the limited detail on the nature and rationale for the intervention. But it is also the case that it reflects the particular symptoms focussed on: there is greater clarity on the biomedical theories of pain and less certainty regarding the mechanisms of symptoms. The studies may also have been made to 'fit' a trial design considered ideal for testing simple, single action, fixed interventions such as drugs. It is possible that the theory of the explanatory model is inadequate to explain the link between the practise and the outcome. Studies from the 'unclassifiable' category offer good examples of how trial design shapes practice. More problematically, lack of detail could be due to implicit assumptions about what acupuncture is. Imposition of a RCT model on acupuncture trials may have confounded clarity over model validity to either a TA or MA form; notwithstanding these two models broadly capture the difference between a symptom-based approach compared to a whole person individualised approach and demand a rationale for the mechanism of action.

The framework not only exposes gaps in reporting, but also highlights issues in exploring effectiveness of acupuncture. If a cumulative evidence base is to be developed on the effectiveness of complex interventions such as acupuncture, ensuring both validity and comparability of the intervention is essential. Trials undertaken within a MA model should be compared with another, and those within a TA model one similarly. Comparison of the two models is only warranted when interest lies in the question of the form, 'does a MA or TA model produce better outcomes?'

Notions of Causality

Different notions of causality may be implicit within the theory of an intervention. In the case of MA, causality is biomedical and linear, whereas in TA the causal logic implied is recursive and non-linear. This adds to the complexity of an intervention. Whatever way the intervention is described, (for instance, 'classical' or 'traditional') it is the rationale that drives the selection of acu-points that exposes the true theoretical basis for treatment.

What are the Gaps in the Knowledge?

It is evident from this selection of clinical trials that some researchers have rationalised an acupuncture intervention for the use on outcomes other than that being measured based on a hypothesized effect that may link the two. Lack of empirical data may be a reasonable rationale for this; but there is a large body of knowledge available in China, through different branches of science and history that may provide better rationales. Confidence in the model of MA, and in the results of the studies, would be increased if it was possible for authors to explain why certain acu-points, as opposed to others, or even any, have anticipated effects on biomedical mechanisms.

Implications

Measuring the effectiveness or efficacy of acupuncture is complex, not least because it is difficult to pick apart characteristic and non-specific effects. Studies that clearly define how they do what they do and why they are doing it are more likely to have results that satisfy a range of possible stakeholders than studies that offer an intervention that lacks model validity. It is also important for authors to avoid descriptive short cuts; describing a traditional needle technique cannot be taken as a proxy for the use of CM theory as a basis for a TA model.

Researchers need to provide sufficient information to demonstrate the way in which they have utilised a particular model of acupuncture and as an aid to others evaluating their studies. To assist this process, Table 3 presents a draft tool as a first step to explore and assess model validity in acupuncture interventions. It presents a set of six core questions and associated guiding prompts to ask of a study of acupuncture and may be especially useful where the desired outcomes or symptoms treated have a more complex or unknown mechanism than, for instance, pain. Such a tool requires further exploration, testing and refinement before it can be adopted.

Table 3 Exploratory guide for assessing acupuncture model validity (for complex outcomes)

For the production of robust data, it is not acceptable to use acupuncture in a clinical trial without explicitly describing the theoretical basis of how it is going to work and the rationale for treatment in the published reports [10]. However, it is important to recognise that hypothesised theoretical mechanisms expounded in published trials may be different from the theory-based practice undertaken in real world settings [40].

It is possible that the alien technical language of CM is not seen as appropriate for publication and thus the mechanism of acupuncture according to CM is often not described in detail, but can be assumed from the description of theory. One possible step forward would be for journal editors to agree to publish further, web-based, information in parallel to the trial report.

The framework as developed proved to be a useful aid for identifying and exploring the theory underlying acupuncture intervention trials. It needs further testing with other studies of acupuncture, or non-English language studies, addressing different symptom groups, in order to assess its wider value. This is particularly important given the extensive literature on acupuncture published within the Chinese language [41, 42].

Summary

The aim of this paper was to present the development of a framework and to see if it helped to identify the explanatory model and underlying rationale in published reports on acupuncture trials. It complements the STRICTA guidelines by exploring in greater detail the 'acupuncture rationale'. The insight provided by use of the framework, that is, 'what is the nature of acupuncture as explored within this research study?' is valuable in three main ways. Firstly, at its most simple, it explicitly draws out 'what (form)' and 'how' acupuncture has been undertaken. Secondly, it enables comparison of trials that use a similar approach with one another, rather than putting all acupuncture trials together as if they were all the same (that is, comparing like with like), both in discussions of the implications of a single trial in relation to others and within a systematic review. Thirdly, it may contribute to current debates concerning 'model validity' and what works in acupuncture. Our application of the framework to these selected English language studies suggests that acupuncture trials frequently fail to either apply or report enough detail when using an acupuncture intervention. We would advocate the application of the acupuncture model validity guide to a range of acupuncture trials, along with its further testing and refinement.