JOURNAL TRANSCRIPT

---

DOI: 10.1111/1471-0528.14528 www.bjog.org

Systematic review

Participation in clinical trials improves outcomes in women’s health: a systematic review and meta-analysis SK Nijjar,a MI D’Amico,a NA Wimalaweera,b NAM Cooper,a J Zamora,a,c KS Khand a

Women’s Health Research Unit, Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London, UK b The London School of Medicine and Dentistry, London, UK c Clinical Biostatistics Unit, Hospital Ramon y Cajal (IRYCIS) and CIBER Epidemiologia y Salud Publica, Madrid, Spain d Multidisciplinary Evidence Synthesis Hub (mEsh), Barts and the London School of Medicine and Dentistry, Queen Mary University, London, UK Correspondence: Maria I. D’Amico, Women’s Health Research Unit, Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London E1 2AB, UK. Email [email protected] Accepted 12 December 2016. Published Online 14 February 2017.

Background Previous reviews examining the effect of participation

in trials on outcomes have not consistently shown benefit. Obstetrics and gynaecology is a unique disease area posing challenges for both researchers and patients. Objectives To determine whether participation in randomised

controlled trials (RCTs), compared with non-participation, has a beneficial effect on women’s health. Search strategy Medline, Embase, the Cochrane Library, and

PsycInfo were searched up to December 2015. Selection criteria We selected studies that reported the same

clinical outcomes for participants in a women’s health RCT and a comparable non-participant cohort. Data collection and analysis Data were extracted on quality,

average (OR 0.75; 95% CI 0.64–0.87; I2 = 64.3%). The beneficial effect of participating in a trial was larger in comparisons where: RCTs were of high quality (OR 0.62; 95% CI 0.50–0.76) versus low (OR 0.92; 95% CI 0.74–1.16); and RCT intervention was not available to non-participants (OR 0.57; 95% CI 0.47–0.69) versus when it was (OR 1.13; 95% CI 0.89–1.44). The effect of trial participation was not influenced by effect size within the RCT (P = 0.48), whether funding was received or not (P = 0.13), whether non-participants received any treatment or not (P = 0.49), and the quality of the comparison of RCT participants with non-participants (P = 0.88). Conclusions Women participating in RCTs on average

experienced better outcomes compared with those outside trials.

characteristics and study results. Outcomes were compared using logistic regression.

Keywords Neonate, randomised, systematic review, trial participation, women’s health.

Main results There were 21 relevant studies (20 160 women,

Tweetable abstract Participants in obstetric and gynaecology

4759 outcome events). Trial participants, compared with nonparticipants, had 25% better odds of improved outcomes on

RCTs experience better outcomes compared with nonparticipants.

Please cite this paper as: Nijjar SK, D’Amico MI, Wimalaweera NA, Cooper NAM, Zamora J, Khan KS. Participation in clinical trials improves outcomes in women’s health: a systematic review and meta-analysis. BJOG 2017; DOI: 10.1111/1471-0528.14528.

Introduction Clinical research aims to benefit people in the future, enabling them to live longer, healthier lives, but might it also be of benefit to its subjects? Trial participants receive care under enhanced oversight1,2 above and beyond usual care, which is ensured by ethics and governance approvals,3 data and safety monitoring,4 and protocol compliance. Yet PROSPERO registration number: CRD42015025007

ª 2017 Royal College of Obstetricians and Gynaecologists

research on the effect of participation in trials versus nonparticipation has not consistently shown evidence of benefit.5–8 This may be because the way innovations tested in trials differ from care outside trials9,10 varies by clinical area, and combining data from a number of specialities into a single synthesis can mask an effect. Previous reviews5–8 on effect of participation in trials have not examined speciality subgroups. Research in women’s health, which tends to be publicly funded, with industry funding lagging behind,11 is unique in many

1

Nijjar et al.

ways:12,13 for example, due to the combination of mother and offspring as a single unit, the social and cultural sensitivities in gynaecology, complex disease processes, interventions and outcomes, which may predispose participants to benefit or harm in a manner that cannot be surmised from the broad syntheses previously reported.5–8 Therefore, there is good reason to believe that the influence of trial participation in women’s health could be distinct from other specialties.14,15 The objective of this systematic review was to test the hypothesis that there is a benefit of participation in women’s health trials, versus non-participation, and to explore reasons for heterogeneity.

Methods We performed the review in line with current methodological practice with a prospectively registered protocol (PROSPERO registration number: CRD42015025007),16 and reported it in accordance with the PRISMA statement.17

Data sources and study selection We conducted a comprehensive literature search to identify eligible studies in the specialty of obstetrics and gynaecology18 adopting well-developed search strategies (Supporting Information Appendix S1).8 Twenty-four obstetrics and gynaecology RCTs were identified in a previous review searching until 2010.8 We updated the search from this review until December 2015 and filtered for studies in women’s health by screening titles to identify additional studies. We used a combination of keywords and word variants: ‘randomised clinical trial’, ‘non trial’, ‘refusers’, ‘decline’, ‘preference trial’ and ‘standard care’ to interrogate MEDLINE (1966 to December 2015), Embase (1980 to December 2015), Cochrane Central Register of Controlled Trials (CENTRAL; 1960 to last quarter of 2015) and PsycINFO (1880 to December 2015). Two reviewers (MD and SN) independently reviewed all citations and abstracts to identify those that potentially fulfilled the selection criteria. The final selection of articles was based on independent review of their full texts. Studies were included in the systematic review if an RCT in the specialty of obstetrics and gynaecology reported the same clinical outcomes for participants and their comparable non-participants outside the trial in the same publication. For meta-analysis, studies were retained if it was possible to construct 2 9 2 data tables. Studies were excluded if they lacked a comparable non-participant group, if the subject was outside obstetrics and gynaecology, if the paper was about in vitro or animal research, or if the outcome reported was not patient-centred. When duplicate studies existed, we only used the most up-to-date articles. Disagreements that could not be resolved between primary reviewers were referred to an independent third adjudicator

2

(JZ or KK) who was not involved in the initial screening. This involved the adjudicator reviewing the full text against the inclusion and exclusion criteria.

Data extraction and study quality assessment Data extraction was performed on a piloted form in triplicate (MD, SN and NW/KK) to ensure reliability and accuracy. Disagreements were resolved via consensus (MD, SN, NW and KK) or arbitration (JZ). For the selection of outcomes we adhered to the principles employed by the definition of core sets based on relevance and importance to stakeholders.19,20 In cases where core sets had not been developed, the most clinically relevant patient-centred outcomes (as judged by obstetrician-gynaecologist reviewers, SN and KK) were identified prior to event rate data extraction for 2 9 2 tables. Extraction of continuous data was complicated by the mixtures of post-intervention and change from baseline results.21 We preferred post-intervention results, but these were not reported in a manner that allowed inclusion in meta-analysis. We also collected data regarding the care setting, study design, and the intervention and comparator within the trial, and their use among non-participants. We attempted to collect missing data by contacting study authors. Quality of RCTs was assessed using the Jadad scale22 (scored 0–5) scrutinising the method of randomisation, blinding and patient withdrawals or dropouts.22 RCTs with a score of ≥3 were considered to be of high quality.23 Quality of the comparison between RCT participants and nonparticipants was assessed using a modified Newcastle Ottawa score (NOS)24 and focused on three domains (scored using a star rating system) addressing the risk of selection bias (baseline comparability), performance bias (exposure to RCT or not) and ascertainment bias (outcome assessment).24 Selection bias assessment (maximum 4 stars) examined whether known confounding factors were measured, reported and found comparable. Performance bias assessment (maximum 3 stars) scrutinised clarity of the correct allocation of participants and non-participants to within or outside RCT groups considering rates of switches from RCT to outside group. Ascertainment bias assessment (maximum 5 stars) evaluated whether RCT participants and non-participants had similar methods of outcome assessment, follow up of groups, and completeness of data. Studies with ≥6 stars were considered to be high quality if they scored ≥2 stars per domain.

Data synthesis We calculated the odds ratio (OR) and 95% confidence interval (CI) for each study using the 2 9 2 data tables for comparison of outcomes amongst RCT participants versus non-participants. We produced a Forest plot to visually inspect variability between studies and computed the

ª 2017 Royal College of Obstetricians and Gynaecologists

Trial participation in women’s health: review

I-squared statistic to quantify its magnitude (assuming 25– 75% indicated moderate heterogeneity).25 We examined funnel asymmetry to test for publication and related biases.26 We pooled the individual OR estimates across studies using a random effects model.27 Multilevel logistic regression models including interaction terms (the product of a covariate and main effect, i.e. participating in a trial or not) were used to explore sources of heterogeneity.28 Interactions found statistically significant were further explored to produce estimates of the effect of participating in a trial for the different subgroups. We hypothesised that the effect of participating in an RCT versus non-participation would be influenced by factors related to the RCT (effect of intervention versus control within trial, quality of RCT, and funding use) and to being outside it (use of RCT intervention among non-participants, availability of other treatments or observation only among non-participants, and quality of the comparison between participants and non-participants); all six factors were analysed. Analysis was performed using STATA software (StataCorp. 2014. Stata Statistical Software: Release 13. College Station, TX, USA: StataCorp LP).

(Figure 3); lack of transparency of randomisation introduced risk of bias in two studies.29,31 The quality of comparison between RCT participants and non-participants was high in 18 studies; the remaining three studies had a risk of bias in the selection and performance domains.29,30,40

Results

Discussion 29–49

Data synthesis Overall, when compared with non-participants, RCT participants had 25% better odds of improved outcomes on average (OR 0.75; 95% CI 0.64–0.87; I2 = 64.3%) (Figure 4). Analysis exploring sources of heterogeneity showed that the beneficial effect of participating in a trial was larger for two subgroups: high quality RCTs (OR 0.62; 95% CI 0.50–0.76) versus low quality RCTs (OR 0.92; 95% CI 0.74–1.16); and when RCT intervention was not available to non-participants (OR 0.57; 95% CI 0.47–0.69) versus when it was (OR 1.13; 95% CI 0.89–1.44). The effect of participating in a RCT was not influenced by the remaining subgroups: effect of intervention within the RCT (P = 0.48), whether funding was received or not (P = 0.13), whether non-participants received treatment or not (P = 0.49), and quality of the comparison of RCT participants with non-participants (P = 0.88).

Of the 17 013 records identified (Figure 1), 21 studies (20 160 women, 4759 outcome events) were included [see Supporting Information Table S1 for excluded studies50–61]. The number of outcome events for RCT participants included were 1055 and for non-participants 3704. There was good agreement for study selection amongst primary reviewers (weighted j = 0.82). Funnel plot analysis exhibited no evidence of publication and related biases with either the Begg’s (P = 0.25) or the Egger’s (P = 0.56) test (Figure 2).

Main findings

Characteristics and quality of included studies

Strengths and limitations

There were 11 obstetric and 10 gynaecological studies, predominately from high-income countries (Supporting Information Table S2). With respect to design, all studies were RCTs, of which eight were partially randomised patient preference trials,35–37,40,42,44–46 and the others conventional RCTs. Size of studies varied (median 293, range 33–5519) with three papers involving 2000 participants.39,43,46,48 All but one of the studies48 were set in secondary care, defined as services provided by medical specialists based in a hospital or clinic setting. The trial interventions can be arranged into three categories: medical 12, surgical 6, and other 3. Of the 15 studies that received funding,29,30,33,34,36,37,39,41–44,46–49 three received funding from commercial organisations.34,43,44 Around half of RCTs were of poor quality mainly due to inadequacies in blinding and in description of withdrawals

To our knowledge, this is the first systematic review investigating the association between trial participation and outcomes exclusively in women’s health. Building on previous works5–8 that investigated this question more generally, we focused on the obstetrics-gynaecology specialty.18 This is the first review to report a significant benefit as a result of trial participation. Due to the distinct nature of women’s health we focused solely on this disease area and within this subgroup analyses we observed an effect not previously seen in overall syntheses. A retrospective study of a regional cancer centre in the USA found clinical trial participation to improve significantly the survival of women with ovarian cancer compared with non-participants (median 46 versus 25 months, 95% CI 1.03–2.15 months, P = 0.03).62 That paper, though not part of our meta-analysis, reached the same conclusion as ours.

ª 2017 Royal College of Obstetricians and Gynaecologists

Our study found that women participating in trials had better health outcomes than those cared for outside trials. Our subgroup analysis demonstrated that the observed effect was enhanced in comparisons involving high quality RCTs and in comparisons where the RCT intervention was not available outside the trial. The benefit accrued by women participating in RCTs was sustained irrespective of whether the RCT intervention was effective or not.

3

Nijjar et al.

Figure 1. Study selection process for the review on effect of participation in randomised controlled trials (RCTs) versus non-participation in women’s health.

Figure 2. Funnel plot for publication and related biases.

4

Gathering data for our meta-analysis was challenging, as few researchers conducting RCTs include concurrent nonrandomised populations in their studies. Our search emphasised looking for comparisons within the same publications to seek studies where our question could be addressed directly. However, if the RCT and the non-randomised cohort were published separately and there was no cross referencing in the published papers, there is a possibility of missing such studies. For example, Cooper et al.63 reported an RCT without reference to their patient preference cohort. Although our search picked up the Cooper et al. RCT, in the absence of any identifiable information within it about the non-randomised cohort, we could not include this comparison in our review. To fully take into account the impact of such omissions, a full further review

ª 2017 Royal College of Obstetricians and Gynaecologists

Trial participation in women’s health: review

Figure 3. Assessment of (a) randomised controlled trials (RCTs) quality and (b) quality of the comparison between RCT participants and nonparticipants (numbers within the bars represent the number of studies).

would be needed, including study quality assessments and re-analyses. However, using the author-reported primary outcome (success of treatment at 6 months) it is possible to see that in this individual comparison there was no significant difference between participants (76%, 334/439) and non-participants (82%, 276/338) (P = 0.06).64 Thus our overall conclusion would not necessarily be invalidated. Given the broad nature of women’s health we were aware that there would not be a sufficient number of studies to conduct a meta-analysis of a specific disease area; therefore we had to pool all studies found, leading to moderate heterogeneity given the varieties of settings, methodologies, outcomes and interventions. In some cases, for obstetric studies we selected outcomes related to the health of the fetus (five studies) or neonate (three studies) and not the woman, as fetal and neonatal outcomes are proportional to the health of the mother and are influenced by maternal interventions. To be pragmatic, we had to adapt our protocol by reducing the number of variables such that each subgroup could have sufficient number of studies for a reliable analysis. A random effects model was appropriate in light of the moderately high I_squared statistic. In a previous review,8 24 obstetrics-gynaecology RCTs were identified but not all had data available for our planned analysis. In Table S1 we detail potentially eligible studies that we excluded for transparency. Our results were not affected by funnel asymmetry. The majority of the comparisons between trial participants and non-participants were of high quality. Several

ª 2017 Royal College of Obstetricians and Gynaecologists

studies included in our meta-analysis were partially randomised patient preference trials,35–37,40,42,44–46 an approach that allows capture of a concurrent non-participant cohort with the same eligibility criteria as those within the RCT, a feature preferred over historical comparisons.65,66 However, this design may allow for selection bias, as trial participants that are given a choice of intervention may have inherent psychological differences. For example, there may be different levels of motivation and compliance in participants given a choice of treatment versus trial participants where choice was not offered. To minimise the risk of bias we captured objective clinically important outcomes not influenced by the subjectivity of preference. The quality assessments (Figure 3) reassure us about the low risk of inherent biases and improve the trustworthiness of our finding in the context of observational comparisons.

Interpretation The public, and even doctors, nurses and allied health professionals, may regard research as an ‘experiment’, i.e. something ‘dangerous’ relative to usual care. Our findings challenge this concept, showing conversely that outcomes are worse outside clinical trials, even if the RCT intervention is not effective. Practitioners frequently provide new care without the safeguards typically used to protect trial participants.67 In this situation, the ‘dangerous’ experiment is usual care outside a trial. As our result has shown an overall benefit statistically, careful dissemination of this information to potential participants is required. It does

5

Nijjar et al.

Figure 4. Forest plot of the effect of participation in randomised controlled trials (RCTs) versus non-participation in women’s health.

not pose a conflict with the ethical principle of the Helsinki declaration that non-participation does not intentionally affect standard care,68 but it does offer a new view for consideration by ethicists: is it really ethical to offer care disengaged from research? Obstetric-gynaecological research poses social and ethical challenges15 including safety of the fetus, limited observation time period (e.g. gestational duration), women’s lifestyle including childcare responsibilities, sensitivity with respect to relationships, and menopause. Further, women are often excluded from clinical trials if they are of childbearing age, pregnant or breast-feeding, when they may still be treated in routine care, thus making it hard to extrapolate results to this subgroup of women who may benefit from the intervention.12 There has also been political, scientific, and public concern over the lack of representation of women in clinical research in recent decades.14,69 Thus,

6

there is a need for further research on the health benefits specifically to women participating in RCTs in other fields. It is important to remember that research can be associated with risk. The Women’s Health Initiative trial was a large RCT that demonstrated that participation in a trial may result in excess personal risk to participants and it was prematurely terminated due to safety concerns.70 It is the duty of researchers and clinicians to inform patients of the potential risk and benefit of engagement in research. The beneficial effect generated by trial participation should galvanise clinicians and women into taking part in research, increasing the value of funding and stimulating rapid accrual to women’s health RCTs.

Implications for clinical policy and practice The relative improvement in outcomes we observed within RCTs versus outside trials merits consideration.

ª 2017 Royal College of Obstetricians and Gynaecologists

Trial participation in women’s health: review

Participation in research should not be perceived as an optional extra to practice. One of the major challenges facing research is recruitment failure.71 The positive effect on health outcomes among women who participate in trials, compared with non-participants, should encourage and engage funding bodies (