JOURNAL TRANSCRIPT
Mathie and Clausen BMC Veterinary Research (2015) 11:236 DOI 10.1186/s12917-015-0542-2
RESEARCH ARTICLE
Open Access
Veterinary homeopathy: Systematic review of medical conditions studied by randomised trials controlled by other than placebo Robert T Mathie1* and Jürgen Clausen2
Abstract Background: No systematic review has previously been carried out on randomised controlled trials (RCTs) of veterinary homeopathy in which the control group was an intervention other than placebo (OTP). For eligible peer-reviewed RCTs, the objectives of this study were to assess the risk of bias (RoB) and to quantify the effect size of homeopathic intervention compared with an active comparator or with no treatment. Methods: Our systematic review approach complied fully with the PRISMA 2009 Checklist. Cochrane methods were applied to assess RoB and to derive effect size using standard meta-analysis methods. Based on a thorough and systematic literature search, the following key attributes of the published research were distinguished: individualised homeopathy (n = 1 RCT)/non-individualised homeopathy (n = 19); treatment (n = 14)/prophylaxis (n = 6); active controls (n = 18)/untreated controls (n = 2). The trials were highly diverse, representing 12 different medical conditions in 6 different species. Results: No trial had sufficiently low RoB to be judged as reliable evidence: 16 of the 20 RCTs had high RoB; the remaining four had uncertain RoB in several domains of assessment. For three trials with uncertain RoB and without overt vested interest, it was inconclusive whether homeopathy combined with conventional intervention was more or was less effective than conventional intervention alone for modulation of immune response in calves, or in the prophylaxis of cattle tick or of diarrhoea in piglets. Conclusion: Due to the poor reliability of their data, OTP-controlled trials do not currently provide useful insight into the effectiveness of homeopathy in animals. Keywords: Veterinary homeopathy, Randomised controlled trials, Systematic review
Background Our group’s systematic analysis of the published literature of randomised controlled trials (RCTs) in veterinary homeopathy identified 38 peer-reviewed papers that we regarded as potentially eligible for detailed review [1]. As emphasised in that article, no systematic review of this research evidence had ever previously been carried out. We have recently reported our review findings for 18 placebo-controlled trials of veterinary homeopathy [2, 3], in which there was indecisive evidence whether the use of homeopathy in animals is distinguishable from placebos. * Correspondence:
[email protected] 1 British Homeopathic Association, Hahnemann House, 29 Park Street West, Luton LU1 3BE, UK Full list of author information is available at the end of the article
The present paper reports our results from a similarly detailed appraisal and analysis of the remaining papers, each of which reported an RCT where the control group was an intervention other than placebo (OTP). Our approach here reflects our original literature analysis [1] and so we continue to distinguish peer-reviewed from non-peer-reviewed articles, individualised from non-individualised homeopathy, and treatment from prophylaxis. We thus report our findings from the appraisal of peer-reviewed, OTP-controlled trials of veterinary homeopathy (individualised or non-individualised, treatment or prophylaxis). Our objective firstly was to assess the study quality (risk of bias) of each eligible RCT [4], together with the direction and statistical significance of treatment or
© 2015 Mathie and Clausen. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Mathie and Clausen BMC Veterinary Research (2015) 11:236
prophylactic effect. For suitable groupings of RCTs (per species-specific medical condition; per type of control group), we aimed then to determine pooled summary statistics by meta-analysis methods and to examine their sensitivity to study quality. In both objectives, our emphasis was focused on trials that satisfied our criteria for reliable evidence [2, 3].
Methods Methods comply with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2009 Checklist (see Additional file 1) [5]. They are compatible with our previously published papers in this series of veterinary reviews [1–3], and with our protocol-based systematic reviews and meta-analyses of homeopathy RCTs in human medicine [6, 7]. Matters connected with study eligibility, research design categories and the literature search strategy were described in detail in our earlier paper [1]. Only brief descriptions are therefore given here, with additional information that is specific to the methods used for the present paper. Identifying papers for full data extraction
Each of the following electronic databases was searched up to and including March 2011 [1], with a follow-up search of the same databases up to the end of 2013: AMED, CINAHL, CENTRAL, Embase, HomVetCR, LILACS, PubMed, Science Citation Index, Scopus. There were no language restrictions. During the peer-review process for the current paper, we became aware that relevant additional literature might be contained in CAB Abstracts [8], which we therefore searched up to and including May 2015, and with the intention to update our review and analysis if significant numbers of eligible RCTs were revealed. In our original literature search [1], 20 records of trials were identified as satisfying the key acceptance criteria for the present systematic review: substantive report of clinical treatment or prophylaxis trial for any medical condition or species in veterinary homeopathic medicine, randomised, controlled by OTP, and published in a peerreviewed journal. One of us screened and categorised each of these potentially relevant papers (plus those identified in our follow-up search – see Results) to assess their eligibility for full data extraction. The other independently appraised these decisions; any differences of opinion were resolved by consensus discussion. Exclusion criteria prior to full data extraction Research using radionically prepared ‘homeopathic’
medicines. The tested intervention is homeopathy combined
with other (complementary or conventional) medicine or therapy.
Page 2 of 16
Data extraction and management
The authors of eligible RCT papers were not approached for clarification on unclear or missing facets of their methods or results [4], though original authors’ crossreference to their previously published study methods were taken into account as appropriate [2]. For each of two assessors working independently, relevant data were extracted and then recorded using a standardised data collection format (Microsoft Excel). No paper reported more than one RCT. For a paper reporting an RCT that involved >2 groups of subjects, we typically pursued data extraction on only one pair of groups: trials that included an OTP as well as a placebo control group had been scrutinised previously [2, 3] and were not reappraised for the current study. For studies that comprised >1 homeopathy group, the total sample size reflects the total numbers of subjects in the homeopathy groups combined [9]. This was the approach in all relevant cases, i.e.: where the same homeopathic medicine was used, and with the same timing of administration but in different potency; where the same homeopathic medicine and potency was used, but with different timing of administration; where a different homeopathic medicine was used. Study appraisal Assessment of risk of bias
Using the standard criteria defined by Cochrane [4], extraction of information from each paper enabled us to answer the question, ‘Is the study free from risk of bias?’: ‘Yes’; ‘Unclear’; ‘No’. The two assessors’ independent judgments were mutually scrutinised and compared, with discrepancies between them resolved by consensus discussion. This approach applied to each of seven assessment domains: I, the method used to generate the random sequence; II, the method of allocation concealment used to implement the random sequence; IIIA, the blinding of trial personnel, including animal owner as appropriate; IIIB, the blinding of outcome assessors; IV, whether all the randomised patients are accounted for in the analysis; V, whether there is evidence of selective outcome reporting; VI, whether there is evidence of other bias, such as extreme data imbalance at baseline. For domain IV, a trial was automatically regarded as no better than ‘unclear’ if there was greater than 20 % participant attrition rate, irrespective of whether intention-totreat analysis had been carried out on the data. For domain V, we based our judgment of reported outcomes on a comparison with the details given in the same paper’s Methods section (original trial protocols have not been published in veterinary homeopathy). For the purposes of the current paper, we assessed domain V as ‘No’ (high risk of bias) if the main outcome data were not extractable for metaanalysis. The source of any research funding/sponsorship,
Mathie and Clausen BMC Veterinary Research (2015) 11:236
or other vested interest such as personnel employment or contract, was not taken into account for risk-of-bias assessment (domain VI), but was reflected in the overarching assessment of risk of bias for each RCT. Assessment per trial for risk of bias
Using the Cochrane approach [4], each trial was designated overall as follows: low risk of bias for all domains; uncertain risk of bias for one or more domains, and no evident risk of bias in any domain; high risk of bias for one or more domains. A trial with overall low risk of bias comprised reliable evidence. For a trial that did not display high risk of bias in any domain, we regarded its evidence as reliable if the study was assessed as free of bias for each of domains I, IIIA, IIIB and IV [2, 3]. Outcome assessment and reporting Identification of ‘main outcome measure’ per trial
For the purposes of risk-of-bias assessment and for assessment of treatment effect, we identified for each trial a single ‘main outcome measure’ using a refinement of the approaches adopted by Linde et al. [10] and by Shang et al. [11]. The main outcome measure of each trial was based on a hierarchical ranking order (consistent with the World Health Organization’s [12] classification system for levels of functioning linked to health condition), and as previously described [2]. This approach ensured that the most clinically important outcome was selected per trial, and also avoided the problem of outcome multiplicity [13]. Unless otherwise indicated, the single end-point (as determined from the start of the intervention) associated with the designated ‘main outcome measure’ was taken as the last follow-up at which data were reported for that outcome [2]. Analysis of outcomes Summary effect measures for ‘main outcome’
For each eligible trial, the effect size was taken as the difference between the homeopathy group and the control group at the designated end-point of the trial, as follows [2]: For a dichotomous measure: odds ratio (OR), with
95 % confidence interval (CI); For a continuous measure: standardised mean
difference (SMD), with 95 % CI. If the original paper did not provide adequate information on the designated main outcome measure to enable data extraction, that trial’s outcome was classified as ‘not estimable’ and a further potentially estimable outcome was not sought. Under the separate group headings of individualised homeopathy and non-individualised homeopathy, and
Page 3 of 16
for the categories below in which there was >1 RCT of a given study design that had extractable data, we aimed to determine pooled summary statistics for: Disease-specific treatment effects per species; Disease-specific prophylactic effects per species.
All calculations and analyses were performed using Review Manager 5.2 (Cochrane). Given our anticipation of heterogeneous data for intervention effects, the random-effects (rather than fixed-effects) model was planned for all meta-analyses [14]. Reflecting study quality overall
Our main analyses used only the data extracted from trials without evident high risk of bias. It was intended that our primary conclusions would be based solely on trials with reliable evidence and without overt vested interest (not funded, directly or indirectly, by a homeopathic pharmacy). Direction of effect of treatment/prophylaxis per trial
Care was taken faithfully to represent the correct direction of change per trial: e.g. an effect favouring homeopathic intervention was higher rate of recovery but lower faecal egg count. The arithmetic content of the OR, or the sign of the SMD, was adjusted appropriately if so indicated. We regarded each RCT as a superiority trial [15] unless the original paper stated explicitly in its Methods section that the study was designed as an equivalence or non-inferiority trial [16, 17], with corresponding power calculation. For each of three study designs (and regarding each design as a superiority trial), we adopted the following rationale for interpretation of the three possible statistical findings: Statistical finding (i): P ≤ 0.05: Direction of effect toward homeopathy a. Study design: active control. Homeopathy is more effective than a conventional intervention; b. Study design: [homeopathy plus active control] versus active control (‘[A + B] versus B’). Homeopathy combined with conventional intervention is more effective than conventional intervention alone; c. Study design: no-treatment control. Homeopathy is more effective than no intervention. Statistical finding (ii): P ≤ 0.05: Direction of effect toward control a. Study design: active control. Homeopathy is less effective than a conventional intervention; b. Study design: [homeopathy plus active control] versus active control (‘[A + B] versus B’). Homeopathy
Mathie and Clausen BMC Veterinary Research (2015) 11:236
combined with conventional intervention is less effective than conventional intervention alone; c. Study design: no-treatment control. Homeopathy is ineffective. Statistical finding (iii): P > 0.05: Direction of effect toward either homeopathy or control a. Study design: active control. Inconclusive whether homeopathy is more or is less effective than a conventional intervention [18, 19]; b. Study design: [homeopathy plus active control] versus active control (‘[A + B] versus B’). Inconclusive whether homeopathy combined with conventional intervention is more or is less effective than conventional intervention alone; c. Study design: no-treatment control. Inconclusive whether homeopathy is more or less effective than no treatment.
Results Demographic details
Additional file 2 illustrates the PRISMA flowchart, which follows on from, and suitably updates [20], that in our earlier paper [1]; our current focus is on OTPcontrolled trials only. Two OTP-controlled trials were rejected from the current review due to their focus on homeopathy combined with another intervention: A29, Dreissman 2010; A30, Lepple 1984. A further two were excluded due to their focus on neither treatment nor prophylaxis: A33, Sharma 1987; A37, Trehan 1994. Four additional eligible records were identified in our follow-up search, and included in analysis. Our subsequent search of CAB Abstracts, conducted during peer review of the current paper, revealed just one potentially eligible trial [21], which we did not consider further. Table 1 details each of the 20 studies eligible for full systematic review: (i) individualised homeopathy/treatment (n = 1); (ii) non-individualised homeopathy/treatment (n = 13); (iii) non-individualised homeopathy/ prophylaxis (n = 6). Data per trial include: nature of the homeopathic intervention; study setting; the RCT’s source of funding. Our definitions of ‘treatment’ and ‘prophylaxis’ were as previously described [1]. All RCTs were identified as superiority trials, despite the fact that in some cases the original authors applied the terms ‘equivalence trial’ or ‘non-inferiority trial’ post-hoc in Results or in Discussion (A15, Faulstich 2006; A39, Braun, 2011). Extreme diversity characterised the studies as regards species, medical condition, homeopathic medicine, and funding source. In the 20 eligible studies, 6 different species are represented: birds (n = 1); cattle (n = 11);
Page 4 of 16
dogs (n = 1); horses (n = 1); pigs (n = 4); sheep (n = 2). Twelve different conditions are represented. None of the trials had a clearly unbiased funding source; we identified overt vested interest in 10 of the 20 trials. Table 2 includes: sample sizes; designated ‘main outcome measure’; type of data (whether dichotomous or continuous); study endpoint. Diversity was again apparent, with large variation in sample sizes, main outcome, and timing of the study endpoint. Risk of bias
Table 3 shows our risk-of-bias judgments for each eligible trial. Many of the papers were written to such a poor standard that risk-of-bias assessments were sometimes challenging. No trial had low risk of bias in all of the Cochrane judgmental domains. Sixteen trials had high risk of bias in one or more domains. Four trials had merely uncertain risk of bias, and the uncertainty was evident in five domains per study (A31, Reis 2006; A32, Reis 2008; A34, Signoretti 2008; A36, Soto 2008). There was therefore no trial that could be designated as reliable evidence. Only five of the 20 trials had adequate randomisation (domain I), and none of the trials had adequate allocation concealment (domain II) or personnel blinding (domain IIIA). See also Fig. 1 (Risk-of-bias summary graph). Analysis of outcomes
Fifteen of the 20 trials had extractable data. Due to the diversity of medical conditions, species, types of homeopathic intervention, study designs and outcome measures, as well as the very unclear quality of the evidence, it was not appropriate to carry out meta-analysis on disease-specific intervention. The direction of effect favoured homeopathy in 11 trials (statistically significantly so in four cases) and favoured control in four trials (statistically significantly so in zero cases) – see Table 4, in which the inference from the statistical findings is given by reference to the numbered study designs in Methods: Direction of effect of treatment/prophylaxis per trial. Trials with uncertain risk of bias
In the absence of any trials with reliable evidence, our main attention has been forced on to the data extracted from three trials with uncertain risk of bias and without overt vested interest (Table 4a): A32, Reis 2008 (modulation of immune response to rabies vaccination in calves); A34, Signoretti 2008 (cattle tick); A36, Soto 2008 (diarrhoea in piglets). All three trials were in the category ‘non-individualised/prophylaxis/‘[A + B] versus B’ design. None of the three showed evidence of a difference between adjunctive homeopathy and active control alone,
i: Individualised/Treatment Condition
Species Ref. First author
Year Design
Control
Homeopathic medicine
Dilution*
Study setting
Mastitis, metritis and agalactia
Pigs
1988 Active
Antibiotic
Individualised
D3-D12
University Remedies were gift veterinary of the manufacturer ambulance; 21 swine herds in Germany (Berlin area)
No
Homeopathic medicine
Dilution
Study setting
Funding
Free from vested interest
A12 Schütte
Funding
Free from vested interest
ii: Non-individualised/Treatment Species Ref. First author
Year Design
Control
Diarrhoea (neonatal)
Cattle
A42 Lohr
2012 Active
Oral electrolytes 3 complex preparations: Nux vomica; Veratrum; Engystol
Not stated
Six farms in Germany
Directly funded study (Heel Pharmacy)
No
Diarrhoea (neonatal)
Pigs
A14 Coelho
2009 Active
Antibiotic
Phosphorus/E. coli
30C
Swine farm, Brazil
The company, Farmácia Sensitiva, 'supplied' the medicines
No
Ectoparasite infestation
Cattle
A41 Catto
2013 Active
Anthelmintic
2 complex preparations of various components (mainly nosodes)
All but one components potentised to 12C and higher (30C, 200C)
Cattle farm, Brazil
None stated
Unclear
Ectoparasite infestation
Cattle
A19 Silva
2008 Active
Chemical dip
Biotherapic' of Boophilus microplus and 14 other organisms
12C
Government research institution, Brazil
None declared'. The homeopathic laboratories, Flora & Fauna Ltd., supplied the medicine
No
Foot-and-mouth disease
Cattle
A40 Lotfollahzadeh 2012 Active
Anti-inflamatory Tarentula cubensis and antibiotic
D5
Cattle farms, Iran
Richter Pharma donated the homeopathic medicine
No
Gastrointestinal nematodes
Sheep
A21 Zacharias
2008 Active
Anthelmintic
3 remedies: Ferrum phosphoricum, Arsenicum album, Calcarea carbonica
Not stated
Government research institution, Brazil
None stated
Unclear
Lameness
Horses
A15 Faulstich
2006 Active
Hyaluronic acid
Complex of 14 homeopathically prepared ingredients
D3 - D8
Two horse clinics in Germany (Berlin area, Munich area)
Study performed by a contract research organisation
No
Mastitis
Cattle
A16 Klocke
2010 Active
Teat-sealer
8 remedies: Mercurius solubilis, D6 Lachesis mutus, Sulfur, Calcium carbonicum, Calcium phosphoricum, Pulsatilla pratensis, Sepia, Silica
Thirteen organic dairy herds, Switzerland
EC grant; Weleda 'provided the remedies'
No
Page 5 of 16
Condition
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 1 Demographic details of 20 OTP-controlled RCTs in veterinary homeopathy
Mastitis
Cattle
A20 Varshney
2005 Active
Antibiotic
Complex of 8 remedies: Healwell VT-6 (Sintex International Limited, Kalol, India), consisting of Phytolacca, Calcarea fluorica, Silica, Belladona, Bryonia, Arnica, Conium, Ipecacuanha "in equal amount"
200C: Phytolacca, Calcarea fluorica. 30c: Silica, Belladona, Bryonia, Arnica, Conium, Ipecacuanha
Dairy farm, India
Sintex International Ltd 'supplied the homeopathic formulation for clinical trial'
Mastitis, metritis and agalactia
Pigs
A39 Braun
2011 Active
Antibiotic
2 complex preparations. Lachesis compositum + Traumeel
Not stated
Three pigletrearing farms in Germany
Directly funded study No (Heel Pharmacy)
Pseudopregnancy Dogs
A13 Beceriklisoy
2008 Active
Naloxone
Thuja occidentalis/Urtica urens
D30
University animal clinic, Turkey
None stated
Unclear
Salmonellosis
Birds
A18 Sandoval
1998 Active
Antibiotic
Baptisia tinctoria
30C
Poultry farm, Mexico
None stated
Unclear
Gastrointestinal nematodes
Sheep
A17 Rocha
2006 (A + B) vs. B Anthelmintic only
Fator Vermes
Not stated
University establishment, Brazil
None stated
Unclear
Homeopathic medicine
Dilution
Study setting
Funding
Free from vested interest
No
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 1 Demographic details of 20 OTP-controlled RCTs in veterinary homeopathy (Continued)
iii: Non-individualised/Prophylaxis Species Ref. First author
Year Design
Control
Ectoparasite i nfestation
Cattle
A34 Signoretti
2008 (A + B) vs. B Protein supplement only
Factor C & MC ® : 16 remedies
All 12C
Government research institution, Brazil
None stated
Unclear
Diarrhoea (neonatal)
Pigs
A36 Soto
2008 (A + B) vs. B Sucrose saline only
Complex of 4 remedies: Echinacea angustifolia, Avena sativa, Ignatia amara, and Calcarea carbonica
6C
Commercial swine herd, Brazil
None stated
Unclear
Handling stress
Cattle
A31 Reis
2006 (A + B) vs. B Mineral salt only
Matricaria chamomilla®
12C (plus non potentised sugar and Bixa orellana [grams])
Cattle farm, Brazil
"This research was supported by the Homeopathic Laboratory Arenales Flora & Fauna Ltd."
No
Immune response to rabies vaccination
Cattle
A32 Reis
2008 (A + B) vs. B Mineral salt only
Matricaria chamomilla®
12C
Cattle farm, Brazil
The homeopathic laboratories, Flora & Fauna Ltd., are thanked by the author non-specifically
Unclear
Infertility
Cattle
A35 Sommer
1972 Un-treated
Complex of 5 remedies
D3-D5
Two cattle farms, University of Hohenheim (Germany)
None stated
Unclear
Untreated
Page 6 of 16
Condition
Infertility
Cattle
A38 Williamson
1991 Un-treated
Untreated
Sepia
200C
Dairy herd, Scotland
British Cattle Veterinary Association. The pharmacy, Ainsworth's, are thanked by the authors nonspecifically
Unclear
*Note on homeopathic dilutions: The number refers to the number of successive serial dilutions to which the starting material has been subjected. The letter refers to the scale on which the dilution has been carried out: the letter D denotes the decimal method of dilution (that is, one part of liquid is added to nine parts of purified water, ethanol, glycerol or lactose); the letter C indicates the centesimal method (one part added to 99 parts of diluent). In homeopathic dilutions above 12C/D24 (10−24 molar) – beyond Avogadro’s constant, 6.02 x 1023 mol−1 – there are, in theory, no material traces of the original substance; such dilutions are known as ‘ultra-molecular’
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 1 Demographic details of 20 OTP-controlled RCTs in veterinary homeopathy (Continued)
Page 7 of 16
i: Individualised/Treatment Ref. First author
Year N start N start N start N end N end N end % Designated 'main (hom.) (cont.) (tot.) (hom.) (cont.) (tot.) Attrition outcome'
Type of Direction of Problem with data change favouring designated 'main homeopathy outcome measure'
End-point
Notes
A12 Schütte
1988 33
Dichot.
28 d
N = No. of sows treated
End-point
Notes
31
64
33
31
64
0.0
Piglet mortality
Lower
Yes: Gives percentage of dead piglets (events) but not number of piglets born (total)
ii: Non-individualised/Treatment Ref. First author
Year N start N start N start N end N end N end % Designated 'main (hom.) (cont.) (tot.) (hom.) (cont.) (tot.) Attrition outcome'
Type of Direction of Problem with data change favouring designated 'main homeopathy outcome measure'
A42 Lohr
2013 53
56
109
46
54
100
8.3
Cure rate (Sumscore (physical condition))
Dichot.
Higher
Only 46 (Hom) and Up to 4 d 54 (Cont) animals regularly completed the study, and are therefore the data for our calculations. The authors themselves calculated cure rate based on 50 and 56 animals respectively.
A14 Coelho
2009 35
11
46
35
9
44
4.3
Proportion of animals without diarrhoea at the end of treatment
Dichot.
Higher
No
12 d
A41 Catto
2013 72
36
108
72
36
108
0.0
Faecal egg count
Contin.
Lower
Yes: Unclear sample sizes – see Notes
Second year
A19 Silva
2008 9
9
18
?9
?9
?18
Unclear
Mean number of engorged ticks per annum
Contin.
Lower
Yes: SD not stated; N at endpoint not stated
1 yr
23
75
50
15
65
13.3
Rectal temperature
Contin.
Lower
A40 Lotfollahzadeh 2012 52
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 2 Sample sizes and outcomes for 20 OTP-controlled RCTs
The numbers of animals in each group based on the assumption that each group comprised 3 paddocks (12 animals in each). No information on dropouts.
14 d Page 8 of 16
A21 Zacharias
2008 7
7
14
?7
?7
?14
Unclear
Faecal egg count
Contin.
Lower
No. Numbers for analysis interpolated from Figure 1A. N not stated for endpoint, but assumed to be N = 7.
68 d
Disparity between data (mean eggs per gram) for conv med group given in text (1,483) and by inspection of Fig 1 (1,300 approx.)
A15 Faulstich
2006 24
22
46
22
19
41
10.9
Treatment success Dichot. (overall effectiveness)
Higher
No : Numbers for analysis from percentage data in Table 9
21 d
Control really active?
A16 Klocke
2010 32
36
68
32
36
68
0.0
Absence of clinical mastitis infection
Dichot.
Higher
No: Data derived from text of Results [line 6])
First 100 days post-calving
A20 Varshney
2005 67
96
163
67
96
163
0.0
Quarter cure-rate (non-fibrosed)
Dichot.
Higher
No : Numbers for analysis from percentage data in Table 2
Up to 28 d
A39 Braun
2011 30
34
64
28
32
60
6.3
Cure rate (MMA-sum score)
Dichot.
Higher
A13 Beceriklisoy
2008 30
8
38
30
8
38
0.0
Treatment 'success' (recovery rate)
Dichot.
Higher
No
Up to 20 d
A18 Sandoval
1998 200
200
400
200
200
400
0.0
Cumulative total mortality (1st quality broilers)
Contin.
Lower
Yes: Data are presented as accumulated mortality – not amenable to meta-analysis
49 d
A17 Rocha
2006 10
10
20
10
10
20
0.0
Proportion of animals Dichot. not requiring antihelminthic treatment
Higher
No: Data derived from text of Results (second paragraph)
4 mo (first phase)
N = quarters, not animals
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 2 Sample sizes and outcomes for 20 OTP-controlled RCTs (Continued)
Up to 4 d (H) or 3 d (C)
N = 1st quality broilers
iii: Non-individualised/Prophylaxis Year N start N start N start N end N end N end % Designated 'main (hom.) (cont.) (tot.) (hom.) (cont.) (tot.) Attrition outcome'
Type of Direction of Problem with data change favouring designated 'main homeopathy outcome measure'
End-point
A34 Signoretti
2008 8
8
16
?8
?8
?16
Unclear
Faecal egg count per gram
Contin.
Lower
No. But assumes zero attrition rate. Data from Table 3.
30 d
A36 Soto
2008 24
24
48
24
24
48
0.0
Number of animal-days with diarrhoea
Dichot.
Lower
No. But relate 7-day cumulated data to total possible number of animal-days per group
7d period
Notes
Hom plus sucrose saline vs. sucrose saline only
Page 9 of 16
Ref. First author
A31 Reis
2006 30
30
60
30
30
60
0.0
Serum cortisol
Contin.
Lower
No. Assumes zero attrition rate based on legend for Figure 1.
60 d
Hom plus mineral salt vs. mineral salt only
A32 Reis
2008 15
15
30
15
?15
?30
Unclear
Rabies-neutralizing antibody titer
Contin.
Higher
No. But assumes zero attrition rate.
60 d after vaccination
Compares Group FEV2 (Hom + salt) with Group V2 (salt only)
A35 Sommer
1972 40
18
58
40
18
58
0.0
Number with infertility disorders
Dichot.
Lower
No. Data from table 2
Not stated
A38 Williamson
1991 101
32
133
?100
?32
?132
Unclear
Number with peripartum disorders
Dichot.
Lower
Yes: Unclear tabulations
None
hom. = homeopathy. cont. = control. tot = total. dichot. = dichotomous. contin. = continuous
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 2 Sample sizes and outcomes for 20 OTP-controlled RCTs (Continued)
Page 10 of 16
i: Individualised/Treatment Ref. First author
Domain I Domain II Domain IIIA Domain IIIB Domain IV Domain V Domain VI (excl. fund) No. of domains for which Risk of bias Reliable evidence criteria fulfilled (excl. vested interest)
A12 Schütte
N
N
N
N
Y
N
U
Y
U
N
1
1
5
High
No
ii: Non-individualised/Treatment Ref. First author
Domain I Domain II Domain IIIA Domain IIIB Domain IV Domain V Domain VI (excl. fund) No. of domains for which Risk of bias Reliable evidence criteria fulfilled (excl. vested interest)
A42 Lohr
Y
U
N
Y
Y
N
N
Y
U
N
3
1
3
High
No
A14 Coelho
U
U
N
U
Y
Y
N
2
3
2
High
No
A41 Catto
U
U
N
U
U
N
U
0
5
2
High
No
A19 Silva
U
U
N
U
U
N
U
0
5
2
High
No
A40 Lotfollahzadeh U
U
N
Y
U
N
Y
2
3
2
High
No
A21 Zacharias
U
U
N
U
U
Y
U
1
5
1
High
No
A15 Faulstich
Y
U
N
Y
Y
Y
Y
5
1
1
High
No
A16 Klocke
Y
U
N
U
Y
Y
Y
4
2
1
High
No
A20 Varshney
U
U
U
U
U
Y
N
1
5
1
High
No
A39 Braun
Y
U
N
Y
Y
Y
N
4
1
2
High
No
A13 Beceriklisoy
U
U
U
U
U
Y
N
1
5
1
High
No
A18 Sandoval
U
U
U
U
U
N
U
0
6
1
High
No
A17 Rocha
N
N
U
U
Y
Y
U
2
3
2
High
No
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 3 Risk-of-bias assessments for 20 OTP-controlled RCTs
iii: Non-individualised/Prophylaxis Ref. First author
Domain I Domain II Domain IIIA Domain IIIB Domain IV Domain V Domain VI (excl. fund) No. of domains for which Risk of bias Reliable evidence criteria fulfilled (excl. vested interest) Y
U
N
U
U
U
U
U
Y
Y
2
5
0
Uncertain
No
A36 Soto
U
U
U
U
Y
Y
U
2
5
0
Uncertain
No
A31 Reis 2006
U
U
U
U
U
Y
Y
2
5
0
Uncertain
No
A32 Reis 2008
U
U
U
U
U
Y
Y
2
5
0
Uncertain
No
A35 Sommer
U
U
N
U
Y
Y
U
2
4
1
High
No
A38 Williamson
U
U
N
U
U
N
U
0
5
2
High
No
Domains I-VI are explained in the main text. Criteria fulfilled for domains: Y = Yes; U = Unclear; N = No
Page 11 of 16
A34 Signoretti
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Page 12 of 16
Domain I Domain II Domain IIIA Domain IIIB Domain IV Domain V Domain VI 0%
25% Low risk of bias
50% Unclear risk of bias
75%
100%
High risk of bias
Fig. 1 Risk-of-bias summary graph. Illustrates, for each assessment domain, the proportion of RCTs with low, unclear and high risk of bias
and so in each case it was statistically inconclusive whether homeopathy combined with conventional intervention was more or was less effective than conventional intervention alone – see Table 4a. Trials at high risk of bias
Data could not be extracted from the single paper on individualised treatment. Data were extractable from 10 of the 13 relevant RCTs of non-individualised treatment; the direction of effect statistically significantly favoured homeopathy in four of the ten cases – see Table 4b. Two of these four yielded what appear to be spuriously high mean OR values (A13, Bekeriklisoy 2008: OR = 95.86; A14, Coelho 2009: OR = 58.09). Of the two RCTs of non-individualised prophylaxis (both with untreated controls), data were extractable from one (A35, Sommer 1972).
Discussion This comprehensive systematic review of the peerreviewed literature has revealed no OTP-controlled RCT of veterinary homeopathy that comprised reliable evidence. Our main conclusions from this study are therefore restricted to the findings of three RCTs, whose quality was unclear and whose methodological diversity contraindicated meta-analysis. Each of these three trials is in the category of non-individualised homeopathic prophylaxis (A32, Reis 2008; A34, Signoretti 2008; A36, Soto 2008), a prescribing approach that some homeopathic practitioners might view as controversial [22]. Each of the three trials is also in the adjunctive category, and manifested statistically inconclusive findings: this result undermines the suggestion that the ‘[A + B] versus B’ study design can ‘generate only positive results’ [23]. For each of the above three trials, the uncertainty regarding risk of bias was so marked that their findings
must be viewed with extreme caution. Given the lack of clarity in so many domains of assessment, this degree of concern would still apply even if we had reflected more leniently the fact that non-blinding of study personnel is a typical feature of an OTP-controlled trial. The admissible evidence in veterinary homeopathy for this study design is therefore unable to provide any compelling information about modulation of immune response in calves (A32, Reis 2008) or adjunctive prophylaxis of cattle tick (A34, Signoretti 2008) or of diarrhoea in piglets (A36, Soto 2008). In assessing all the studies, we had reason on occasions to question the veracity of the control group as an ‘active’ or effective intervention (e.g. mineral or protein supplements, or naloxone in pseudopregnancy), and so raised doubts about the ‘other than placebo’ nature of some of the included trials. We were careful not to attribute ‘equivalence’ to superiority trials that found no statistically significant difference between homeopathy and active control [16, 19]. For OTP-controlled RCTs of non-individualised homeopathic treatment, there is nothing that we can reasonably conclude, given that each one of the trials was judged to be at high risk of bias. Thus, the positive inference we attributed to four trials in this category cannot be regarded as admissible evidence; the spuriously large treatment effect evidenced in two of the trials reinforces those serious concerns. It remains unknown whether non-individualised homeopathy is effective in the prophylaxis of fertility disorders in cattle. Likewise, no conclusions can be made regarding individualised homeopathic treatment, since the single trial in this category was of very low quality and yielded no extractable data for analysis. Our most recent literature search, conducted during the process of the current paper’s peer-review, revealed
a) UNCERTAIN RISK OF BIAS iii: Non-individualised/Prophylaxis Ref. First author
Year Condition
Species Outcome measure
Hom.
A34 Signoretti
2008 Ectoparasite infestation
Cattle
Faecal egg count per gram
A36 Soto
2008 Diarrhoea (neonatal) Pigs
A31 Reis
2006 Handling stress
A32 Reis
2008 Immune response Cattle to rabies vaccination
Cattle
Cont.
Summary Effect size effect measure (95 % CI)
Direction of Direction P change favouring of effect value homeopathy
Inference*
340 (sd, 214); 207 (sd, 131); N=8 N=8
SMD
0.71 [−0.31, 1.73]
Lower
Cont.
0.17
[iii: b]
Number of animal-days with diarrhoea
34/192
25/192
OR
0.70 [0.40, 1.22]
Lower
Cont.
0.20
[iii: b]
Serum cortisol
5.1 (sd, 1.8); N = 30
5.3 (sd, 1.45); N = 30
SMD
−0.12 [−0.63, 0.39]
Lower
Hom.
0.64
[iii: b]
Rabies-neutralizing antibody titer
10.8 (sd, 9.5); N = 15
14.4 (sd, 11.1); N = 15
SMD
−0.34 [−1.06, 0.38]
Higher
Cont.
0.36
[iii: b]
b) HIGH RISK OF BIAS i: Individualised/Treatment Ref. First author
Year Condition
Species Outcome measure
Hom.
Cont.
Summary Effect size effect measure (95 % CI)
Direction of Direction P change favouring of effect value homeopathy
Inferencea
A12 Schütte
1988 Mastitis, metritis and agalactia
Pigs
X
X
X
X
X
Hom.
Cont.
Summary Effect size effect measure (95 % CI)
Direction of Direction P change favouring of effect value homeopathy
Inference*
None useable
X
X
X
Mathie and Clausen BMC Veterinary Research (2015) 11:236
Table 4 Summary statistics for: (a) trials at uncertain risk of bias; (b) trials at high risk of bias
ii: Non-individualised/Treatment Year Condition
Species Outcome measure
A42 Lohr
2013 Diarrhoea (neonatal)
Cattle
Cure rate (Sum-score 37/50 [physical condition])
42/56
OR
0.95 [0.40, 2.28]
Higher
Cont.
0.91
[iii: a]
A14 Coelho
2009 Diarrhoea (neonatal)
Pigs
Proportion of animals 35/35 without diarrhoea at the end of treatment
5/9
OR
58.09 [2.73, 1234.82]
Higher
Hom.
0.009
[i: a]
A41 Catto
2013 Ectoparasite infestation
Cattle
None useable
X
X
X
X
X
X
X
X
A19 Silva
2008 Ectoparasite infestation
Cattle
None useable
X
X
X
X
X
X
X
X
A40 Lotfollahzadeh 2012 Foot-and-mouth disease
Cattle
Rectal temperature
38.6 (sd, 0.2); N = 50
39.0 (sd, 0.1); N = 15
SMD
−2.16 [−2.86, −1.47]
Lower
Hom.