Herbal Evidence - Combining evidence from clinical trials of herbal medications

Herbal Evidence was created by a biostatistician and epidemiologist who were frustrated with the lack of access the general public had to the results from clinical trials, particularly from trials investigating alternative treatments. We believe this has led to unnecessary confusion regarding the efficacy of some of these treatments, leaving people unable to make informed decisions about the usefulness of taking particular treatments for conditions.

In addition, apparently conflicting results may be observed where more than one clinical trial has been conducted for a treatment and condition. Systematic reviews and meta-analyses are vital for compiling and summarising all the available evidence, and while these are occasionally conducted by academic researchers, they are not published frequently enough to capture all incoming evidence. Herbal Evidence provides a living meta-analysis, regularly updated to incorporate the latest published clinical trials. We believe that making the up-to-date findings from clinical trials available to all empowers consumers, researchers and medical practitioners.

Clinical trials are scientific experiments used to evaluate the safety and efficacy of treatments in humans. They are used by organisations, hospitals, pharmaceutical companies and academic institutions. Randomised controlled trials are considered the gold standard for assessing the effects of an intervention and are conducted to assess medications, medical devices, procedures and policies.

Here trial participants are randomly assigned to one of the treatment arms; an active treatment or a control (often a placebo). This random assignment helps to reduce confounding, a common feature in observational studies. For example, suppose an epidemiological study found that people who took a particular herbal supplement had fewer annual occurrences of the common cold compared with people who did not take that herb. However, people who take the supplement may be less likely to smoke, may get more sleep or may experience less stress; all of which may also contribute to having fewer colds. These characteristics also associated with the outcome are called confounders.

A randomised controlled trial can balance the treatment arms with respect to these and other characteristics about the participants so that the two groups look similar at the start of the trial, before receiving the treatments. That is, the two groups would be similar with regards to age distribution, proportion of smokers, proportion of females and so on. Therefore, any differences in the results between the two treatment arms will be more likely to be due to the actual treatment and not due to the potential confounders.

Many clinical trials, particularly investigating herbal medications, recruit small numbers of patients and lead to inconclusive results. Subsequently, multiple independent investigations may be conducted into the effectiveness of a treatment for a condition. In order to reach appropriate conclusions from all of the relevant available trials, a statistical analysis can be conducted to pool the evidence across the clinical trials and provide an overall assessment of the treatment effect. These meta-analyses are based on systematic reviews and play a key role in evidence-based medicine. The results of meta-analyses may be graphically displayed in forest plots.

Systematic reviews are literature reviews that aim to identify, appraise and synthesise the findings from all studies that address a specific research question and meet prespecified eligibility criteria. Results from each study may be extracted for use in a meta-analysis.

Herbal Evidence conducts systematic reviews for each herb or compound included on the site. For details on the search terms and databases used for a particular herb please contact info@herbalevidence.com.

Forest plots are graphical representations of the results from a meta-analysis. Herbal Evidence extracts binary outcomes from studies found in the systematic reviews, for example the numbers of patients in each treatment arm who did and did not have a successful event. Consequently, the individual study and pooled results are displayed as odds ratios.

Here an odds ratio (OR) represents the ratio of the odds of a successful event in the active treatment arm to the odds of a successful event in the control arm, where the odds of an event is the probability of the event happening divided by the probability of the event not happening. As these are all only estimates of the true OR for a treatment there is some uncertainty around the values and these are therefore presented alongside a range of values for these estimates that have a 95% probability of containing the true OR; 95% confidence intervals. If a 95% confidence interval includes the value 1.0 then that indicates that there was no evidence that the active treatment performed better than placebo in producing a successful event. In the forest plot the OR and 95% confidence interval for each study are represented by a box and confidence interval whisker. If a whisker crosses the line of no effect (null) then that also indicates there is no evidence for a difference between the treatment arms.

The below annotated forest plot is an example of a meta-analysis for the effect of ginseng (the active treatment) versus placebo (the control) on successfully treating or preventing the common cold. The analysis includes five clinical trials, of which the first and fourth studies found no evidence that ginseng worked better than placebo. However, the results from pooling all the studies provide some evidence that ginseng may perform better than placebo (95% confidence interval for the Overall Effect excludes 1.0, the blue diamond does not cross the null line and is on the right-hand side of the plot).

Systematic reviews are used to identify randomised controlled trials for each herbal medication included on the site. For details on the search terms and databases used for a particular herb please contact info@herbalevidence.com.

All studies with at least 25 evaluable patients, at least an English language abstract and results provided for a binary study outcome are included in the database. Some publications reporting quantitative outcomes did not additionally provide information on clinically meaningful changes from baseline and the number of patients who achieved those criteria. In those cases, their findings were not included in our analyses.

Herbal Evidence aims to include all relevant clinical trials irrespective of the study results. If you believe your study met the inclusion criteria and has not been included in our summaries, please contact info@herbalevidence.com.