Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses

Authors: Cook D, Reeve B, Guyatt G, Heyland D, Griffith L, Buckingham L, Tryba M.
Source: JAMA. 275:308-14. January 24/31, 1996.
Institutions: McMaster University, Ontario; University Hospital, Bochum, Germany.
Financial support: None reported.

Summary

Background

In 1991, Cook et al reported two meta-analyses on the utility of various stress ulcer prophylactic regimens in critically ill patients. That same year, Tryba reported another meta-analysis of similar drugs. These two groups agreed in some aspects of their studies, but disagreed in others. In order to try to resolve the discrepancies between their findings, and to update their studies, the authors pooled resources and performed a new meta-analysis, the results of which are detailed here.

Methods

Results Authors' Discussion

The authors discuss the main reasons their initial meta-analyses reached some different conclusions. The main reasons for differences between these studies are: different agents included (Tryba included pirenzipine and prostaglandins, Cook did not); some different studies included in the meta-analyses; different definitions of bleeding; different analytic approaches when performing the meta-analyses.

The authors then give a brief summary of how their initial studies agreed and disagreed, and compared them to the current study.

Comment

It is encouraging to see two independent groups (one from Canada, one from Germany) collaborate on a mega-meta-analysis, to try to resolve conflicts between their previous studies.

When comparing the results of the prior meta-analyses, in some cases the results were the same; in some cases a statistically significant result in one study was a trend that did not reach significance in the other; in some cases a significant result in one study was a trend in the opposite direction in the other study. In no cases, however, were diametrically opposite statistically significant results reported. This is somewhat reassuring, since meta-analyses should, ideally, resolve conflicts in the literature, not create new ones.

Although meta-analyses are very well suited to "transforming" trends in a number of studies into a statistically significant result, I'm not sure that a trend in a meta-analysis itself is any more meaningful than a trend in an individual study. In this paper, there is much mentioning of trends, and even a rigorously defined distinction between trends and strong trends. I believe there is a risk that a trend in a meta-analysis will be accorded more weight than a similar trend in an individual study, which is not warranted. A trend is not statistically significant, whether in a trial or in a meta-analysis. I did not include any of the trends reported in this paper in my summary.

2/6/96


Reader comments

Date: Sat, 10 Feb 96
From: "J.M.Pontious M.D." <f779@REX.RE.uokhsc.edu>
Organization: OU/Enid Family Medicine
Subject: Trend Comment

This area of literature review is quite confusing to me (your comment about trending not being significant and should have no weight in either multi-study analysis or in the individual study).

For those of us who are not as steeped in statistical analysis, could you expand this comment/concept? Intuitively, it would seem that if several studies showed trends that this should have a greater weight in trusting the data to be something other than chance...Is the point that you are making that a tendency that does not reach significance is not to be trusted?

J.M.Pontious M.D.
OU/Enid Family Medicine



From: jleemd@umich.edu
Date: Fri, 22 Mar 1996
Subject: meta-analysis of Stress Ulcer Prophylaxis

I think it is critical to point out that meta-analyses are nothing more than statistical "toys". For example, consider the recent meta-analysis by Furberg et al, Circulation. The same issue contains a rebuttal meta-analysis of the same studies, but with a non-significant result. This was accomplished by a minor adjustment of the data used for the meta-analysis.

Clinicians should pay more heed to well-performed studies that have significant results. If two such studies exist that conflict, then one should pick the study that applies most to one's own practice and experience. Meta-analyses, once the studies to be used have been chosen, do not give any more weight to the better studies, beyond that of number of patients studied. There are a lot of mediocre studies that receive equal weight with landmark studies.

--Swamy@hosp.med.umich.edu



Subject: Metaanalysis and stress ulceration
Date: Mon, 22 Apr 1996
From: Peter Ellis <pellis@extro.ucc.su.oz.au>

I would like to make an additional comment regarding metaanalysis. Metaanalysis, like any randomised controlled trial, can be subject to bias if it is not well performed. It is therefore just as important to critically appraise an article on metaanalysis as any other published research.

Several authors have published guidelines for appraisal of overviews and metaanalysis.Two that I find useful are 'Checklists for review articles. Oxman A. BMJ 1994:309;648-651' and 'Users guide to the medical literature VI: How to use an overview. Oxman A, Cook DJ and Guyatt GH. JAMA 1994:272(17);1367-1371'.

The results of a metaanalysis should not routinely be accepted just as the results of an RCT should not be. Rather they should be critically appraised. Other articles in the JAMA series, 'Users guide to the medical literature may also prove useful for this purpose. Alternatively this site can be located at

http://hiru.hirunet.mcmaster.ca/ebm/userguid/default.htm [note: this site is no longer active; try: http://www.cche.net/usersguides/main.asp -- mj]

Peter Ellis MBBS FRACP
Dept Cancer Medicine, Sydney University, Australia


Journal Club on the Web home page

Submit a comment about this article



Site Meter