Intermittent etidronate therapy to prevent corticosteroid-induced osteoporosis
CommentThis trial investigated the use of a bisphosphonate, etidronate, in preventing osteoporosis among patients receiving long-term glucocorticoid treatment. The cyclic administration of etidronate for one year was associated with a significant improvement in vertebral and femoral trochanter bone mass. It was also associated with a significant decrease in the number of patients with new vertebral fractures, but only in the subgroup of post-menopausal women (from 22% to 3%).
This paper illustrates three topics that are recurring themes: the reporting of subgroup analysis, the reporting of results that do not reach statistical significance and the extrapolation of results to a class of drugs.
Subgroup analysis is a problem because the chance of finding a spurious effect increases with the number of subgroups that are analyzed a posteriori. Strictly speaking, the statistical significance of a result only applies to endpoints that are specified a priori. Of course, if a particular subgroup analysis yields a positive result that makes perfect sense from a pathophysiologic point of view, it is much more likely to be real than if no such explanation can be found. Nevertheless, subgroup analysis must be viewed with caution.
In this paper, the reduction in lumbar vertebral fracture rate was only significant in the subgroup of postmenopausal women. The significance in this subgroup was between p=0.03 and p=0.05. Since it certainly makes intuitive and pathophysiological sense that those patients at highest risk for fractures would be most likely to benefit, I believe that the results in this subgroup are likely to be valid. Nevertheless, it must be kept in mind that this was not a primary endpoint.
The problem of the reporting of results that are not "statistically significant" has been mentioned elsewhere in Journal Club (including a review of a paper on alendronate). I do not believe that one must ignore and not report all results which fail to strictly fit the criteria of p<0.05. I do believe, however, that such results should only be considered meaningful if they make sense in a wider context, if the p-value is reasonable and if the failure to meet the accepted test of significance is made very clear.
In this paper, a result that does not reach statistical significance (the reduction in vertebral fractures in the whole group) is reported. The authors state that "the relative risk of fractures in the etidronate group as compared with the placebo group was 0.6 (95% confidence interval 0.2 to 1.6)" and in the editorial, it is stated that "the proportion of patients with new vertebral fractures was halved in the etidronate group". Is this (statistically not significant) result plausible and meaningful, and should it have been reported this way? I do not believe so. Among men there were 4/19 patients with new fractures in the etidronate group and 3/25 in the placebo group. Among premenopausal women, there were no new fractures in either group. The decrease in vertebral fractures seen with etidronate was entirely the result of the decrease found among postmenopausal women, and should not be claimed for the group as a whole.
Finally, what about the extrapolation of the results using etidronate to the bisphosphonates as a whole? Etidronate has not been approved in the United States for the treatment of osteoporosis, although it has been used for this indication prior to the introduction of alendronate and is used for it in other countries. It is significantly cheaper when used cyclically than alendronate, but has one potential drawback: it inhibits bone mineralization as well as bone resorption and may cause osteomalacia. This is the rationale behind cyclical administration, which is meant to decrease the risk for osteomalacia.
Will etidronate now be approved in the U.S. for the treatment of steroid-induced osteoporosis, based on the results of this and prior trials? Or will the results be extrapolated to alendronate and other similar agents? The final thrust of the editorial is on the use of "bisphosphonates" rather than "etidronate", and Merck may well be more likely to benefit from this study than Procter & Gamble.
August 19, 1997
ReferencesReferences related to this article from the NLM's PubMed database.
Reader CommentsDate: Thu, 21 Aug 97
From: "Mark Leber" <email@example.com>
Were the racial composition and habits such as cigarette smoking and
alcohol equal in both control and experimental groups? In the post hoc
analysis, was the significant p-value adjusted downward based on multiple
comparisons as is done in the Bonferroni test? How were outliers explained
and adjusted for? What effect did outliers have on the data?
When multiple comparisons (subgroup analyses) are made, more stringent p values (i.e. smaller than 0.05) are usually required to meet the test of statistical significance. The subgroups looked at here (men, premenopausal women and postmenopausal women) were all specified a priori and randomization was stratified according to them. Thus, it is not clear to me that a Bonferroni correction or similar adjustment would be appropriate here. Nevertheless, the fact that these subgroups do not seem to have been specified as part of the primary endpoint analysis remains a bit troubling.
No data is given concerning outliers (not unusual for this type of study). -- mj
Date: Tue, 26 Aug 1997
An additional point in follow-up to the interesting discussion. Re proportion of vertebral fractures in postmenopausal women (1/31 treatment group vs 7/32 placebo group, P = 0.05), reported midway down the right column on p. 385 in the paper. A two by two table analysis of this data reveals an odds ratio of 0.12 with a 95% confidence interval of 0.01-1.09 (ie. not significant).
It's probably because the numbers are small, and I don't mean to try to remove any possibility that a benefit exists re fracture reduction, but it is an argument for providing confidence intervals whenever P values are presented, because they inform more about strength of evidence.
John Foxworth, PharmD
The fact that the p-value is a bit greater than 0.05 explains why the 95% confidence interval for the odds ratio extends beyond 1.0 (my program gives me an odds ratio of 0.12 with a 95% confidence interval of 0.003 to 1.06). -- mj
November 30, 1997
A recent editorial in the Lancet (November 8, 1997) addresses the issue of alendronate vs. etidronate (without coming to a clear conclusion).
January 5, 1998
to the editor about this article, from the December 28, 1997 New England
Journal of Medicine.
Journal Club homepage