When Pooling Studies Hides the Answer

A meta-analysis of 323,439 patients asked whether tirzepatide prevents more cardiovascular events than the GLP-1 drugs it competes with. The answer was no significant difference — hazard ratio 0.85, with the interval crossing 1.^[1] The more important number is buried beside it: I² = 90.4%, which means the studies being pooled disagree with each other almost completely.

What I² measures, and why 90% is a problem

When several studies of the same question produce different answers, some of that spread is ordinary sampling noise and some is genuine difference — different populations, different definitions, different confounding. I² estimates what proportion is the second kind.

A rough convention treats 25% as low, 50% as moderate and 75% as high. Ninety percent is off the end of that scale.

A pooled estimate from studies that disagree this much describes none of them.

The likely source is visible in the study list. Seven of the eight studies were observational — drawn from different health systems, with different patients, different comparison groups and different residual confounding. That those seven disagree is unsurprising. What it means is that pooling them does not resolve the disagreement, it conceals it.

The result, such as it is

All three point the same direction, which is worth something. None reaches significance, and the third stops exactly on the boundary at 1.00 — a result that will be described as significant by someone.

Exploratory subgroup analyses in people with type 2 diabetes did reach significance: all-cause mortality 0.85 (0.76–0.94) and heart failure 0.75 (0.58–0.97). Those are the numbers that will be quoted, and they are subgroups of an exploratory analysis inside a meta-analysis whose primary comparison was null and heterogeneous.

It agrees with the other evidence, which matters more than the pooling

Set the meta-analysis aside and look at what else exists. A claims-based study that benchmarked its own method against two randomized trials before answering compared these drugs head to head on heart failure outcomes and found 0.86 (0.70–1.06) — a null, and almost the same numbers — in how to trust an observational study.

That completes a picture this register has been assembling. Tirzepatide beats semaglutide decisively on weight in the randomized head-to-head — 20.2% against 13.7%, in the head-to-head. On cardiovascular outcomes, no advantage has been demonstrated, and semaglutide is the agent with the mortality evidence across 262 trials.

None of that means tirzepatide is worse for the heart. It means the trial that would settle it has not reported, and the authors of this analysis say exactly that: further randomized trials are needed.

How to read the next meta-analysis you meet

• Find I² before the summary estimate. Above about 75%, the pooled number is describing a disagreement rather than a finding.
• Count how many included studies were randomized. One of eight, here.
• Check whether the significant results are subgroups. They usually are, and they usually get quoted.
• Prefer agreement between methods over precision within one. Two different approaches landing on the same answer is worth more than a tight interval around an average of contradictions.