# How to Trust an Observational Study

By Nora Bissett, Pricing Editor (https://glpwatchdog.org/authors/nora-bissett) | Published: 2026-08-20 | Updated: 2026-10-30

Canonical page: https://glpwatchdog.org/research/how-to-trust-an-observational-study

> We have twice warned that a very large hazard ratio from an observational study is telling you about the study. This one, reporting 0.42 and 0.58, deserves trust, and the article names the specific techniques that earn it.

## Key points

- They first EMULATED two randomized trials in the same data and reproduced their results.
- Negative control outcomes did not improve, so it is not simply a healthier group.
- Semaglutide HR 0.58, tirzepatide HR 0.42 vs sitagliptin, on HF hospitalization or death.
- Head to head: tirzepatide vs semaglutide HR 0.86 (0.70–1.06). No meaningful difference.
- Sitagliptin is a placebo proxy, not a placebo, and claims data cannot see an echocardiogram.

We have twice warned that a very large hazard ratio from an observational study is usually telling you about the study rather than the drug. This one reports hazard ratios of **0.42 and 0.58** and deserves considerably more trust, because of a set of specific techniques you can learn to recognize.[1]

## The technique that earns the trust

Before asking their real question, the researchers used their claims database to *emulate two randomized trials whose results were already published*, STEP-HFpEF DM and SUMMIT. They rebuilt each trial’s eligibility criteria, exposures and outcomes inside the data, ran the analysis, and compared what came out against what the trials had found.

> The emulations agreed with the trials on every prespecified metric.

Only then did they widen the eligibility criteria to the kind of patients seen in ordinary practice (the people trials exclude) and ask the question nobody had randomized evidence for.

> **That is a positive control for an entire study design.** A method that reproduces answers we already know has earned some credit when it reports an answer we do not. It is the same logic as testing a detector on a known signal before trusting it on an unknown one, and almost no observational study in this field does it.

The second technique is **negative control outcomes**: outcomes the drug has no plausible reason to affect. If the results were being produced by healthier people getting the newer drug, those outcomes would improve too. They did not, which is evidence the effect is not simply a healthier group in disguise.

## What they found

The question was heart failure with preserved ejection fraction alongside obesity or type 2 diabetes, a condition where early trials were promising but rested on very few actual events. The outcome was a composite of hospitalization for heart failure or death from any cause, over up to 52 weeks. Sitagliptin, a drug with no expected effect here, served as the comparator.

*US claims data, 2018 to 2024, with propensity score weighting.[1]*

| Comparison | Patients | Hazard ratio (95% CI) |
| --- | --- | --- |
| Semaglutide vs sitagliptin | 58,333 | **0.58 (0.51–0.65)** |
| Tirzepatide vs sitagliptin | 11,257 | **0.42 (0.31–0.57)** |
| **Tirzepatide vs semaglutide** | 28,100 | **0.86 (0.70–1.06)** |

Secondary endpoints, subgroups and sensitivity analyses were consistent, and no substantial safety signal appeared.

## The third row is a null, and it matters

Set head to head, tirzepatide showed **no meaningfully lower risk than semaglutide**. The hazard ratio was 0.86, with an interval running from 0.70 to 1.06 that includes no difference.

This matters because tirzepatide produces substantially more weight loss. If the benefit here ran through weight, the drug that removes more of it should win. It did not, which fits the larger pattern we have covered: [the cardiovascular benefit does not track weight loss](https://glpwatchdog.org/research/losing-weight-predicted-more-heart-attacks), and a synthesis of [262 trials](https://glpwatchdog.org/research/what-262-trials-say-together) similarly found the drug with the mortality evidence sitting fifth on the weight table.

## What still limits it

- **Sitagliptin is a placebo proxy, not a placebo.** It is chosen because it should not affect these outcomes, and it is still a drug given to people for reasons.
- **Claims data cannot see an echocardiogram.** Preserved ejection fraction is inferred from diagnostic codes, so some patients are certainly misclassified.
- **Fifty-two weeks**, in a chronic condition.
- **Benchmarking validates the method against two specific trials**, in the populations those trials enrolled. It does not guarantee the method behaves as well in the expanded population, which is precisely where no randomized answer exists.

That last caveat is the real limit of the whole approach, but it does not undo the value of the exercise. A method shown to work where it can be checked is a better instrument than one never checked at all.

## The portable version

When you meet a large effect from an observational study, these are the questions that separate the credible from the merely striking.

- **Did they benchmark against a known answer?** Emulating an existing trial and reproducing it is the strongest signal available.
- **Are there negative control outcomes?** If everything improved, the patients differed, not the treatment.
- **Is the comparator an active drug or nothing?** Comparing against another treatment partly controls for who gets prescribed what, which is why [a study of GLP-1s and epilepsy](https://glpwatchdog.org/research/glp1s-and-epilepsy-risk) measured them against a different diabetes drug.
- **Was the outcome recorded for everyone?** One surgery-versus-drug comparison balanced 30,458 people, then [took its weight figures from the 1.6% of the drug arm whose weight was on record](https://glpwatchdog.org/research/the-weight-comparison-used-257-people).
- **Is the effect size physiologically plausible?** This remains the first filter (see [when a hazard ratio is too good](https://glpwatchdog.org/research/when-a-hazard-ratio-is-too-good)).

## Frequently asked questions

### Do GLP-1 drugs help heart failure with preserved ejection fraction?

In this analysis of US claims data, semaglutide and tirzepatide were each associated with substantially lower risk of heart failure hospitalization or death compared with sitagliptin, with hazard ratios of 0.58 and 0.42 over up to 52 weeks.

### Why trust these numbers when other large observational results are unreliable?

Because the researchers first emulated two randomized trials inside the same data and reproduced their published results, then expanded to the wider population. A method that reproduces known answers has earned credit on an unknown one.

### Is tirzepatide better than semaglutide here?

No meaningful difference appeared. The hazard ratio was 0.86, with a confidence interval from 0.70 to 1.06 that includes no difference, despite tirzepatide producing more weight loss.

### What is a negative control outcome?

An outcome the treatment has no plausible reason to affect. If it improves too, that suggests the treated group was simply healthier rather than that the drug worked.

### What are the main limitations?

Sitagliptin is a stand-in for placebo rather than a placebo, preserved ejection fraction is inferred from diagnostic codes rather than measured, follow-up is 52 weeks, and the benchmarking validates the method in the trial populations rather than the expanded one.

## References

1. Krüger N, Schneeweiss S, Fuse K, et al. Semaglutide and Tirzepatide in Patients With Heart Failure With Preserved Ejection Fraction. JAMA. 2025. PMID: 40886075. https://pubmed.ncbi.nlm.nih.gov/40886075/

---

Cite as: GLP Watchdog, Nora Bissett, "How to Trust an Observational Study", published 2026-08-20, updated 2026-10-30. https://glpwatchdog.org/research/how-to-trust-an-observational-study

Canonical HTML page: https://glpwatchdog.org/research/how-to-trust-an-observational-study

Medical disclaimer: Nothing here is medical advice. We report on this subject; we do not treat anyone. Whether to begin a medication, supplement, or treatment, whether to adjust it, and whether to stop are questions for a clinician licensed to answer them, who knows your history.
