The survival-inferred fragility index (SIFI) calculated for phase 3 randomized controlled trials (RCTs) of immune checkpoint inhibitors (ICIs) suggests that their survival data may not robustly show clinical benefit, particularly for cancers of the head and neck, liver, and small-cell lung cancer (SCLC), according to results of a study published in JAMA Network Open.1
SIFI “is the iterative reassignment of the best survivors from the experimental group to the control group, until positive significance is lost,” the authors wrote. Because the proportion of patients eligible for ICIs who benefit from these agents is decreasing for unknown reasons, the purpose of this study was to determine the vulnerability of the survival data from phase 3 trials.
The cross-sectional study included 45 phase 3 RCTs of ICIs across all cancer types. The majority of trials evaluated anti–PD-1 antibodies (56%), followed by anti–PD-L1 antibodies (27%), anti–CTLA-4 antibodies (13%), and a combination of anti–CTLA-4 and anti–PD-1 antibodies (7%). The median sample size for eligible trials was 559 patients.
Continue Reading
The SIFI was calculated using Kaplan-Meier overall survival curves, in which the best survivors in the ICI groups were reassigned to the control group until the statistical significance of P <.05 was lost. The resulting number was SIFI, with a lower number indicating more vulnerability, and therefore greater uncertainty, in the data, and a higher number indicating greater robustness of the data.
Across all ICI trials, the median SIFI for the intention-to-treat population was 5 (interquartile range, -4 to 12). The absolute SIFI was 1% or less of the sample size in 35% of the ITT populations.
Among specific cancer types, the SIFI was larger — indicating more robust data — in trials of non-small cell lung cancer, renal cell carcinoma, and melanoma. In contrast, the SIFI was lowest in trials of hepatocellular carcinoma, head and neck squamous cell carcinoma, and small cell lung cancer.
“The SIFI was not explained entirely by the variation in P values,” the authors noted. For some trials, the SIFI differed despite having similar P values, hazard ratios, and sample sizes (the SIFI was 2-fold higher in KEYNOTE-02440 compared with IMpower133, and in ATTRACTION-233 compared with CheckMate 06736 monotherapy).
In “these examples,” the authors said, “statistical significance depends on the distribution of the longest-surviving patients, with more fragile studies relying on fewer patients to drive the significance.” The duration of follow-up did not affect the SIFI.
The authors concluded that “the results of this study suggest that many phase 3 RCTs evaluating ICI therapies are fragile and challenge the confidence in rejecting or concluding superiority for these drugs compared with standard treatments.”
Disclosure: Some of the authors of the original work reported financial relationships with pharmaceutical companies. For a full list of disclosures, please refer to the original study.
Reference
Bomze D, Asher N, Ali OH, et al. Survival-inferred fragility index of phase 3 clinical trials evaluating immune checkpoint inhibitors. JAMA Netw Open. 2020;3(10):e2017675. doi: 10.1001/jamanetworkopen.2020.17675
