Cancer Therapy Advisor: You noted in your letter that the inclusion of falsified data in meta-analyses has the potential to (incorrectly) influence clinical and policy changes. What was the magnitude of influence of the false apixaban data? Do you have any evidence that the inclusion of these data in meta-analyses may have caused actual patient harm?
Dr Garmendia: Our research showed a couple of things:
- Meta-analyses are susceptible to including publications with falsified data.
- Nearly half of all meta-analyses saw a considerable change in outcomes/conclusions made when they included original publications that knowingly published falsified data.
This study did not correlate adverse events with the meta-analyses that contained falsified data. Our goal was to see if the inclusions of meta-analyses could be affected. Future studies could provide greater clarity on the outcomes of recommendations from meta-analyses with falsified data.
Cancer Therapy Advisor: Why are people still using trials and studies that have wrong information? Are the journal editors clearly marking studies as problematic or not? Why do you think incorrect data are still being looped into studies even as recently as 2017?
Dr Garmendia: In the ARISTOTLE example, authors never notified journals that FDA had 1 suspected, and 2 identified issues (falsified data). This issue with the ARISTOTLE trial only came to the attention of medical journals through Seife’s study.2 According to Seife, the articles have never been clarified to note the inclusion of falsified data.
Cancer Therapy Advisor: Although you looked specifically at apixaban, which is not technically a cancer drug (although it has recently been flagged as potentially being useful for patients with cancer),3,4 do you think this same conclusion is likely with some of the cancer medications that Seife flagged in his 2015 analysis? 2
Dr Garmendia: We do not believe apixaban is an isolated incident; it was merely chosen because it had the most publications with falsified data. The most problematic drugs from our point of view would be those publications for which the agency intended to disqualify the investigator, meaning FDA felt it could no longer trust any data from that clinical investigator. One should also be wary of any study rejected by the FDA but still published; however, determining which studies these are is nearly impossible. In regard to oncology clinical trials, given their complexity — [these] tend to be the most complicated — we believe these results could be duplicated with cancer drugs.
Cancer Therapy Advisor: You found that 32 of 99 analyses had conclusion-changing results and 31 of those 32 no longer favored apixaban over other therapies. Can you speak to the impact factor of the journals that published those 31 analyses? Is there any correlation between a poor impact factor and the likelihood of including a study with falsified data in it?
Dr Garmendia: Stay tuned for this analysis, we are writing a follow-up to include more detailed information regarding specific analyses. However … there was no change in the journal impact factor between all the meta-analyses that included falsified data and those that were affected by the falsified data. This means that the quality of the journal does not shield a meta-analysis from the effects of falsified data.
- Seife C. Research misconduct identified by the US Food and Drug Administration: out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. 2015;175(4):567-577.
- Garmendia CA, Nassar Gorra L, Rodriguez AL, Trepka MJ, Veledar E, Madhivanan P. Evaluation of the inclusion of studies identified by the FDA as having falsified data in the results of meta-analyses: the example of the apixaban trials [published online March 4, 2019]. JAMA Intern Med. doi: 10.1001/jamainternmed.2018.7661
- Carrier M, Abou-Nassar K, Mallick R, et al. Apixaban to prevent venous thromboembolism in patients with cancer. N Engl J Med. 2019;380(8):711-719.
- Storrar NPF, Mathur A, Johnson PRE, Roddie PH. Safety and efficacy of apixaban for routine thromboprophylaxis in myeloma patients treated with thalidomide‐ and lenalidomide‐containing regimens [published online May 22, 2018]. Br J Haematol. doi: 10.1111/bjh.15392