Statistical challenges posed by uncontrolled master protocols: sensitivity analysis of the Vemurafenib study

Abstract

Within the evidentiary hierarchy of experimental inquiry, randomized trials are the gold standard. Oncology patients enter clinical studies with diverse lifestyles, treatment pathways, host tissue environments, and competing co-morbidities. Randomization attempts to balance prognostic characteristics among study arms, thereby enabling statistical inference of "average benefit" and attribution to the studied therapies. In contrast, interpretations of uncontrolled trials require additional scrutiny to attempt to place the findings in the context of external evidence. Counter-factual reasoning and speculation across trials may be obscured by the disproportionate enrollment of prognostic subpopulations which may be unknown from publications of trial reports. Recent modifications to the regulatory environment (Food and Drug Administration Safety and Innovation Act) have elevated the importance of non-comparative trials. Moreover, the emergence of recent innovations in precision medicine have yielded trial designs that partition potentially heterogeneous subpopulations into "statistically exchangeable" cohorts by histologies, or genetic alterations, further elevating the importance of single-cohort analyses. As patient cohorts become ever more refined into smaller targeted subsets, consumers of reports of uncontrolled trials should be further empowered with improvements in reporting practices that better describe the enrolled prognostic subpopulations and importantly their association with study endpoints. This article demonstrates the issue with a sensitivity analysis of the findings reported in a recent trial which was devised to evaluate the preliminary clinical efficacy of vemurafenib in BRAFV⁶⁰⁰ mutation–positive nonmelanoma cancers.