Adaptive Trial Design: An Overview

In our recent inaugural SOCCA journal club, we discussed the RENOVATE trial. High-flow nasal oxygen was found to be non-inferior to non-invasive ventilation in 4 out of the 5 patient groups with acute respiratory failure (nonimmunocompromised with hypoxemia, acute cardiogenic pulmonary edema, hypoxemic COVID-19, COPD exacerbation with respiratory acidosis) (1). However, there were several concerns raised with the trial design, so we decided to use this opportunity to introduce the concept of adaptive trial design.

Clinical trials frequently require time, large sample size and financial resources; yet, they often lack the power to evaluate efficacy overall or in individual subgroups, leading to further ineffective subgroup analyses (3). Traditional trials are also inflexible - as all the learning takes place at the end of enrollment. Adaptive trials, in contrast, through continual modifications while data collection is ongoing, increase the efficiency of randomized clinical trials. Importantly, one must be careful that error rates (particularly type 1) do not significantly change during this process. Therefore, statistics play an integral role. The planning phase involves multiple simulations to understand the benefits and consequences of all potential adaptations, as well as of the specified “decision rules.” The most common decision rules are formulaic with endpoints that result in allocation ratio changes between groups or termination of an individual treatment arm vs the whole trial (2). These simulations should be transparent and conducted with input from the clinical team. It is also critical that the agreed upon statistical analysis plan covers future interim analyses as well as the final analysis. In the RENOVATE trial, Bayesian adaptive simulation was utilized as the primary statistical analysis; with this method, posterior probabilities determine non-inferiority, futility or superiority.

Adaptive trials can be broadly divided into exploratory and confirmatory types. Exploratory trials typically deal with effective/safe dosing questions or dose-dependent modeling (3); these trials are less relevant for the broader clinical community. Confirmatory trials, as mentioned above, make planned changes to the future course of a trial based on data analysis from the trial itself; this takes place after at least 20-30 patients in each arm - otherwise known as the “burn-in period (2).” Types of common modifications include but are not limited to: changing the sample size, ending treatment arms or doses, changing allocation ratios, identifying treatment arms most likely to benefit (and directing recruitment to those groups) and ending the trial early (for success or futility) (5). In the RENOVATE trial, several of these modifications were implemented. It incorporated a group-sequential design, which allowed the trial to terminate the immunocompromised arm early for futility after only 98 patients. It also utilized dynamic borrowing, which allowed for smaller sample sizes in certain treatment arms while maintaining statistical rigor. Most apparent, the creation of a hypoxemic COVID-19 treatment arm reflects its multi-arm/multi-stage design.

Trial modifications inherently increase the overall trial complexity. To ensure the validity of adaptive trials, careful oversight is needed from all parties. For example, if decision rules are designed inappropriately, bias enters the interim analyses. Or, if any adaptations are not predetermined, the validity of the trial suffers. Unfortunately, a recent survey found that at least one-fourth of adaptive trials did not provide details on interim analyses and about one-fifth of adaptive trials made unplanned adaptations (4). The Food and Drug Administration recently issued guidance on adaptive trial designs; however, no concrete recommendations were given. Therefore, establishing independent data-monitoring committees is critical while the adoption of adaptive clinical trials continues to grow. In the same vein, the RENOVATE trial should be commended for using a blinded steering committee; the update in statistical analysis plan was also published in a widely-read journal in a transparent manner (6).

REFERENCES:

1. RENOVATE Investigators and the BRICNet Authors; Maia IS, Kawano-Dourado, et al. High-Flow Nasal Oxygen vs Noninvasive Ventilation in Patients With Acute Respiratory Failure: The RENOVATE Randomized Clinical Trial. JAMA. 2024 Dec 10.
2. Thorlund K, Haggstrom J, Park JJ, Mills EJ. Key design considerations for adaptive clinical trials: a primer for clinicians. BMJ. 2018 Mar 8;360:k698.
3. Bhatt DL, Mehta C. Adaptive Designs for Clinical Trials. N Engl J Med. 2016 Jul 7;375(1):65-74.
4. Wang Y, Yao M, Liu J, Liu Y, Ma Y, Luo X, Mei F, Xiang H, Zou K, Sun X, Li L. A systematic survey of adaptive trials shows substantial improvement in methods is needed. J Clin Epidemiol. 2024 Mar;167:111257.
5. Pallmann P, Bedding AW, Choodari-Oskooei B, Dimairo M, Flight L, Hampson LV, Holmes J, Mander AP, Odondi L, Sydes MR, Villar SS, Wason JMS, Weir CJ, Wheeler GM, Yap C, Jaki T. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018 Feb 28;16(1):29.
6. Maia IS, Kawano-Dourado L, Damiani LP, Fitzgerald M, Lewis RJ, Cavalcanti AB. Update in statistical analysis plan of the RENOVATE trial. Crit Care Resusc. 2023 Jul 26;25(3):113-114.

Author