Reinforcement Learning Strategies for Clinical Trials in Nonsmall Cell Lung Cancer
Summary Typical regimens for advanced metastatic stage IIIB/IV nonsmall cell lung cancer (NSCLC) consist of multiple lines of treatment. We present an adaptive reinforcement learning approach to discover optimal individualized treatment regimens from a specially
designed clinical trial (a “clinical reinforcement trial”) of an experimental treatment for patients with advanced NSCLC who have not been treated previously with systemic therapy. In addition to the complexity of the problem of selecting optimal compounds for first‐ and
second‐line treatments based on prognostic factors, another primary goal is to determine the optimal time to initiate second‐line therapy, either immediately or delayed after induction therapy, yielding the longest overall survival time. A reinforcement learning method called Q‐learning
is utilized, which involves learning an optimal regimen from patient data generated from the clinical reinforcement trial. Approximating the Q‐function with time‐indexed parameters can be achieved by using a modification of support vector regression that can utilize
censored data. Within this framework, a simulation study shows that the procedure can extract optimal regimens for two lines of treatment directly from clinical data without prior knowledge of the treatment effect mechanism. In addition, we demonstrate that the design reliably selects the
best initial time for second‐line therapy while taking into account the heterogeneity of NSCLC across patients.
Document Type: Research Article
Global Biostatistics and Epidemiology, Amgen Inc., One Amgen Center Drive, Thousand Oaks, California 91320, U.S.A.
Department of Biostatistics, University of North Carolina at Chapel Hill, 3101 McGavran-Greenberg, CB 7420, Chapel Hill, North Carolina 27599, U.S.A.
Department of Medicine, University of North Carolina at Chapel Hill, Physicians Office Building, 170 Manning Drive, Chapel Hill, North Carolina 27599, U.S.A.
Publication date: December 1, 2011