Pretrained transformers applied to clinical studies improve predictions of treatment efficacy and associated biomarkers

Abstract

Cancer treatment has made significant advancements in recent decades, leading to improved outcomes and quality of life for many patients. Despite the array of available therapies, including targeted, hormone, and checkpoint blockade immunotherapy, many patients experience treatment failure or eventual resistance. Attempts to predict the efficacy of therapies, particularly immuno-oncology therapies, have suffered from limited accuracy and difficulties in identifying molecular and other determinants of response. Improving treatment prediction alone is insufficient to create clinically meaningful research tools; additional prerequisites for this goal involve accommodating small data sets, effectively handling sparse features, integrating diverse clinical data, addressing missing measurements, ensuring interpretability, and extracting valuable biological insights for both clinical context and further research. Multimodal deep-learning models offer a promising avenue to surmount these challenges by leveraging their capacity and flexibility to learn from expansive and varied clinical and molecular data sets. Similar to their application in natural language and other domains, deep-learning models can uncover complex relationships within data that are pertinent to survival and treatment response. In this study, we introduce an explainable transformer-based deep-learning framework that addresses these challenges. This framework yields predictions of survival outcomes, as quantified by concordance index, that surpass the performance of state-of-the-art methods such as Cox proportional hazards, survival random forest, and tumor mutation burden, across diverse independent data sets. We developed the clinical transformer, a deep neural-network survival prediction framework that capitalizes on the flexibility of the deep-learning model, including training strategies like gradual and transfer learning, to maximize the use of available data to enhance survival predictions and generate actionable biological insights. Finally, we illustrate the future potential of the clinical transformer’s generative capability in early-stage clinical studies. By perturbing molecular features associated with immune checkpoint inhibition treatment in immunotherapy-naive patient profiles, we identified a subset of patients who may benefit from immunotherapy. These findings were subsequently validated across three independent immunotherapy treatment cohorts. We anticipate that this research will empower the scientific community to further harness data for the benefit of patients.