PT - JOURNAL ARTICLE AU - McCaw, Zachary R AU - Shcherbina, Anna AU - Shah, Yajas AU - Huang, Davey AU - Elliott, Serra AU - Szabo, Peter M AU - Dulken, Benjamin AU - Holland, Sacha AU - Tagari, Philip AU - Light, David AU - Koller, Daphne AU - Probert, Christopher TI - Machine learning enabled prediction of digital biomarkers from whole slide histopathology images AID - 10.1101/2024.01.06.24300926 DP - 2024 Jan 01 TA - medRxiv PG - 2024.01.06.24300926 4099 - http://medrxiv.org/content/early/2024/01/09/2024.01.06.24300926.short 4100 - http://medrxiv.org/content/early/2024/01/09/2024.01.06.24300926.full AB - Current predictive biomarkers generally leverage technologies such as immunohis-tochemistry or genetic analysis, which may require specialized equipment, be time-intensive to deploy, or incur human error. In this paper, we present an alternative approach for the development and deployment of a class of predictive biomarkers, leveraging deep learning on digital images of hematoxylin and eosin (H&E)-stained biopsy samples to simultaneously predict a range of molecular factors that are relevant to treatment selection and response. Our framework begins with the training of a pan-solid tumor H&E foundation model, which can generate a universal featurization of H&E-stained tissue images. This featurization becomes the input to machine learning models that perform multi-target, pan-cancer imputation. For a set of 352 drug targets, we show the ability to predict with high accuracy: copy number amplifications, target RNA expression, and an RNA-derived “amplification signature” that captures the transcriptional consequences of an amplification event. We facilitate exploratory analyses by making broad predictions initially. Having identified the subset of biomarkers relevant to a patient population of interest, we develop specialized machine learning models, built on the same foundational featurization, which achieve even higher performance for key biomarkers in tumor types of interest. Moreover, our models are robust, generalizing with minimal loss of performance across different patient populations. By generating imputations from tile-level featurizations, we enable spatial overlays of molecular annotations on top of whole-slide images. These annotation maps provide a clear means of interpreting the histological correlates of our model’s predictions, and align with features identified by expert pathologist review. Overall, our work demonstrates a flexible and scalable framework for imputing molecular measurements from H&E, providing a generalizable approach to the development and deployment of predictive biomarkers for targeted therapeutics in cancer.Competing Interest StatementThe authors are employees or consultants of insitro.Funding StatementThis study did not receive any funding.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Publicly available data from The Cancer Genome Atlas were accessed through https://www.cancer.gov/tcga. Data from the ORIEN network, collected under the Total Cancer Care protocol, were used with permission and received IRB approvals from the participating institutions, see: https://www.orientcc.org/about-orien/.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesData produced in the present study are available upon reasonable request to the authors.