ABSTRACT
Introduction Confounding bias threatens the reliability of observational data and identifying confounders poses a significant scientific challenge. We hypothesize that adjustment sets of literature-derived confounders could also improve causal inference. This paper shows how to exploit literature-derived knowledge to identify confounders for causal inference from observational data. We show how semantic constraint search over literature-derived computable knowledge helps reduce confounding bias in statistical models of EHR-derived observational data.
Methods We introduce two methods (semantic vectors and string-based confounder search) that query the literature for potential confounders and use this information to build models from EHR-derived data to more accurately estimate causal effects. These methods search SemMedDB for indications TREATED BY the drug that is also known to CAUSE the adverse event. For evaluation, we attempt to rediscover associations in a publicly available reference dataset containing expected pairwise relationships between drugs and adverse events from empirical data derived from a corpus of 2.2M EHR-derived clinical notes. For our knowledge-base, we use SemMedDB, a database of computable knowledge mined from the biomedical literature. Using standard adjustment and causal inference procedures on dichotomous drug exposures, confounders, and adverse event outcomes, varying numbers of literature-derived confounders are combined with EHR data to predict and estimate causal effects in light of the literature-derived confounders. We then compare results from the adjustment and inference procedures with naive (χ2, reporting odds ratio) measures of association.
Results and Conclusions Logistic regression with ten vector space-derived confounders achieved the most improvement with AUROC of 0.628 (95% CI: [0.556,0.720]), compared with baseline χ2 0.507 (95% CI: [0.431,0.583]). Bias reduction was improved more often in modeling methods using more rather than less information, and using semantic vector rather than string-based search. We found computable knowledge useful for improving automated causal inference, and identified opportunities for further improvement, including a role for adjudicating literature-derived confounders by subject matter experts.
Highlights
This paper introducs a framework to facilitate quasi-automated causal inference from EHR.
We search computable knowledge for indications TREATED BY the drug that CAUSE the outcome.
We test the performance of vector-based versus string-based confounder search.
Computable knowledge helps interpret and explain data captured in clinical narratives.
Causal models informed with semantic vector-based confounders improved upon string-based models.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This research was supported by the US National Library of Medicine grants: R01 LM011563, 2 T15 LM007093$-$26, 5 T15 LM007059$-$32, NIH/BD2K supplement R01 LM011563$-$02S1, NCATS Grant U54 TR002804, Cancer Prevention Research Institute of Texas (CPRIT) Precision Oncology Decision Support Core RP150535, and CPRIT Data Science and Informatics Core for Cancer Research (RP170668).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
University of Texas and University of Pittsburgh IRBs.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
- justification for the arbitrary threshold on the number of confounders per variable - improved exposition of the training and querying processes Predication-based Semantic Indexing - more details on the notion of causal relevance - new experiment testing the effect of including irrelevant covariates
Data Availability
Following IRB approval and a data usage agreement, we obtained permission to use the EHR data for this study. These data require approval to access. However, we have included the models and other analytic data from this project.
