What to do when everything happens at once: Analytic approaches to estimate the health effects of co-occurring social policies

Abstract

Social policies have great potential to improve population health and reduce health disparities. Thus, increasing empirical research seeks to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence which must be handled analytically for valid inferences. Although this is a substantial methodological challenge for studies aiming to isolate social policy effects, limited prior work has systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated seven analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We leveraged an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only 17 (31%) reported checking for any co-occurring policies, although 36 (67%) used at least one approach that helps address policy co-occurrence. The most common approaches were: adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen future studies on the health effects of social policies.