What to do when everything happens at once: Analytic approaches to estimate the health effects of co-occurring social policies
==============================================================================================================================

* Ellicott C. Matthay
* Laura M. Gottlieb
* David Rehkopf
* May Lynn Tan
* David Vlahov
* M. Maria Glymour

## Abstract

Social policies have great potential to improve population health and reduce health disparities. Thus, increasing empirical research seeks to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence which must be handled analytically for valid inferences. Although this is a substantial methodological challenge for studies aiming to isolate social policy effects, limited prior work has systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated seven analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We leveraged an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only 17 (31%) reported checking for any co-occurring policies, although 36 (67%) used at least one approach that helps address policy co-occurrence. The most common approaches were: adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen future studies on the health effects of social policies.

## INTRODUCTION

Social policies are promising mechanisms to improve population health and reduce health disparities. Analyses of the health effects of social policies routinely leverage policy changes occurring in one or multiple different places at different times, with differences-in-differences or similar study designs (1). In epidemiology and related fields, empirical health research using these methods has proliferated rapidly and yielded important findings (2–4). However, the validity of this approach is threatened when multiple related policies are adopted simultaneously or in close succession in the same jurisdiction. Bundles of related policies with similar potential health effects are often changed together, creating “co-occurrence” that must be addressed analytically for valid inference. Specifically, analyses that do not account for co-occurring policies are likely to be confounded, while analyses that incorporate measures of co-occurring policies can encounter imprecise or unstable estimates and bias resulting from data sparsity (5, 6).

While a rich literature exists on confounding and consequent data sparsity generally (7–12), several aspects of the policy co-occurrence problem make it important to consider separately from issues that arise with other exposures. By nature, policymaking may create correlations among policy variables that are much stronger than those typically observed in non-policy studies (13–15). Governments may respond to the desires of their constituents by adopting multiple related policies at the same time. For example, a state that moves to overhaul its social safety net is likely to change multiple related policies (e.g. income support and food insecurity benefit generosity) at the same time. The most promising analytic solutions may also be different. For example, if a set of policies are always adopted together, then estimating their combined effect is informative for real-world decision-making, whereas analyses of the combined effect of an exposure of interest and closely related confounders may be less useful. Additionally, some data sparsity problems can be addressed by increasing sample sizes, but policy studies are typically based on a small, fixed set of jurisdictions. Meanwhile, stronger theories or substantive knowledge about how a specific social policy functions could guide causal analyses evaluating the health effects of mediating variables (16). For example, if it is understood that compulsory schooling laws affect all-cause mortality by increasing educational attainment for some individuals, then such laws could serve as an instrument for studying the effects of changing educational attainment on health (17). Thus, the policy co-occurrence problem presents distinct challenges and possible analytic solutions beyond typical confounding.

In many policy domains, adopting groups of policies as a set is common (see Part 1 in this paper series). In these cases, researchers can implement a variety of study designs or statistical strategies to address potential bias or imprecision resulting from policy co-occurrence. Among these approaches, an overarching distinction is whether the approach aims to disentangle the effects of individual policies, or conceptualizes the co-occurring policies as a group and evaluates their joint effects. For a given study, either approach may be policy-relevant, depending on whether the goal is to deliver actionable evidence on the effects of a single policy or on the effects of a set of policies that would likely be adopted together.

To date, most approaches to handling social policy co-occurrence have been ad hoc. Applied studies in fields including epidemiology, political science, and health economics have acknowledged the issue by critiquing existing studies that have failed to account for co-occurring policies or by employing specific analytic solutions (18–23). Methodological work in specific fields such as environmental epidemiology, statistics, substance use, political science, and economics have also discussed individual analytic solutions relevant to these domains (6,8,11,24–28). The problem of multicollinearity is widely recognized in econometrics as a threat to causal inference (29–32), but to our knowledge, none have specifically addressed applications to research on the health effects of social policies. In this paper, we aimed to address the need for a systematic assessment of the analytic solutions that are applicable to research on the health effects of social policies, how often these solutions are used in practice, and the tradeoffs to consider in selecting an approach.

This is the second paper in a series on the social policy co-occurrence problem. The first paper demonstrated that co-occurring social policies are pervasive and that adequate adjustment for co-occurring policies is likely to substantially reduce the precision of estimated effects. Given this, delineating methods appropriate for this context is a high priority for the next generation of research on the health effects of social policies. Here we aim to describe analytic strategies prior researchers have adopted to address social policy co-occurrence with the goal of attaining valid inferences. We categorized these approaches based on the type of causal question they answer (e.g. the effect of an individual policy on a population subgroup versus the effect of a bundle of policies on the overall population). Using the sample of social policy evaluations developed in Part 1 of this series, we measured the proportion of studies in which authors assessed policy co-occurrence and the proportion applying each of the different analytic approaches to account for policy co-occurrence. We discuss the advantages and disadvantages of each approach and provide guidance on selecting among them.

## METHODS

### Identification of social policy studies

We leveraged an existing sample of studies on the health effects of social policies to review common strategies for addressing policy co-occurrence. The details of this review are described in Part 1 of this paper series. Briefly, we selected a multidisciplinary set of journals that publish health-related social policy research and are leading journals in their respective fields (*American Journal of Public Health, American Journal of Epidemiology, Journal of the American Medical Association, New England Journal of Medicine, The Lancet, American Journal of Preventive Medicine, Social Science and Medicine, Health Affairs, Demography,* and *American Economic Review*). We screened all 6,794 articles published in these journals in 2019 and included all original, empirical studies aiming to estimate the causal effects of one or more social policies on health-related outcomes (N=55). “Social policies” were defined as non-medical, population-based or targeted policies adopted at a community or higher level, and hypothesized to affect health or health inequalities via changes in social or behavioral determinants. We defined “health-related outcomes” broadly, to include morbidity, mortality, health conditions, and factors such as smoking, homelessness, and sales of unhealthy products. Given our focus on social interventions, we excluded studies that pertained to health care, health insurance, interventions delivered in the clinical setting, medications, or medical devices, including studies of the Affordable Care Act or Medicaid expansion.

### Categorization of analytic approaches

Our framework (Table 1) applies to research questions about the health effects of one particular policy (the “index” policy) in a defined target population. We assume investigators have identified relevant co-occurring policies that might confound the index policy.

View this table:
[Table 1:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/T1)

Table 1: 
Types of analytic approaches to address policy co-occurrence with corresponding causal research questions

A priori, we designated seven categories of analytic approaches that researchers could adopt in the face of policy co-occurrence, based on the causal question each method could answer (Table 1). We identified these approaches by reviewing multidisciplinary scientific literature on the study of co-occurring exposures, consulting with experts, and drawing on methods used in our own fields of research. We focused on methods that apply to study designs leveraging policy changes occurring in different places and different times, including aggregate or multi-level differences-in-differences and panel fixed effects. We defined “co-occurring” policies as policies whose adoption or implementation was correlated in space and time with an index policy (i.e. places and times with the index policy are also likely to have the “co-occurring” policy) and that likely affected the health outcome under study. This co-occurrence could be at multiple jurisdictional levels (e.g. cities within states), although single-jurisdiction-level studies were the norm.

### Approach 1: Adjust for co-occurring policies

If co-occurrence of related policies with the index policy is not severe (see Part 1 of this paper series), the researcher can adjust for measures of the other policies—for example, by controlling for co-occurring policy measures in a regression. Under conventional assumptions, the resulting estimand corresponds to the effect of the index policy on the health outcome. This approach will often rely on some degree of model-based extrapolation, because not all possible combinations of policies actually occur. It is incumbent on the investigator to confirm that any extrapolation is well-founded in theory or evidence.

As an example, Raifman and colleagues estimated the effect of state same-sex marriage laws on adolescent suicide attempts using a differences-in-differences analysis while controlling for policies banning sexual orientation-based employment discrimination (33). Because co-occurrence between the two types of policies was only moderate, regression adjustment for the co-occurring was sufficient to isolate the index policy.

Policy co-occurrence will be severe if a co-occurring policy aligns perfectly or nearly perfectly in place and time with the policy of interest. After adjusting for co-occurring policies, there will be insufficient independent variation in the index policy left to study, giving extremely imprecise estimates. The only analytic solutions are based on modifying the research question. Approaches 2-7 involve such alternative research questions and corresponding analytic approaches to assessing the impact of the index policy.

### Approach 2: Restrict the study sample to the region of common support

The issue of policy co-occurrence can be conceptualized as a form of strong confounding of the index policy by the other co-occurring policies. This confounding and consequent data sparsity result in a lack of common support in the data, also known as a violation of the positivity assumption (7). Positivity violations occur when some confounder strata do not have variation in the exposure— for example, because the confounding policy and index policy are always adopted as a set. This situation can be resolved by restricting the analysis to the confounder strata for which there is variation in the index policy, i.e., the data region of “common support”. Similar to Approach #1, this approach usually involves adjusting for co-occurring policies, but here, extrapolation is avoided by restricting the study sample. This approach changes the target population, so the corresponding causal question refers to the effect of the index policy on the health outcome in the restricted sample, and results are only generalizable to the population represented by the restricted sample. In the extreme, if an index policy and co-occurring policy are always adopted as set, then there may be no region of common support and alternative approaches must be considered (e.g. evaluating the combined impacts of a bundle of policies).

One way to implement this approach is to restrict the study sample to a subpopulation for whom exposure to the non-index co-occurring policies does not vary. If policies that might confound the association of interest do not vary within a particular study population, then they cannot cause confounding. For example, many households are eligible for multiple social welfare programs including the Supplemental Nutrition Assistance Program (SNAP) and the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). This makes it difficult to disentangle program effects. Liu and colleagues addressed this co-occurrence by studying outcomes for dual beneficiaries of SNAP and WIC versus WIC alone (34). Those without WIC could be considered “off-support” and are excluded, so all study participants are WIC beneficiaries. This approach changes the target population, so the corresponding causal question refers to the effect of the index policy (e.g. SNAP) on the health outcome in the restricted sample (e.g. WIC beneficiaries) and results generalize to the population represented by the restricted sample.

When there are many confounder strata, one accessible way to assess positivity and identify the region of common support is using propensity scores. In the context of assessing policy impacts, the propensity score (35) is the probability of adopting the index policy, given the confounding policies. Units that are “on-support” are those with propensity scores within the range of observed propensity scores both for units adopting the index policy and for units not adopting the index policy. A wide variety of matching and weighting methods involve using propensity scores to identify and restrict to the region of common support (10, 36). Numerous variations on this restriction have also been proposed, including restricting to units with propensity scores within a prespecified range (e.g. 0.1 to 0.9) or dynamic optimization procedures for selecting propensity score cutoffs (7–9).

An alternative approach to using propensity scores is to directly restrict the sample based on the distribution of the co-occurring policies themselves. Several approaches have been proposed, including restricting to units inside the convex hull of the covariate space defined by the secondary policies (11, 37), restricting to a sufficiently data-dense, rectangular region of the covariate space defined by the co-occurring policies (38–40), or tree-based methods (see for example (41)). These approaches are less common, but most can be readily implemented using existing software. In all cases, assessing the region of common support helps ensure that estimates are not relying on extrapolation to policy combinations which are never observed. The restricted study population should be well-defined, so that the investigator can transparently describe the places and times to which the results apply (38).

Chang and colleagues applied this approach to study the impacts of prescription drug monitoring programs (PDMPs) and pill mill laws using a comparative interrupted time series analysis (42). Rather than using all states in the analysis, some of which implemented other opioid policy changes in concert with PDMPs or pill mill laws, the authors restricted their analysis to Florida, which adopted PDMP and pill mill laws, and Georgia, which had a similar policy profile to Florida but did not implement the index policies during the study period. The authors determined that the combination of PDMPs and pill mill laws were associated with reductions in high-risk opioid prescribing for the Florida population.

### Approach 3: Define the outcome on subpopulations likely to be affected by the index policy but not other co-occurring policies

Identifying health effects that are specific to the index policy can be achieved by changing the outcome measure to one that is both closely aligned with the index policy and unlikely to be affected by other co-occurring policies. In particular, if the outcome is focused on a particular population subgroup (e.g. defined by age, gender, place, or time) that is likely to be most-affected by the index policy and unlikely to be affected by other co-occurring policy changes, study results can provide pointers to the impacts of the individual policy.

For example, changes in state Earned Income Tax Credit (EITC) policies have often co-occurred with other changes in other social welfare policies such as SNAP (43). Rehkopf and colleagues took advantage of the fact that EITC cash benefits are typically delivered in February, March, and April, while other benefits do not have the same seasonal dispersal pattern, to examine the association of EITC policies with health (44). They used a differences-in-differences approach to compared health outcomes that can change on a monthly basis (e.g. health behaviors, cardiovascular and metabolic biomarkers) for EITC-eligible versus non-eligible individuals in months of income supplementation versus non-supplementation. The authors were thus able to measure some potential short-term health impacts of EITC independent of other social welfare policies that do not have this seasonality.

This approach can be further strengthened by incorporating falsification tests or negative control analyses. Rehkopf and colleagues strengthened their findings by confirming that treating non-EITC transfer months as “treated” months produced null findings, and by confirming null associations for outcomes that do not change on a monthly basis.

### Approach 4: Select a less-correlated measure of policy exposure

Studies that use binary (0 or 1) characterizations of policy adoption are widespread in studies of the health impacts of social policies. However, more detailed characterizations of individual policies—for example, the amount of funding allocated, benefit generosity, participation rate, or population reach of a program; the size of a tax; or the number of years a policy has been in place—can deliver policy measures that are less correlated with other related policies, or opportunities to examine dose-response effects among jurisdictions adopting a policy. For example, adoption of more generous unemployment benefits, in terms of dollar amounts and durations for different types of households, tend to change in tandem with other worker protection and leave policies, but researchers have effectively disentangled effects of unemployment benefits by leveraging continuous measures of maximum allowable unemployment benefit levels across states (45, 46). Similar approaches have been taken to studying the effects of alcohol taxes (47, 48), tobacco taxes (49), and EITC benefit generosity (50, 51).

One useful modification to this approach is to study factors that may specifically mediate the relationship between the index policy and the health outcomes. For example, Matthay and colleagues generated evidence on the impacts of policies regulating gun shows by examining the impacts of gun show events themselves on firearm-related injuries in differing policy environments (52). Similarly, the impacts of medical cannabis provisions allowing supply through dispensaries (as opposed to home-cultivation) can be quantified by studying the direct effects of dispensaries on health (53, 54).

Studying mediators may also offer the opportunity to identify policy effects via the Front Door Criterion (16), a rarely-used alternative to confounder-control or instrument-based methods (4). If all the pathways by which the index policy affects the outcome can be measured, and there are no unmeasured confounders of the index policy-mediator relationship or of mediator-outcome relationship (conditional on the index policy), then the effect of the index policy on the outcome can be identified without measuring the co-occurring policies. For example, Bellemare and colleagues used the Front Door Criterion to estimate the effect of authorizing Uber and Lyft ride sharing with strangers on tipping drivers (55). They propose that they only way in which sharing authorization affects tipping is through whether a ride is actually shared (the mediator). This mediator is used as a tool to estimate the effect of sharing authorization while circumventing confounders of sharing authorization such as rider experience, mood, and social preferences. Although applications of the Front Door Criterion remain rare, a similar approach could be applied to social policy evaluations.

### Approach 5: Use formal Bayesian methods

Bayesian methods can be used to integrate information gleaned from other approaches to addressing policy co-occurrence; Bayesian methods can also be used as a way to treat estimation problems arising from policy co-occurrence without linking to other approaches we have mentioned. Several approaches to addressing policy co-occurrence depend on incorporating prior knowledge about the policies, determinants of the outcome, or hypothesized mechanisms of effect. For example, researchers may apply judgements about which policies affect the outcome or modify other policy processes. These insights can guide decisions about which policies need to be controlled and how. Bayesian methods offer a formal statistical method to incorporate prior knowledge about the plausible effects of the co-occurring and index policies, and to combine these with newly gathered empirical data.

When used alone, Bayesian methods can help address estimation issues and recover precision when highly co-occurring policies lead to convergence problems or imprecision. In particular, Bayesian approaches can stabilize estimates (i.e. address data sparsity-related problems of imprecision and sensitivity to different model specifications and influential data points) by constraining the effect sizes or interaction effects among policies and “shrinking” coefficients towards the specified prior distributions (22). This can be done without changing the set of adjustment variables, without restricting the study sample, and without changing the exposure or outcome measure. This approach is common in the environmental epidemiology literature as way to study multiple co-occurring exposures such as air pollutants (6, 26).

Harper used a Bayesian differences-in-differences approach to estimate the effects of adopting stronger enforcement of state seat belt laws on motor vehicle crash deaths (56). Because other road safety policies may also affect motor vehicle crash deaths and change in tandem with seat belt enforcement policies, Harper also adjusted for laws controlling maximum speed limits, blood alcohol concentration limits, graduated driver’s license programs, and annual state policy per capita as a proxy for traffic safety enforcement. Because these variables are correlated across states and years with each other and with seat belt enforcement policies, adjusting for them in a frequentist analysis reduces the precision of the estimated effect of interest—a major problem resulting from co-occurring policies. Harper enhanced precision by applying a Bayesian approach, drawing on existing evidence of the effects to seat belt laws to place empirical priors on the estimated effect of seat belt enforcement policies.

### Approach 6: Identify and evaluate the impacts of policy clusters

If a set of policies are typically adopted as a group, the effect of the combined set of policies may be the most pertinent parameter to estimate. By conceptualizing policy clusters as the exposure of interest, the investigator can preserve the original target population and outcome measure. For example, if two or more policies are highly co-occurring, it may be possible to estimate their combined impact (e.g., comparing health outcomes if both policies were adopted versus if neither policy were adopted) (57). Policy clusters can be defined based on substantive or policymaking considerations; this is useful if decisionmakers are considering adopting a set of policies. Alternatively, numerous data-driven clustering algorithms are applicable. Clusters or categories are defined based on how frequently policies co-occur (i.e., the extent to which policies co-occur in the same place and time). Methods include hierarchical cluster analysis, latent class analysis (LCA), or principal components analysis (PCA) (6,58,59). Clusters might also be defined based on the strength of the relationship with the outcome—for example using supervised PCA (6, 26). No one algorithm is considered optimal for all settings (6, 26).

Among data-driven algorithms, it is common to distinguish between “variable-centered” methods that group similar policy variables (e.g. PCA) and “person-centered” methods that group similar observations (e.g. LCA). The underlying mechanics of variable-centered and person-centered approaches are distinct, but both ultimately result in a small set of variables that summarize the policies to which each observation is exposed. This smaller set of variables is then used to assess health impacts. Erickson and colleagues used LCA to classify US states based on their position on 18 alcohol control policies (60). The analysis categorized each state into one of four unordered groups which the authors interpreted as: weak except serving policies, average, strong for underage use, and strong policies overall. State policy category was then associated with levels of past-month alcohol consumption.

### Approach 7: Use an overall policy stringency or generosity scor

If the investigator is interested the effects of the overall policy environment on health, one method is to use a summary score of the stringency or generosity of a set of policies. In comparison with Approach 6, this approach similarly involves reducing many policy variables to a few, but it typically focuses on creating an ordering along a pre-defined unidimensional scale such as stringency. In contrast, policy clusters are usually unordered and defined based on the co-variation amongst the policy measures themselves without regard for underlying characterizations such as stringency, although such characterizations may be applied after the fact when interpreting or describing the clusters. This approach also differs from Approach 4—in which one might characterize a single policy on a continuous scale to help disentangle the effects of that policy from other policies— because here we characterize a collection of policies with respect to their likely combined impact.

A simple way to apply this approach is to sum the number of policies in the set that apply in each place and time. Policies must be coded in the same direction so that the presence of more policies indicates greater restrictiveness, or vice versa. This method is easy to operationalize, but it implies that all policies carry equal weight and are interchangeable in achieving health effects.

A more sophisticated application is to weight policies based on existing evidence or expert opinion about the strength of the relationship with the outcome. This may be based on efficacy, restrictiveness, implementation, enforcement, enforceability, reach, or other metrics. Investigators have applied this approach in literature on firearm policy (61, 62), alcohol policy (20,63–65), and cannabis policy (66, 67). Although there are an infinite number of ways a set of policies can be ranked or weighted, use of systematic methods can enhance rigor and replicability. For example, the Delphi technique is a structured communication approach to elicit consensus from a panel of experts and can be used to rank or score policies based on stringency or effectiveness (68, 69). Assigned weights are typically outcome-specific—for example, weighting state alcohol policies with different levels effectiveness for binge drinking versus impaired driving and for adults versus youth (20, 63). Investigators can also explore different methods of weighting in sensitivity analyses (63).

### Data extraction and analysis

For each social policy study, we reviewed the full text. Our main focus was on the primary analytic specification, which we understood to be the authors’ leading approach to estimate the effect of the social policy for the health outcome(s) of interest. We also reviewed any sensitivity analyses reported in the main text. We assessed: (a) the overall analytic approach (e.g. differences-in-differences); (b) whether the authors reported checking for any co-occurring policies related to the health outcome of interest that might pose a threat to validity; (c) the authors determination of whether or not any co-occurring policies did, in fact, threaten validity (e.g. based on their analysis or prior literature); (d) whether there was any other indication that co-occurring policies exist for the study’s application (e.g. a co-occurring policy mentioned in the limitations); (e) if policy co-occurrence was identified as at threat, what analytic strategy the authors used to address it; and (f) any other aspects of the analytic strategy that may help address co-occurring policies, whether they were identified as a threat or not. We also documented whether studies utilized any approaches to address policy co-occurrence not identified a priori. We then tabulated these characteristics.

## RESULTS

We assessed 55 studies of social policies encompassing diverse topics, countries, and jurisdictional levels (70–125) (Appendix Table 1). Studies included, for example, a comparative interrupted time series evaluation of the impacts of lowering the blood alcohol concentration limit for drivers on road traffic accidents in Scotland (73) and a differences-in-differences analysis of the effects of state paid family leave policies on breastfeeding (98). The most common domains were poverty and social welfare policies such as the Supplemental Nutrition Assistance Program (14 studies); food and beverage policies such as sugar-sweetened beverage taxes (6 studies); firearm restrictions (5 studies); unemployment, sick leave, and pension benefit policies (4 studies); tobacco control (4 studies); alcohol control (4 studies); and immigration (4 studies).

Figure 1 presents a flowchart of the included studies, broken down by whether the authors evaluated policy co-occurrence and used techniques to address policy co-occurrence. Of the 55 studies, 4 involved methods for which assessing policy co-occurrence was not relevant: One involved a national policy with randomized rollout across village clusters, and for three others, the primary research question was about the overall policy environment and authors employed policy stringency scores. Of the remaining 51 studies, only 17 reported checking for at least one co-occurring policy. Of these 17, 10 reported identifying at least one co-occurring policy while 7 suggested that no co-occurring policies were a threat. For example, in a study of state texting-while-driving bans and traffic injuries, the authors acknowledged that administrative license suspension, speed limits, seatbelt requirements, and graduated driver licensing laws had also changed over the study period and might affect traffic injuries; they therefore controlled for measures of these policies in their differences-in-differences analysis (112). In contrast, a study of the effects of tuition-free primary education on access to family planning and health decision-making evaluated potentially co-occurring paid family leave policies (86); they determined that these policies did not substantially co-occur with tuition-free primary education but acknowledged that there may be other unmeasured co-occurring policies. Of the 34 studies that did not report checking for at least one co-occurring policy, 5 had some other indication that policy co-occurrence may be a threat.

![Figure 1:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2021/05/15/2020.10.05.20205963/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/F1)

Figure 1: 
Flowchart of included social policy studies by evaluation of policy co-occurrence and use of techniques to address policy co-occurrence

Table 2 summarizes the analytic methods used in each study, irrespective of whether the authors checked for co-occurring policies. Overall, 36 of the 55 studies (65%) incorporated at least one approach that addressed policy co-occurrence. Among studies that utilized at least one approach, the most common approaches were: adjusting for co-occurring policies (18 studies, 50%); defining the outcome on subpopulations likely to be affected by the index policy (but not other co-occurring policies) (14 studies, 39%); and selecting a less-correlated measure of policy exposure (7 studies, 19%). Twelve (33%) used more than one approach. None used formal Bayesian methods. Two studies reported no co-occurring policies but, without naming co-occurring policies as the motivation, nonetheless applied at least one approach that helps address co-occurring policies (Figure 1).

View this table:
[Table 2:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/T2)

Table 2: 
Overall analytic methods and approaches used to address policy co-occurrence in social policy studies

Studies employed a range of designs, the most common being differences-in-differences (12 studies), before-after (e.g. t-tests or interrupted time series; 9 studies), and other regression approaches without place-specific controls (hereafter, “other regression approaches”; e.g. multilevel regression of an overall policy stringency score on an individual-level health outcome without fixed effects; 7 studies) (Table 2). Studies using differences-in-differences, panel fixed effects, or other regression approaches were more likely to use at least one approach to address policy co-occurrence than studies using before-after designs.

Of the 10 studies that explicitly reported identifying one or more co-occurring policies, 9 attempted to address it (Figure 1). Of the 41 studies that did not check for or identify policy co-occurrence, 23 nonetheless used at least one technique that helps address policy co-occurrence. Several studies used multiple approaches in the same analysis. For example, a study of the effects of losing SNAP benefits controlled for participation in WIC and Temporary Assistance for Needy Families (TANF) program (Approach 1); excluded those with potential concurrent benefit changes in Supplemental Security Income (SSI) (Approach 2); and compared outcomes among SNAP participants who lost benefits to similar individuals with continuous benefits (Approach 3) (95).

## DISCUSSION

Co-occurring policies are common and can threaten the validity of studies of the health effects of social policies. In this paper, we describe seven analytic approaches to address policy co-occurrence. The approaches either seek to disentangle the effects of individual policies or estimate the combined effects of clusters of policies. Using a sample of contemporary studies on the health effects of social policies, we found that potential policy co-occurrence is frequently unidentified and unaddressed: only 33% of studies reported checking for policy co-occurrence as a potential threat to validity and only 65% incorporated any approach that helps to address policy co-occurrence, regardless of whether the authors checked for policy co-occurrence. Several studies that estimated the individual effect of the index policy used multiple, possibly complementary approaches to address co-occurrence in the same analysis; this may further enhance validity, although it is not guaranteed.

In future applied studies, systematically evaluating and reporting on policy co-occurrence would facilitate the evaluation of validity and interpretation of findings. Many studies (35%) did not report any approach to address policy co-occurrence. Authors may not have considered it or may have checked for it but not reported their assessment, particularly if it was not found to be a concern. However, because failure to address policy co-occurrence (if it exists) poses a major threat to validity, readers need to understand if the authors believe that no policy co-occurrence exists or if they believe it has been addressed (and if so, through what analytic strategies). Stringent word limits on many medical and public health journals preclude presenting full analytic exploration of issues such as policy co-occurrence. An openness to incorporating such discussions, at least in appendices, would enhance the rigor and interpretability of social policy studies.

We assessed whether the studies in our sample checked for *any* co-occurring policies, but ideally, researchers would evaluate *all* policies and related social, economic, and political phenomena that co-occur with the index policy and that could affect the health outcome of interest. This is a formidable task, especially given that innumerable policies are continuously being passed at all levels of government, that databases measuring relevant policies often do not exist, or that the policies that may affect an outcome are not fully understood. Policy libraries such as the University of Kentucky Center for Poverty Research state welfare database (43), the Policy Surveillance Program at Temple University (126), and the University of Iowa State Policy Innovation and Diffusion Database (127) are increasingly valuable resources, but they are burdensome to develop and maintain and require infrastructure support. Substantial, regular support for policy surveillance as well as scientific endeavors to link and harmonize large administrative datasets would support these efforts (128–130). Given that diverse policies across numerous disciplines are likely to affect health, interdisciplinary collaboration is also essential to these efforts.

### Tradeoffs between different approaches

This study describes how approaches to address policy co-occurrence have been used practice; a logical next question is which approaches are best-suited for different circumstances. The preferred approach to address policy co-occurrence should be driven by the target causal question (131). Investigators should select the approach that best-answers their causal question, while achieving sufficient accuracy and precision (e.g. based on likely sources of bias and evidence on the precision of different estimators). If an unbiased, precise estimate can be derived with simple adjustment for measures of the co-occurring policies (Approach 1), this option will often be preferable, because it does not change the target causal question or study population. However, if the target causal question is deemed unanswerable due to severe policy co-occurrence, a different analytic approach, potentially corresponding with a modified causal question, is necessary (131). More severe policy co-occurrence may necessitate larger departures from the original causal question. The extent to which alternative approaches change the causal question depends on the application—for example, whether restricting the study sample to the region of common support involves dropping many units or only a few—but Appendix Figure 1 shows an approximate ordering.

Table 3 summarizes tradeoffs of different approaches to addressing policy co-occurrence. Overall, approaches that preserve estimates of the independent effect of the index policy may be particularly useful for decisionmakers comparing specific policy options. However, these approaches generally sacrifice some aspect of generalizability by restricting the analysis to certain populations, subgroups, outcomes, or time periods for which policy effects can be estimated. Results may therefore serve as markers of policy impacts rather than measures of overall impact. Estimating the combined effects of a group of co-occurring policies sacrifices estimates of the independent effects of the index policy, but preserves generalizability to the original target population, outcomes, and time period under study. The preferred approach depends both on what options are viable (i.e. unconfounded, sufficient precision) and which causal question is of greatest interest. For example, if certain combinations of policies are always adopted together, then their independent effects may be neither estimable nor of interest.

View this table:
[Table 3:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/T3)

Table 3: 
Advantages and disadvantages of alternative approaches used to address policy co-occurrence in studies of the health effects of social policies

Among the individual approaches, key considerations include the circumstances in which the approach is feasible (e.g. controlling for co-occurring policies is not possible if policy co-occurrence is severe), the availability of evidence to support making analytic decisions (e.g. on how to use propensity scores, select weighting schemes for policy scores, or choose a clustering method), the extent to which the approach provides evidence that is relevant to the original causal question, ease of implementation, available data and measures, and interpretability of the results (see Table 3 for details). All of the approaches discussed here can also be used to evaluate whether policy co-occurrence is a concern by comparing results of analyses that do not account for policy co-occurrence to results from analyses that do. While none of these approaches will answer identical research questions, findings should generally align and comparison across methods can serve as a robustness check.

For all of the approaches, we note two important limitations. First, none of the seven approaches discussed here are guaranteed to resolve the analytic challenges presented by co-occurring policies. For example, one approach might isolate the effects of the primary policy of interest from some co-occurring policies but not others; another approach might help reduce problems of statistical power arising from policy co-occurrence for one outcome of interest but not another. Second, all of the approaches rely on accurate measurement of all of the relevant policies. Missing or mis-measured policies may lead to bias. Careful attention to the structure and potential impact of measurement error, along with analytic tools such as quantitative bias analysis, can enhance validity (132).

### Limitations

The seven approaches presented here are not an exhaustive list of all analytic solutions that could be applied to address policy co-occurrence and many sub-options exist. However, we did not encounter any other method that addresses policy co-occurrence in our sample of studies. Additionally, this study is based on a systematically gathered set of exemplar studies of the health effects of social policies; a comprehensive review of all studies of the health effects of social policies would be valuable in future research—for example to characterize patterns of methods utilization across journals and disciplines, and to assess whether studies are trending towards more rigorous approaches over time. Finally, as with all studies, there may be some misclassification. In particular, if an analytic approach was applied but not identified as for the purpose of addressing co-occurring policies (or an analogous problem under any other name), we may have missed it.

## Conclusions

Policy co-occurrence plagues most research on the health effects of social policies. In combination with Part 1 of this paper series—which illustrated how to assess the pervasiveness and consequences of policy co-occurrence—this review offers guidance on how to address this challenge. While randomization of policy rollouts can best estimate the causal effects of social policies, when not available, other methods can nonetheless indicate causality. These other methods demand careful selection of the research question and analytic approach and, guided by deep substantive knowledge and creativity, can help to overcome policy co-occurrence and deliver stronger evidence on the health effects of social policies.

## Data Availability

All data used in this study arise from published studies that are publicly available.

## Appendix

View this table:
[Appendix Table 1:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/T4)

Appendix Table 1: 
Social policy studies identified in systematic sample

![Appendix Figure 1:](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2021/05/15/2020.10.05.20205963/F2.medium.gif)

[Appendix Figure 1:](http://medrxiv.org/content/early/2021/05/15/2020.10.05.20205963/F2)

Appendix Figure 1: 
Schema of alternative approaches to address policy co-occurrence by severity of policy co-occurrence and degree of departure from original causal question

## Footnotes

*   Funding: This work was supported by the Evidence for Action program of the Robert Wood Johnson Foundation and the National Institute on Alcohol Abuse and Alcoholism (grant number K99 AA028256)

*   Conflicts of interest None: declared.

*   Revised terminology from "clustering" to "co-occurrence"; Revised typology of analytic strategies; additional detail on methods; clarification of study goals; added appendix.

*   Received October 5, 2020.
*   Revision received May 14, 2021.
*   Accepted May 15, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.Angrist JD, Pischke J-S. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press; 2008.
    
    

2.  2.Abstracts of the 46th Annual SER (Society for Epidemiologic Research) Meeting. Boston, Massachusetts, USA. June 18-21, 2013. Am. J. Epidemiol. 2013;177 Suppl 11:S1-181.
    
    

3.  3.Kaufman J. Chapter 26: Social Epidemiology. In: Modern Epidemiology. Philadelphia, PA: Lippincott Williams & Wilkins; 2008
    
    

4.  4.Matthay EC, Hagan E, Gottlieb LM, et al. Alternative causal inference methods in population health research: Evaluating tradeoffs and triangulating evidence. SSM - Population Health. 2020;10:100526.
    
    

5.  5.Petersen ML, Porter KE, Gruber S, et al. Diagnosing and responding to violations in the positivity assumption. Stat Methods Med Res. 2012;21(1):31–54.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0962280210386207&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21030422&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

6.  6.Stafoggia M, Breitner S, Hampel R, et al. Statistical Approaches to Address Multi-Pollutant Mixtures and Multiple Exposures: the State of the Science. Curr Environ Health Rep. 2017;4(4):481–490.
    
    

7.  7.Petersen ML, Porter KE, Gruber S, et al. Diagnosing and responding to violations in the positivity assumption Diagnosing and responding to violations in the positivity assumption. Stat Methods Med Res. 2012;21(1):31–54.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0962280210386207&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21030422&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

8.  8.Crump RK, Hotz VJ, Imbens GW, et al. Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand. National Bureau of Economic Research; 2006 (Accessed December 4, 2019).([http://www.nber.org/papers/t0330](http://www.nber.org/papers/t0330)). (Accessed December 4, 2019)
    
    

9.  9.Crump RK, Hotz VJ, Imbens GW, et al. Dealing with limited overlap in estimation of average treatment effects. Biometrika. 2009;96(1):187–199.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/biomet/asn055&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000263406100014&link_type=ISI) 

10. 10.Ho DE, Imai K, King G, et al. Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis. 2007;15(3):199–236.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/pan/mpl013&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000248544000002&link_type=ISI) 

11. 11.King G, Zeng L. The Dangers of Extreme Counterfactuals. Political Analysis. 2006;14(2):131–159.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/pan/mpj004&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000235924100001&link_type=ISI) 

12. 12.Dehejia RH, Wahba S. Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs. Journal of the American Statistical Association. 1999;94(448):1053–1062.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/2669919&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000083879800007&link_type=ISI) 

13. 13.Stone DA. Policy Paradox: The Art of Political Decision Making. 1997
    
    

14. 14.Cohen J. Statistical power analysis for the behavioral sciences. Second. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1988.
    
    

15. 15.(Em)powering Population Health Decision-Making: Maximizing the Potential of Social Interventions Research | Evidence for Action. ([https://www.evidenceforaction.org/empowering-population-health-decision-making-maximizing-potential-social-interventions-research](https://www.evidenceforaction.org/empowering-population-health-decision-making-maximizing-potential-social-interventions-research)). (Accessed September 8, 2020)
    
    

16. 16. Pearl J. Causality.  Cambridge University Press; 2009 486 p.
    
    

17. 17.Hamad R, Elser H, Tran DC, et al. How and why studies disagree about the effects of education on health: A systematic review and meta-analysis of studies of compulsory schooling laws. Social Science & Medicine. 2018;212:168–178.
    
    

18. 18.RAND Corporation. The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of Gun Policies in the United States. Santa Monica, CA: RAND Corporation; 2018.([https://www.rand.org/pubs/research_reports/RR2088.html](https://www.rand.org/pubs/research_reports/RR2088.html))
    
    

19. 19.Association between Florida’s Opioid Crackdown and Opioid-Related Mortality: The Role of Economic Factors and Mortality Misclassification | American Journal of Epidemiology | Oxford Academic. ([https://academic.oup.com/aje/advance-article-abstract/doi/10.1093/aje/kwaa016/5741088?redirectedFrom=fulltext](https://academic.oup.com/aje/advance-article-abstract/doi/10.1093/aje/kwaa016/5741088?redirectedFrom=fulltext)). (Accessed March 26, 2020)
    
    

20. 20.Nelson TF, Xuan Z, Babor TF, et al. Efficacy and the strength of evidence of U.S. alcohol control policies. Am J Prev Med. 2013;45(1):19–28.
    
    

21. 21.Dench D, Joyce T. The earned income tax credit and infant health revisited. Health Economics. 2020;29(1):72–84.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

22. 22.MacLehose RF, Dunson DB, Herring AH, et al. Bayesian Methods for Highly Correlated Exposure Data. Epidemiology. 2007;18(2):199.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/01.ede.0000256320.30737.c0&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17272963&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000244422000005&link_type=ISI) 

23. 23.Ahern J, Cerda M, Lippman SA, et al. Navigating non-positivity in neighbourhood studies: an analysis of collective efficacy and violence. J Epidemiol Community Health. 2013;67(2):159–165.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo4OiI2Ny8yLzE1OSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzA1LzE1LzIwMjAuMTAuMDUuMjAyMDU5NjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. 24.Kleinberg J, Ludwig J, Mullainathan S, et al. Prediction Policy Problems. American Economic Review. 2015;105(5):491–495.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1257/aer.p20151023&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27199498&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

25. 25.Schuler MS, Griffin BA, Cerdá M, et al. Methodological challenges and proposed solutions for evaluating opioid policy effectiveness. Health Serv Outcomes Res Method [electronic article]. 2020;([https://doi.org/10.1007/s10742-020-00228-2](https://doi.org/10.1007/s10742-020-00228-2)). (Accessed January 11, 2021)
    
    

26. 26.Taylor KW, Joubert BR, Braun JM, et al. Statistical Approaches for Assessing Health Effects of Environmental Chemical Mixtures in Epidemiology: Lessons from an Innovative Workshop. Environ Health Perspect. 2016;124(12):A227–A229.
    
    

27. 27. Lazarevic Nina,  Barnett Adrian G.,  Sly Peter D., et al. Statistical Methodology in Studies of Prenatal Exposure to Mixtures of Endocrine-Disrupting Chemicals: A Review of Existing Approaches and New Alternatives. Environmental Health Perspectives. 2019;127(2):026001.
    
    

28. 28.Wang J, He X, Xu G. Debiased Inference on Treatment Effect in a High-Dimensional Model. Journal of the American Statistical Association. 2019;():1–27.
    
    

29. 29.Imbens GW, Wooldridge JM. Recent Developments in the Econometrics of Program Evaluation. Journal of Economic Literature. 2009;47(1):5–86.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1257/jel.47.1.5&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000264807200002&link_type=ISI) 

30. 30.Heckman JJ, Vytlacil EJ. Chapter 70 Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation. In: Heckman JJ, Leamer EE, eds. Handbook of Econometrics. Elsevier; 2007 (Accessed July 5, 2018):4779–4874.([http://www.sciencedirect.com/science/article/pii/S1573441207060709](http://www.sciencedirect.com/science/article/pii/S1573441207060709)). (Accessed July 5, 2018)
    
    

31. 31.Arceneaux K, Huber GA. What to Do (and Not Do) with Multicollinearity in State Politics Research. State Politics & Policy Quarterly. 2007;7(1):81–101.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/153244000700700105&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000254347400005&link_type=ISI) 

32. 32.Farrar D, Glauber R. Multicollinearity in Regression Analysis: The Problem Revisited on JSTOR. The Review of Economics and Statistics. 1967;49(1):92–107.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/1937887&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1967ZA40800009&link_type=ISI) 

33. 33.Raifman J, Moscoe E, Austin SB, et al. Difference-in-Differences Analysis of the Association Between State Same-Sex Marriage Policies and Adolescent Suicide Attempts. JAMA Pediatr. 2017;171(4):350–356.
    
    

34. 34.Liu J, Kuo T, Jiang L, et al. Food and drink consumption among 1–5-year-old Los Angeles County children from households receiving dual SNAP and WIC v. only WIC benefits. Public Health Nutrition. 2017;20(14):2478–2485.
    
    

35. 35.Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/biomet/70.1.41&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1983QH66900005&link_type=ISI) 

36. 36.Stuart EA. Matching methods for causal inference: A review and a look forward. Stat Sci. 2010;25(1):1–21.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1214/09-STS313&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20871802&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

37. 37.Stuart EA, King G, Imai K, et al. MatchIt: Nonparametric Preprocessing for Parametric Causal Inference. Journal of Statistical Software [electronic article]. 2011;([https://dash.harvard.edu/handle/1/11130519](https://dash.harvard.edu/handle/1/11130519)). (Accessed December 11, 2019)
    
    

38. 38.Fogarty CB, Mikkelsen ME, Gaieski DF, et al. Discrete Optimization for Interpretable Study Populations and Randomization Inference in an Observational Study of Severe Sepsis Mortality. Journal of the American Statistical Association. 2016;111(514):447–458.
    
    

39. 39.Rosenbaum PR. Two Simple Models for Observational Studies. In: Rosenbaum PR, ed. Design of Observational Studies. New York, NY: Springer; 2010 (Accessed December 12, 2019):65–94.([https://doi.org/10.1007/978-1-4419-1213-8_3](https://doi.org/10.1007/978-1-4419-1213-8_3)). (Accessed December 12, 2019)
    
    

40. 40.Eckstein J, Hammer PL, Liu Y, et al. The Maximum Box Problem and its Application to Data Analysis. Computational Optimization and Applications. 2002;23(3):285–298.
    
    

41. 41.Chipman HA, George EI, McCulloch RE. BART: Bayesian additive regression trees. Ann. Appl. Stat. 2010;4(1):266–298.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1214/09-AOAS285&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000283528300012&link_type=ISI) 

42. 42.Chang H-Y, Lyapustina T, Rutkow L, et al. Impact of prescription drug monitoring programs and pill mill laws on high-risk opioid prescribers: A comparative interrupted time series analysis. Drug and Alcohol Dependence. 2016;165:1–8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.drugalcdep.2016.04.033&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27264166&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

43. 43.University of Kentucky Center for Poverty Research. UKCPR National Welfare Data, 1980-2018. Lexington, KY: 2019 (Accessed April 13, 2020).([http://ukcpr.org/resources/national-welfare-data](http://ukcpr.org/resources/national-welfare-data)). (Accessed April 13, 2020)
    
    

44. 44.Rehkopf DH, Strully KW, Dow WH. The short-term impacts of Earned Income Tax Credit disbursement on health. Int J Epidemiol. 2014;43(6):1884–1894.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyu172&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25172139&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

45. 45.Cylus J, Glymour MM, Avendano M. Do generous unemployment benefit programs reduce suicide rates? A state fixed-effect analysis covering 1968-2008. Am. J. Epidemiol. 2014;180(1):45–52.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwu106&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24939978&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000339808200006&link_type=ISI) 

46. 46.Cylus J, Glymour MM, Avendano M. Health Effects of Unemployment Benefit Program Generosity. Am J Public Health. 2014;105(2):317–323.
    
    

47. 47.Elder RW, Lawrence B, Ferguson A, et al. The Effectiveness of Tax Policy Interventions for Reducing Excessive Alcohol Consumption and Related Harms. American Journal of Preventive Medicine. 2010;38(2):217–229.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amepre.2009.11.005&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20117579&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

48. 48.Xuan Z, Chaloupka FJ, Blanchette JG, et al. The relationship between alcohol taxes and binge drinking: evaluating new tax measures incorporating multiple tax and beverage types. Addiction. 2015;110(3):441–450.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/add.12818&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25428795&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

49. 49.Chaloupka FJ, Straif K, Leon ME. Effectiveness of tax and price policies in tobacco control. Tobacco Control. 2011;20(3):235–238.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6InRvYmFjY29jb250cm9sIjtzOjU6InJlc2lkIjtzOjg6IjIwLzMvMjM1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDUvMTUvMjAyMC4xMC4wNS4yMDIwNTk2My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

50. 50.Evans WN, Garthwaite CL. Giving Mom a Break: The Impact of Higher EITC Payments on Maternal Health. American Economic Journal: Economic Policy. 2014;6(2):258–290.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1257/pol.6.2.258&link_type=DOI) 

51. 51.Larrimore J. Does a Higher Income Have Positive Health Effects? Using the Earned Income Tax Credit to Explore the Income-Health Gradient. The Milbank Quarterly. 2011;89(4):694–727.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1468-0009.2011.00647.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22188352&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000298357400006&link_type=ISI) 

52. 52.Matthay EC, Galin J, Rudolph KE, et al. In-State and Interstate Associations Between Gun Shows and Firearm Deaths and Injuries: A Quasi-experimental Study. Annals of Internal Medicine. 2017;167(12):837.
    
    

53. 53.Mair C, Freisthler B, Ponicki WR, et al. The impacts of marijuana dispensary density and neighborhood ecology on marijuana abuse and dependence. Drug and Alcohol Dependence. 2015;154:111–116.
    
    

54. 54.Freisthler B, Gaidus A, Tam C, et al. From Medical to Recreational Marijuana Sales: Marijuana Outlets and Crime in an Era of Changing Marijuana Legislation. J Primary Prevent. 2017;38(3):249–263.
    
    

55. 55.Bellemare MF, Bloem JR, Wexler N. The Paper of How: Estimating Treatment Effects Using the Front-Door Criterion. 2020 (Accessed February 1, 2021).([http://marcfbellemare.com/wordpress/wp-content/uploads/2020/09/BellemareBloemWexlerFDCSeptember2020.pdf](http://marcfbellemare.com/wordpress/wp-content/uploads/2020/09/BellemareBloemWexlerFDCSeptember2020.pdf)). (Accessed February 1, 2021)
    
    

56. 56.Harper S. Would Stronger Seat Belt Laws Reduce Motor Vehicle Crash Deaths?: A Semi-Bayesian Analysis. Epidemiology. 2019;30(3):380–387.
    
    

57. 57.Ng M, Colson KE, Fullman N, et al. Assessing the Contribution of Malaria Vector Control and Other Maternal and Child Health Interventions in Reducing All-Cause Under-Five Mortality in Zambia, 1990–2010. The American Journal of Tropical Medicine and Hygiene. 2017;97(3_Suppl):58–64.
    
    

58. 58.Nylund-Gibson K, Choi AY. Ten frequently asked questions about latent class analysis. Translational Issues in Psychological Science. 2018;4(4):440–461.
    
    

59. 59.Maimon O, Rokach L. Clustering Methods. In: The data mining and knowledge discovery handbook. Tel Aviv University, Israel: Springer; 2005
    
    

60. 60.Erickson DJ, Lenk KM, Toomey TL, et al. The alcohol policy environment, enforcement and consumption in the United States. Drug Alcohol Rev. 2016;35(1):6–12.
    
    

61. 61.Fleegler EW, Lee LK, Monuteaux MC, et al. Firearm Legislation and Firearm-Related Fatalities in the United States. JAMA Intern Med. 2013;173(9):732–740.
    
    

62. 62.Goyal MK, Badolato GM, Patel SJ, et al. State Gun Laws and Pediatric Firearm-Related Mortality. Pediatrics [electronic article]. 2019;144(2). ([https://pediatrics.aappublications.org/content/144/2/e20183283](https://pediatrics.aappublications.org/content/144/2/e20183283)). (Accessed December 17, 2019)
    
    

63. 63.Naimi TS, Blanchette J, Nelson TF, et al. A new scale of the U.S. alcohol policy environment and its relationship to binge drinking. Am J Prev Med. 2014;46(1):10–16.
    
    

64. 64.Xuan Z, Blanchette J, Nelson TF, et al. The Alcohol Policy Environment and Policy Subgroups as Predictors of Binge Drinking Measures Among US Adults. Am J Public Health. 2014;105(4):816–822.
    
    

65. 65.Erickson DJ, Lenk KM, Toomey TL, et al. Measuring the Strength of State-Level Alcohol Control Policies. World Med Health Policy. 2014;6(3):171–186.
    
    

66. 66.Chapman SA, Spetz J, Lin J, et al. Capturing Heterogeneity in Medical Marijuana Policies: A Taxonomy of Regulatory Regimes Across the United States. Subst Use Misuse. 2016;51(9):1174–1184.
    
    

67. 67.Klitzner MD, Thomas S, Schuler J, et al. The New Cannabis Policy Taxonomy on APIS: Making Sense of the Cannabis Policy Universe. J Primary Prevent. 2017;38(3):295–314.
    
    

68. 68.Hsu C-C, Sandford BA. The Delphi Technique: Making Sense Of Consensus. Practical assessment, research & evaluation. 12(10):1–8.
    
    

69. 69.Landeta J. Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change. 2006;73(5):467–482.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.techfore.2005.09.002&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000237961000001&link_type=ISI) 

70. 70.Litwin A, Perova E, Reynolds SA. A conditional cash transfer and Women’s empowerment: Does Bolsa Familia Influence intimate partner violence? Social Science & Medicine. 2019;238:112462.
    
    

71. 71.Myran DT, Chen JT, Bearnot B, et al. Alcohol Availability Across Neighborhoods in Ontario Following Alcohol Sales Deregulation, 2013–2017. Am J Public Health. 2019;109(6):899–905.
    
    

72. 72.Lira MC, Xuan Z, Coleman SM, et al. Alcohol Policies and Alcohol Involvement in Intimate Partner Homicide in the U.S. American Journal of Preventive Medicine. 2019;57(2):172–179.
    
    

73. 73.Haghpanahan H, Lewsey J, Mackay DF, et al. An evaluation of the effects of lowering blood alcohol concentration limits for drivers on the rates of road traffic accidents and alcohol consumption: a natural experiment. The Lancet. 2019;393(10169):321–329.
    
    

74. 74.Gertner AK, Rotter JS, Shafer PR. Association Between State Minimum Wages and Suicide Rates in the U.S. American Journal of Preventive Medicine. 2019;56(5):648–654.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amepre.2018.12.008&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

75. 75.Roberto CA, Lawman HG, LeVasseur MT, et al. Association of a Beverage Tax on Sugar-Sweetened and Artificially Sweetened Beverages With Changes in Beverage Prices and Sales at Chain Retailers in a Large Urban Setting. JAMA. 2019;321(18):1799–1810.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2019.4249&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31087022&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

76. 76.Sivaraman JJ, Ranapurwala SI, Moracco KE, et al. Association of State Firearm Legislation With Female Intimate Partner Homicide. American Journal of Preventive Medicine. 2019;56(1):125–133.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

77. 77.Ghiani M, Hawkins SS, Baum CF. Associations Between Gun Laws and Suicides. Am J Epidemiol. 2019;188(7):1254–1261.
    
    

78. 78.Rajmil L, Fernández de Sanmamed M-J. Austerity Policies and Mortality Rates in European Countries, 2011–2015. Am J Public Health. 2019;109(5):768–770.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

79. 79.Lin M, Wang Q. Center-based childcare expansion and grandparents’ employment and well-being. Social Science & Medicine. 2019;240:112547.
    
    

80. 80.Ahmed S, Fielding D. Changes in maternity leave coverage: Implications for fertility, labour force participation and child mortality. Social Science & Medicine. 2019;241:112573.
    
    

81. 81.Lin B-H, Guthrie JF, Smith TA. Dietary Guidance and New School Meal Standards: Schoolchildren’s Whole Grain Consumption Over 1994–2014. American Journal of Preventive Medicine. 2019;57(1):57–67.
    
    

82. 82.Delaruelle K, van de Werfhorst H, Bracke P. Do comprehensive school reforms impact the health of early school leavers? Results of a comparative difference-in-difference design. Social Science & Medicine. 2019;239:112542.
    
    

83. 83.Chen X, Wang T, Busch SH. Does money relieve depression? Evidence from social pension expansions in China. Social Science & Medicine. 2019;220:411–420.
    
    

84. 84.Högberg B, Strandh M, Petersen S, et al. Education system stratification and health complaints among school-aged children. Social Science & Medicine. 2019;220:159–166.
    
    

85. 85.Mooney AC, Neilands TB, Giannella E, et al. Effects of a voter initiative on disparities in punishment severity for drug offenses across California counties. Social Science & Medicine. 2019;230:9–19.
    
    

86. 86.Bose B, Heymann J. Effects of tuition-free primary education on women’s access to family planning and on health decision-making: A cross-national study. Social Science & Medicine. 2019;238:112478.
    
    

87. 87.Rummo PE, Noriega D, Parret A, et al. Evaluating A USDA Program That Gives SNAP Participants Financial Incentives To Buy Fresh Produce In Supermarkets. Health Affairs. 2019;38(11):1816–1823.
    
    

88. 88.Matthay EC, Farkas K, Rudolph KE, et al. Firearm and Nonfirearm Violence After Operation Peacemaker Fellowship in Richmond, California, 1996–2016. Am J Public Health. 2019;109(11):1605–1611.
    
    

89. 89.Angeles G, de Hoop J, Handa S, et al. Government of Malawi’s unconditional cash transfer improves youth mental health. Social Science & Medicine. 2019;225:108–119.
    
    

90. 90.Evans WN, Kroeger S, Palmer C, et al. Housing and Urban Development–Veterans Affairs Supportive Housing Vouchers and Veterans’ Homelessness, 2007–2017. Am J Public Health. 2019;109(10):1440–1445.
    
    

91. 91.Wright M, McKelvey W, Curtis CJ, et al. Impact of a Municipal Policy Restricting Trans Fatty Acid Use in New York City Restaurants on Serum Trans Fatty Acid Levels in Adults. Am J Public Health. 2019;109(4):634–636.
    
    

92. 92.Parmar D, Banerjee A. Impact of an employment guarantee scheme on utilisation of maternal healthcare services: Results from a natural experiment in India. Social Science & Medicine. 2019;222:285–293.
    
    

93. 93.Massri C, Sutherland S, Källestål C, et al. Impact of the Food-Labeling and Advertising Law Banning Competitive Food and Beverages in Chilean Public Schools, 2014–2016. Am J Public Health. 2019;109(9):1249–1254.
    
    

94. 94.Boertien D, Vignoli D. Legalizing Same-Sex Marriage Matters for the Subjective Well-being of Individuals in Same-Sex Unions. Demography. 2019;56(6):2109–2121.
    
    

95. 95.Ettinger de Cuba S, Chilton M, Bovell-Ammon A, et al. Loss Of SNAP Is Associated With Food Insecurity And Poor Health In Working Families With Young Children. Health Affairs. 2019;38(5):765–773.
    
    

96. 96.Regidor E, Mateo A, Barrio G, et al. Mortality in Spain in the Context of the Economic Crisis and Austerity Policies. Am J Public Health. 2019;109(7):1043–1049.
    
    

97. 97.Hernandez EM, Vuolo M, Frizzell LC, et al. Moving Upstream: The Effect of Tobacco Clean Air Restrictions on Educational Inequalities in Smoking Among Young Adults. Demography. 2019;56(5):1693–1721.
    
    

98. 98.Hamad R, Modrek S, White JS. Paid Family Leave Effects on Breastfeeding: A Quasi-Experimental Study of US Policies. Am J Public Health. 2018;109(1):164–166.
    
    

99. 99.Everson EM, Dilley JA, Maher JE, et al. Post-Legalization Opening of Retail Cannabis Stores and Adult Cannabis Use in Washington State, 2009–2016. Am J Public Health. 2019;109(9):1294–1301.
    
    

100.100.Reinhard E, Carrino L, Courtin E, et al. Public Transportation Use and Cognitive Function in Older Age: A Quasiexperimental Evaluation of the Free Bus Pass Policy in the United Kingdom. Am J Epidemiol. 2019;188(10):1774–1783.
    
    

101.101.Hughes C. Reexamining the Influence of Conditional Cash Transfers on Migration From a Gendered Lens. Demography. 2019;56(5):1573–1605.
    
    

102.102.Torche F, Sirois C. Restrictive Immigration Law and Birth Outcomes of Immigrant Women. American Journal of Epidemiology. 2019;188(1):24–33.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

103.103.Doucette ML, Crifasi CK, Frattaroli S. Right-to-Carry Laws and Firearm Workplace Homicides: A Longitudinal Analysis (1992–2017). Am J Public Health. 2019;109(12):1747–1753.
    
    

104.104.Kingsley M, Setodji CM, Pane JD, et al. Short-Term Impact of a Flavored Tobacco Restriction: Changes in Youth Tobacco Use in a Massachusetts Community. American Journal of Preventive Medicine. 2019;57(6):741–748.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amepre.2019.07.024&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

105.105.Stallings-Smith S, Hamadi HY, Peterson BN, et al. Smoke-Free Policies and 30-Day Readmission Rates for Chronic Obstructive Pulmonary Disease. American Journal of Preventive Medicine. 2019;57(5):621–628.
    
    

106.106.Heflin C, Arteaga I, Hodges L, et al. SNAP benefits and childhood asthma. Social Science & Medicine. 2019;220:203–211.
    
    

107.107. Cuba Sae de, Bovell-Ammon AR, Cook JT, et al. SNAP, Young Children’s Health, and Family Food Security and Healthcare Access. American Journal of Preventive Medicine. 2019;57(4):525–532.
    
    

108.108.Tessler RA, Mooney SJ, Quistberg DA, et al. State-Level Beer Excise Tax and Firearm Homicide in Adolescents and Young Adults. American Journal of Preventive Medicine. 2019;56(5):708–715.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

109.109.Stacey N, Mudara C, Ng SW, et al. Sugar-based beverage taxes and beverage prices: Evidence from South Africa’s Health Promotion Levy. Social Science & Medicine. 2019;238:112465.
    
    

110.110.Lee MM, Falbe J, Schillinger D, et al. Sugar-Sweetened Beverage Consumption 3 Years After the Berkeley, California, Sugar-Sweetened Beverage Tax. Am J Public Health. 2019;109(4):637–639.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2105/AJPH.2019.304971&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30789776&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

111.111.Franckle RL, Thorndike AN, Moran AJ, et al. Supermarket Purchases Over the Supplemental Nutrition Assistance Program Benefit Month: A Comparison Between Participants and Nonparticipants. American Journal of Preventive Medicine. 2019;57(6):800–807.
    
    

112.112.Ferdinand AO, Aftab A, Akinlotan MA. Texting-While-Driving Bans and Motor Vehicle Crash–Related Emergency Department Visits in 16 US States: 2007–2014. Am J Public Health. 2019;109(5):748–754.
    
    

113.113.Klarevas L, Conner A, Hemenway D. The Effect of Large-Capacity Magazine Bans on High-Fatality Mass Shootings, 1990–2017. Am J Public Health. 2019;109(12):1754–1761.
    
    

114.114.Pilkauskas N, Michelmore K. The Effect of the Earned Income Tax Credit on Housing and Living Arrangements. Demography. 2019;56(4):1303–1326.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 

115.115.Heflin CM, Ingram SJ, Ziliak JP. The Effect Of The Supplemental Nutrition Assistance Program On Mortality. Health Affairs. 2019;38(11):1807–1815.
    
    

116.116.Shahidi FV, Muntaner C, Shankardass K, et al. The effect of unemployment benefits on health: A propensity score analysis. Social Science & Medicine. 2019;226:198–206.
    
    

117.117.Ku L, Brantley E, Pillai D. The Effects of SNAP Work Requirements in Reducing Participation and Benefits From 2013 to 2017. Am J Public Health. 2019;109(10):1446–1451.
    
    

118.118.Park W, Baek J. The impact of employment protection on health: Evidence from fixed-term contract workers in South Korea. Social Science & Medicine. 2019;233:158–170.
    
    

119.119.Hamad R, Batra A, Karasek D, et al. The Impact of the Revised WIC Food Package on Maternal Nutrition During Pregnancy and Postpartum. Am J Epidemiol. 2019;188(8):1493–1502.
    
    

120.120.Bruzelius E, Baum A. The Mental Health of Hispanic/Latino Americans Following National Immigration Policy Changes: United States, 2014–2018. Am J Public Health. 2019;109(12):1786–1788.
    
    

121.121.Levy DT, Yuan Z, Li Y, et al. The Minnesota SimSmoke Tobacco Control Policy Model of Smokeless Tobacco and Cigarette Use. American Journal of Preventive Medicine. 2019;57(4):e103–e115.
    
    

122.122.Liao Z, Zhou Y, Li H, et al. The Rates and Medical Necessity of Cesarean Delivery in the Era of the Two-Child Policy in Hubei and Gansu Provinces, China. Am J Public Health. 2019;109(3):476–482.
    
    

123.123.Couttenier M, Petrencu V, Rohner D, et al. The Violent Legacy of Conflict: Evidence on Asylum Seekers, Crime, and Public Policy in Switzerland. American Economic Review. 2019;109(12):4378–4425.
    
    

124.124.Patler C, Hamilton E, Meagher K, et al. Uncertainty About DACA May Undermine Its Positive Impact On Health For Recipients And Their Children. Health Affairs. 2019;38(5):738–745.
    
    

125.125.Lebihan L, Mao Takongmo C-O. Unconditional cash transfers and parental obesity. Social Science & Medicine. 2019;224:116–126.
    
    

126.126.LawAtlas. Policy Surveillance Portal. 2021;([http://lawatlas.org/](http://lawatlas.org/)). (Accessed February 1, 2021)
    
    

127.127.Boehmke FJ. State Policy Innovation and Diffusion (SPID) Database v1.0. 2018;([https://dataverse.harvard.edu/citation?persistentId](https://dataverse.harvard.edu/citation?persistentId)=doi:10.7910/DVN/CVYSR7). (Accessed February 1, 2021)
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7910/DVN/CVYSR7&link_type=DOI) 

128.128.Min S, Martin LT, Rutter CM, et al. Are Publicly Funded Health Databases Geographically Detailed and Timely Enough to Support Patient-Centered Outcomes Research? J GEN INTERN MED. 2019;34(3):467–472.
    
    

129.129.Davis JC, Holly BP. Regional Analysis Using Census Bureau Microdata at the Center for Economic Studies. International Regional Science Review. 2006;29(3):278–296.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0160017606289898&link_type=DOI) 

130.130.Erdem E, Korda H, Haffer S “Chris, et al. Medicare Claims Data as Public Use Files: A New Tool for Public Health Surveillance. Journal of Public Health Management and Practice. 2014;20(4):445.
    
    

131.131.Petersen ML, van der Laan MJ. Causal Models and Learning from Data. Epidemiology. 2014;25(3):418–426.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/EDE.0000000000000078&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24713881&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000337313300012&link_type=ISI) 

132.132.van Smeden M, Lash TL, Groenwold RHH. Reflection on modern methods: five myths about measurement error in epidemiological research. International Journal of Epidemiology. 2020;49(1):338–347.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyz251&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31821469&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F05%2F15%2F2020.10.05.20205963.atom)