Over- and under-estimation of vaccine effectiveness =================================================== * Hilla De-Leon * Dvir Aran ## ABSTRACT **Background** The effectiveness of SARS-CoV-2 vaccines has been a subject of debate, with varying results reported in different studies, ranging from 60-95% vaccine effectiveness (VE). This range is striking when comparing two studies conducted in Israel at the same time, as one study reported VE of 90-95%, while the other study reported only ~80%. We argue that this variability is due to inadequate accounting for indirect protection provided by vaccines, which can block further transmission of the virus **Materials and Methods** We developed a novel analytic heterogenous infection model and extended our agent-based model of disease spread to allow for heterogenous interactions between vaccinated and unvaccinated across close-contacts and regions. We applied these models on real-world regional data from Israel from early 2021 to estimate VE using two common study designs: population-based and secondary infections **Results** Our results show that the estimated VE of a vaccine with efficacy of 85% can range from 70-95% depending on the interactions between vaccinated and unvaccinated individuals. Since different study designs capture different levels of interactions, we suggest that this interference explains the variability across studies. Finally, we propose a methodology for more accurate estimation without knowledge of interactions **Discussions and Conclusions** Our study highlights the importance of considering indirect protection when estimating vaccine effectiveness, explains how different study designs may report biased estimations, and propose a method to overcome this bias. We hope that our models will lead to more accurate understanding of the impact of vaccinations and inform public health policy. ## INTRODUCTION Hundreds of millions of individuals have been infected with the SARS-CoV-2 virus, and millions of them died because of the infection since it was first detected in China at the end of 2019. In response, several companies and research institutions developed vaccines to help suppress the pandemic, and less than a year after identifying of the virus, several clinical trials reported results of proven vaccine efficacy (1,2). The randomized clinical trials of the Pfizer/BioNTech and Moderna vaccines showed vaccine efficacy of around 95% in the first months after the second dose; however, the studies had relatively low numbers of infected individuals, and this level of efficacy can only be regarded as a proxy to the real-world vaccine effectiveness across divergent populations, demographics, and clinical features. By the end of 2020, many countries launched mass vaccination campaigns. The data collected through these mass vaccination campaigns was used to evaluate the effectiveness of the vaccines in a wide range of populations (3–15). Individual-level study designs for vaccine effectiveness evaluation can be split into two main approaches: (1) A population-based approach (PB), where the number of vaccinated and unvaccinated infections are counted after correcting for possible confounders (3); (2) A secondary infections approach (SI), which allows to overcome a major limitation of the first approach of unknown exposure. In this approach, infections are counted after a known exposure, such as secondary household infections (4). Interestingly, when examining the vaccine effectiveness estimations from studies that use each approach, there are major differences. In PB studies, vaccine effectiveness was usually estimated at 90%-95% (1–3,5), while in SI studies, much lower vaccine effectiveness estimations are usually reported (60%-80%) (4,11,12). Moreover, observational studies in nursing home residents reported varying vaccine effectiveness levels ranging from 53% to 92% against SARS-CoV-2 infection (6,13,14). The difference between the two effectiveness levels is emphasized when we compare the 90%-95% effectiveness of Dagan et al. (3) with the 80% effectiveness of Gazit et al. (4). Both studies evaluated the effectiveness of the same vaccine in a similar population in Israel at the same time, and only the study design differed. We hypothesize that the main reason for this gap can be explained by the proportion of interactions of vaccinated individuals with other vaccinated individuals that diverges between the two settings. Our hypothesis is based on one feature of the vaccines that has been broadly neglected in current vaccine effectiveness studies. While the vaccines were designed mainly to reduce symptomatic disease, severe disease, and death, studies from 2021 have shown that they also provide efficient transmission-blocking. A study from Sweden found that the number of immune members in a family was negatively correlated with the likelihood of incidence of infection of non-immune family members (16). Similarly, a study from Israel found reduced infection rates in communities as vaccination rates increased (17). Previous studies before COVID-19 have shown how interference, the potential outcomes of one individual are affected by the treatment assignment of other individuals, can affect vaccine effectiveness estimations (18). It seems that an underlying assumption made by the COVID-19 vaccine effectiveness studies is that there is no interference, which is an unfounded assumption. Previous studies presented causal inference frameworks with interference to deal with the effects of indirect protection on vaccine effectiveness estimations (18–23). The main novelty of this work stems from our attempt to use a numerical model that incorporates real-world data on vaccination and infection rates to demonstrate how different study designs may lead to divergent estimations of VE. Here, we developed both an analytical approach and a numerical approach which utilizes our Monte Carlo Agent-based Model (MAM) (24) to examine the influence of interference between vaccinated and unvaccinated individuals on data of COVID-19 vaccinations and infections in Israel (25,26). Our analyses show that different assumptions may lead to significant under- or over-estimations and might explain the differences in estimations between the different study design approaches. We suggest that this divergence is inversely related to the rate of mixing between vaccinated and unvaccinated individuals, in line with previous studies on interference (19,20,23). Since it is not simple to estimate the level of mixing, which diverges across time and communities, we developed a simple approach that can overcome this issue and provide accurate estimations of vaccine effectiveness regardless of mixing levels. ## MATERIALS and METHODS The main aim of this paper is to quantify vaccine effectiveness estimations in terms of interference between vaccinated and unvaccinated individuals. Interference occurs when the treatment received by other individuals influences the effects of an exposure or intervention on an individual. Because interference can affect the relationship between exposure and outcome, it can be harder to determine causality. In terms of this work, we define interference as the effect on the risk of infection for a vaccinated individual due to exposure to an unvaccinated individual. All the data and code for modelling and figures presented in this manuscript are available on: https://github.com/hdeleon1/Over-and-under-estimation- ### Analytical infection model First, we developed a simple heterogenous analytical infection model to find a quantifiable relationship between the amount of interference in a network and the effectiveness of vaccines (see **Supplementary Methods**; **Table S1-2** for more information). Within the model, we define a network of vaccinated and unvaccinated nodes. Each node is strongly connected to all the (*N**in* − 1) nodes in its clique, and weakly contacted to all *N**out* other nodes (*N**all* *= N**in* *+ N**out*) with *δ* as the ratio between the number of infections that originate from the inner clique and the other infections. Using the model we can define the observed vaccine effectiveness, ![Graphic][1], as a function of the average fraction of edges with unvaccinated nodes to the vaccinated nodes, ⟨*I*⟩: ![Formula][2] With *v* as the fraction of unvaccinated people in the total population, ![Graphic][3] and ![Graphic][4]. To summarize, this analytical model is a relatively simple model that simulates infections in a population with non-uniform vaccination rates and at heterogenous circuits of infection risks. It considers the risk of infection in vaccinated households and allows to quantify the population ![Graphic][5], corrected by the interference, as opposed to the individual-level vaccine effectiveness. ### MAM – A flexible Monte-Carlo Agent based Model To model the infections occurred in the real-world we use MAM, a flexible Monte-Carlo Agent based Model for modelling COVID-19 spread that accurately predicts infection dynamics in Israel (24). In this work, we expand this model to include complex interaction networks, such as differing between household interactions, which represent around half of infections in Israel (27), and other interactions. The spatial feature of MAM allows it to be used to predict both the local and global spread of COVID-19 by simultaneously modeling spatial interactions between families (high interaction), neighborhoods (medium interaction), and cities (low interaction). By applying MAM to public data from Israel, we could model the actual spread of COVID-19 during Israel’s first vaccination campaign for 9.2 million particles across 1,578 separate communities, where each community has its own vaccination rate. ## RESULTS Most transmissions of the virus leading to infection are in close interactions. These interactions are often cliques, where all individuals in the clique have close interaction with each other (i.e., households). Heterogenous vaccination rates between those cliques bias the estimations of vaccine effectiveness. Consider this simple example: two nursing homes, each with 60 people that all interact with each other. Assuming that in one home 50 people are vaccinated with a vaccine with 80% efficacy and the rest are unvaccinated, while in the other it is the opposite (10 vaccinated). Both homes had an outbreak, in the first house one vaccinated and one unvaccinated were infected, which is what we expect with 80% protection. In the second home 25 unvaccinated and one vaccinated were infected, again translates to 80% protection. However, if we calculate ![Graphic][6] for both homes together (120 people in total), we substantially overestimate ![Graphic][7] at 92.3%, which is 2.6-fold risk reduction. Similar analysis can be performed with secondary infection models to show any desired level of observed ![Graphic][8]. We present here an analytical framework to model this observation. ### Analytical infection model For illustration purposes, we present three possible mixtures described using three networks. In all networks, there is a similar amount of vaccinated and unvaccinated nodes, and there are precisely four close connections for each node (*N**in* *=* 5, *N**out* *=* 20) (**Figure 1A-C**). In each network all the nodes are numbered, such that for the *i**th* node we indicate with vector *A**i* its vaccination status and the number of connections it has with the unvaccinated nodes in the network (see **Supplementary Methods**). The only difference between the three networks is the dispersal of the vaccinated and unvaccinated nodes: in network (A), there is complete segregation between vaccinated and unvaccinated nodes (⟨*I*⟩ = 0). This network might simulate nursing homes that were vaccinated early, while most of the rest of the population was not yet vaccinated. In network (B), for each vaccinated node, half of the edges are to vaccinated nodes and half to unvaccinated nodes (⟨*I*⟩ = 0.5). In network (C), each vaccinated node is connected to one vaccinated node and three unvaccinated nodes (⟨*I*⟩ = 0.75). Network (C) might simulate households where two members are vaccinated (e.g., parents) and three are not, e.g., children. The *A**i* values for each node in the networks are provided in the **Supplementary Methods** (**Table S3**). ![Figure 1.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2023/02/06/2022.01.24.22269737/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2023/02/06/2022.01.24.22269737/F1) Figure 1. Infection networks with different types of interactions between vaccinated and unvaccinated nodes. **A-C**. Orange individuals are vaccinated nodes, blue nodes are unvaccinated nodes. Each circle is a tightly connected clique. Weak interactions with individuals outside of the clique. **D**. ![Graphic][9] as a function of⟨I⟩, for δ = 1, and vE = 0.85: Blue band: ![Graphic][10] for 0.2 < δ < 2; Blue dashed line: **δ = 1**; Solid line: the input vE. For each of the networks we count the average fraction of edges with unvaccinated nodes to the vaccinated nodes, ⟨*I*⟩, and calculate ![Graphic][11] for *VE* = 0.85 and *v* = 0.6 (proportion of the population that are unvaccinated), and *δ* = 1 (ratio between inner clique and other infections, thus 1 represents that half of the infections are household infections). Using the equations described in the analytical infection model, we find that the risk of infection in (B) for vaccinated nodes is 0.072 and 0.52 for unvaccinated nodes, which translates to ![Graphic][12], like *VE*. However, in (A) and (C) we see a diversion from *VE*: in (A) risk for vaccinated is 0.046 and 0.63 for unvaccinated, which translates to an overestimation of almost double reduced risk ![Graphic][13], and in (C) these numbers are 0.1 and 0.4, respectively, which translate to an underestimation of ![Graphic][14] Generally, we observed that for ⟨*I*⟩ around 0.6 *VE* is similar to ![Graphic][15] (**Figure 1D** and Eq (S-18) in the **Supplementary Methods**); **Figure 1D** shows ![Graphic][16] for 0 ≤ ⟨*I*⟩ ≤ 1 and 0.2 ≥ *δ* ≥2. Lower ⟨*I*⟩ levels produce overestimations of ![Graphic][17] up to 95%, and higher ⟨*I*⟩ levels produce underestimations towards 70%. ### Numerical analysis: using MAM to estimate vaccine effectiveness in Israel We next attempted to model the infections occurred in real-world and show how calculations of vaccine effectiveness can be skewed based on different assumptions regarding ⟨*I*⟩. Using MAM, we simulated the spread of COVID-19 infections in Israel from the beginning of the vaccination campaign on December 20, 2020. The particles were numbered and represented by the vector *A**i*, similarly to the analytical model. However, since this is a time-dependent model, and the model predicts the spread of infection while the population is actively being vaccinated, we use *A**i*(*t*) where the vaccination rate is determined by the actual vaccination rates across 1,578 communities in Israel (25,26) (**Figure S1**). We created ten different mixing scenarios between vaccinated and unvaccinated individuals in each community. For each scenario, we created a different array of *A**i*(*t*) according to the real vaccination rates, which differ from each other in their *I**i*(*t*), which represents the level of close interactions of a vaccinated individual *i* with unvaccinated individuals as a function of time *t* (**Supplementary Methods**). We simulated different scenarios of infections throughout February 2021. In each date we calculated the observed vaccine effectiveness, ![Graphic][18], using both the population-based approach (PB) and the secondary infection approach (SI). In the PB-simulated analysis, we performed a simple adjustment to eliminate the heterogeneity in vaccination rates among communities by matching the number of vaccinated and unvaccinated in each community, which can be seen as 1:1 matching. In the SI-simulated analysis we only considered infections within the close-contact circuit, which is similar to a situation of *δ* = 0 in the analytical model (**Supplementary Methods**). Thus, low ⟨*I*⟩ represents a scenario where most close-contacts circuits are partially vaccinated (i.e., a family with only one parent vaccinated), and high ⟨*I*⟩ represents a scenario where most close-contact circuits are either fully vaccinated or not vaccinated at all. In all analyses, we observed clear negative correlations between ![Graphic][19] and ⟨*I*⟩, and ![Graphic][20] ranged between ~70% and ~95% (**Figure 2A**). We can see that the input *VE* was only obtained for ⟨*I*⟩ at levels around 0.5-0.6 for both the PB and SI simulations. However, the PB-based Dagan et al. study (3) probably had an ⟨*I*⟩ of ~0.25, which is in line with the lockdown that was imposed at this time, while the SI-based Gazit et al. study (4) probably had an ⟨*I*⟩ of ~0.75, which is in line with the limited availability of vaccinations to adults only at this time. ![Figure 2.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2023/02/06/2022.01.24.22269737/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2023/02/06/2022.01.24.22269737/F2) Figure 2. Estimations of ![Graphic][21] in different levels of mixing between vaccinated and unvaccinated individuals. **A**. Estimations of ![Graphic][22] as a function of ⟨**I**⟩. Red line: Population-based (PB) simulations with matching of vaccinated and unvaccinated individuals in each region; Green line: Secondary infection-based (SI) simulations; Blue line: Matched PB with additional exclusion of individuals in close-contact circuits with more than 50% vaccination. **B**. ![Graphic][23] estimations for matched-PB as a function of time. Estimations are shown from January 31 to end of February 28 for three mixing scenarios. For both panels, Grey line – input VE =0.85; Grey band: VE within a 5% uncertainty. As we can see, even careful matching would be insufficient to retrieve the actual *VE*. This is due to the additional circuits we added in the modelling that aim to mimic household contacts. This creates heterogeneity of ⟨*I*⟩ at the close contacts-level, not just at the community-level. To overcome this issue, we derived an additional analysis that considers the level of mixing. Since it is not simple to adjust for different ⟨*I*⟩ levels in real-world data for PB analysis, we performed the analysis by including only close-contact circuits with over 50% vaccinated individuals. This analysis showed that for PB, it is possible to achieve relatively accurate estimations of *VE* with this relatively simple approach, regardless of ⟨*I*⟩ and time (**Figure 2A** and **Figure S2**). Of note, in SI analysis, both ⟨*I*⟩ and *v* can be theoretically obtained for each family (and *δ* = 0), therefore, ![Graphic][24] can be adjusted more easily in real-world data (full analysis for ![Graphic][25] for PB and SI is given in the **Supplementary Methods**). Finally, we analyzed how ![Graphic][26] changed over time in the different scenarios (**Figure 2B**). Interestingly, even for ⟨*I*⟩ ≈ 0.5, where the average ![Graphic][27] in February was ~85% (PB method; **Figure 2A**), we observed underestimation in early February and overestimation in late February. In general, we noticed that as vaccination rates increased with time, ![Graphic][28] estimations increased accordingly, in line with early assessments of vaccine effectiveness in Israel compared to later assessments. This temporal relationship between the distribution of ⟨*I*⟩ and the vaccination rate adds another layer to the complexity of vaccine effectiveness estimation and can explain divergent results across studies that were performed in different time points. ## DISCUSSION and CONCLUSIONS We demonstrated here that vaccine effectiveness estimations are sensitive to the dispersion of vaccinations across the population. We found that the level of mixing between vaccinated and unvaccinated individuals can alter the observed vaccine effectiveness. High level of mixing creates indirect protection to unvaccinated individuals and on the other hand increases the risk of infection for vaccinated people, since they frequently interact with unvaccinated people, which in turn leads to underestimation of vaccine effectiveness, while low level of mixing leads to additional protection to the already protected individuals, and in turn to overestimation of vaccine effectiveness. Regardless of what is the real mixing between vaccinated and unvaccinated, which we assume is far from random, it is clear from our analyses that the observed vaccine effectiveness can significantly differ from the real individual-level protection offered by the vaccine. This study is subject to several limitations. First, this work did not consider that part of the population is protected by previous infection (i.e., recovered). However, the infections data from Israel indicates that by November 2020, a relatively low proportion of the population (no more than 5%) had been infected and recovered. It may affect the assumptions and calculations in this work, but this oversight should result in only a minor error, since we assume that some of the recovered population is also vaccinated. Second, it is not simple to estimate the level of mixing in the population. However, we note that during the time of first estimations of vaccine effectiveness coming from Israel (3), Israel was mostly under a lockdown that was imposed between January 8, 2021, and February 9, 2021. Under the lockdown there were limited interactions overall, which enable us to assume relatively low interaction within the population in Israel and especially low number of interactions between the vaccinated elderly population and the rest of the population which were vaccinated later. It is therefore safe to assume that ⟨*I*⟩ at the time of the study was trending towards 0, or at least much lower than later on. Based on this, we argue that the high vaccine effectiveness reported 95% for adults over 70 years old (3) is substantially overestimated. Furthermore, the lower vaccine effectiveness reported for the 40-69 years old group in that study (90%) is a result of the higher mixing rate of that group. Later, Israel imposed “green passports”, which limit entrance of unvaccinated individuals to crowded places. This is another measure that reduces vaccinated-unvaccinated interactions and may again skew towards overestimation of vaccine effectiveness. On the other hand, we argue that study designs of secondary household infections underestimate vaccine effectiveness. Gazit et al. used this approach to estimate vaccine effectiveness in Israel and found vaccine effectiveness to be 80% (4). In households, especially in a country with relatively big families as in Israel, most interactions of vaccinated individuals are with unvaccinated children that were not vaccinated at the time of the study. This translates to high ⟨*I*⟩ levels, and as we show this leads to significant underestimation. We note that some of the lower vaccine effectiveness in household infections can be attributed to the prolonged exposure, and in theory, the vaccination is less effective in such scenario. We are aware that the number of family members in Israel is not uniform, however, since the average number of children in Israel is around 3 (26), assuming five people per household is a reasonable assumption. Our study is not the first to caution about vaccine effectiveness estimations confounded by indirect protection. Previous studies developed a causal inference framework with interference to deal with this issue (19–23). Interestingly, although this issue is well known, in our literature review of vaccine effectiveness studies for COVID-19, we did not identify any study that takes this issue into account. We believe that analytical model provides a novel and relatively simple framework for dealing with vaccine interference, and, as we show here, can be implemented in real-world settings. The analytical model is structured to differentiate between different types of interactions (close and remote), and therefore, better represents real-world infection networks. In conclusion, we argue here that observed vaccine effectiveness estimations of 90%-95% and 80% stems from a vaccine that probably provides individual-level protection of around 85%. We suggest that by adjusting to the fraction of unvaccinated individuals in a household, in addition to demographic and clinical features, it is possible to reduce the bias when estimating vaccine effectiveness. We hope that future studies will consider the possible bias we highlight in this study and attempt to correct for it, as the rate of effectiveness of the vaccines have high impact on strategies for controlling the spread of the pandemic. ## Supporting information Supplementary materials [[supplements/269737_file03.pdf]](pending:yes) ## Data Availability All data are available online [https://www.cbs.gov.il/EN/Pages/default.aspx](https://www.cbs.gov.il/EN/Pages/default.aspx) [https://data.gov.il/dataset/covid-19](https://data.gov.il/dataset/covid-19). ## ACKNOWLEDGMENTS DA is supported by the Azrieli Faculty Fellowship and is a Deloro Fellow. We thank Yaniv Erlich, Barak Rave and Michael Geller for fruitful discussions. ## Footnotes * * E-mail: hdeleon{at}campus.technion.ac.il * † E-mail: dviraran{at}technion.ac.il * All the paper was revised * Received January 24, 2022. * Revision received February 5, 2023. * Accepted February 6, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial 4.0 International), CC BY-NC 4.0, as described at [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) ## REFERENCES 1. 1.Polack FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine. 2020 Dec 31;383(27):2603–15. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2034577&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 2. 2.Baden LR, el Sahly HM, Essink B, Kotloff K, Frey S, Novak R, et al. Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. New England Journal of Medicine. 2021 Feb 4;384(5):403–16. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2035389&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 3. 3.Dagan N, Barda N, Kepten E, Miron O, Perchik S, Katz MA, et al. BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting. New England Journal of Medicine. 2021 Apr 15;384(15):1412–23. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMOA2101765&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 4. 4.Gazit S, Mizrahi B, Kalkstein N, Neuberger A, Peretz A, Mizrahi-Reuveni M, et al. BNT162b2 mRNA Vaccine Effectiveness Given Confirmed Exposure: Analysis of Household Members of Coronavirus Disease 2019 Patients. Clinical Infectious Diseases. 2022 Aug 24;75(1):e734–40. 5. 5.Hunter P, Brainard J. Estimating the effectiveness of the Pfizer COVID-19 BNT162b2 vaccine after a single dose. A reanalysis of a study of ‘real-world’ vaccination outcomes from Israel. 2021. 6. 6.Nanduri S, Pilishvili T, Derado G, Soe MM, Dollard P, Wu H, et al. Effectiveness of Pfizer-BioNTech and Moderna Vaccines in Preventing SARS-CoV-2 Infection Among Nursing Home Residents Before and During Widespread Circulation of the SARS-CoV-2 B.1.617.2 (Delta) Variant — National Healthcare Safety Network, March 1–August 1, 2021. MMWR Morb Mortal Wkly Rep. 2021 Aug 27;70(34):1163–6. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 7. 7.Self WH, Tenforde MW, Rhoads JP, Gaglani M, Ginde AA, Douin DJ, et al. Comparative Effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) Vaccines in Preventing COVID-19 Hospitalizations Among Adults Without Immunocompromising Conditions — United States, March–August 2021. MMWR Morb Mortal Wkly Rep. 2021 Sep 24;70(38):1337–43. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7038e1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34555004&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 8. 8.Self WH, Tenforde MW, Rhoads JP, Gaglani M, Ginde AA, Douin DJ, et al. Comparative Effectiveness of Moderna, Pfizer-BioNTech, and Janssen (Johnson & Johnson) Vaccines in Preventing COVID-19 Hospitalizations Among Adults Without Immunocompromising Conditions — United States, March–August 2021. MMWR Morb Mortal Wkly Rep. 2021 Sep 24;70(38):1337–43. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7038e1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34555004&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 9. 9.Braeye T, Cornelissen L, Catteau L, Haarhuis F, Proesmans K, de Ridder K, et al. Vaccine effectiveness against infection and onwards transmission of COVID-19: Analysis of Belgian contact tracing data, January-June 2021. Vaccine. 2021 Sep;39(39):5456–60. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2021.08.060&link_type=DOI) 10. 10.Lopez Bernal J, Andrews N, Gower C, Gallagher E, Simmons R, Thelwall S, et al. Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant. New England Journal of Medicine. 2021 Aug 12;385(7):585–94. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2108891&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 11. 11.de Gier B, Andeweg S, Joosten R, ter Schegget R, Smorenburg N, van de Kassteele J, et al. Vaccine effectiveness against SARS-CoV-2 transmission and infections among household and other close contacts of confirmed cases, the Netherlands, February to May 2021. Eurosurveillance. 2021 Aug 5;26(31). 12. 12.Braeye T, Cornelissen L, Catteau L, Haarhuis F, Proesmans K, de Ridder K, et al. Vaccine effectiveness against infection and onwards transmission of COVID-19: Analysis of Belgian contact tracing data, January-June 2021. Vaccine. 2021 Sep 15;39(39):5456–60. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2021.08.060&link_type=DOI) 13. 13.Mazagatos C, Monge S, Olmedo C, Vega L, Gallego P, Martín-Merino E, et al. Effectiveness of mRNA COVID-19 vaccines in preventing SARS-CoV-2 infections and COVID-19 hospitalisations and deaths in elderly long-term care facility residents, Spain, weeks 53 2020 to 13 2021. Eurosurveillance. 2021 Jun 17;26(24). 14. 14.Britton A, Slifka KMJ, Edens C, Nanduri SA, Bart SM, Shang N, et al. Effectiveness of the Pfizer-BioNTech COVID-19 vaccine among residents of two skilled nursing facilities experiencing COVID-19 outbreaks—Connecticut, December 2020–February 2021. Morbidity and Mortality Weekly Report. 2021;70(11):396. 15. 15.Dean NE, Hogan JW, Schnitzer ME. Covid-19 Vaccine Effectiveness and the Test-Negative Design. New England Journal of Medicine. 2021 Oct 7;385(15):1431–3. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMe2113151&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34496195&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 16. 16.Nordström P, Ballin M, Nordström A. Association between risk of COVID-19 infection in nonimmune individuals and COVID-19 immunity in their family members. JAMA Intern Med. 2021;181(12):1589–95. 17. 17.Milman O, Yelin I, Aharony N, Katz R, Herzel E, Ben-Tov A, et al. Community-level evidence for SARS-CoV-2 vaccine protection of unvaccinated individuals. Nat Med. 2021 Aug 10;27(8):1367–9. 18. 18.Halloran ME, Hudgens MG. Causal Inference for Vaccine Effects on Infectiousness. Int J Biostat. 2012 Jan 6;8(2):1–40. 19. 19.Hudgens MG, Halloran ME. Toward Causal Inference With Interference. J Am Stat Assoc. 2008 Jun 1;103(482):832–42. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214508000000292&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19081744&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 20. 20.Hudgens MG, Halloran ME. Causal Vaccine Effects on Binary Postinfection Outcomes. J Am Stat Assoc. 2006 Mar;101(473):51–64. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214505000000970&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19096723&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000235958400006&link_type=ISI) 21. 21.Tchetgen EJT, VanderWeele TJ. On causal inference in the presence of interference. Stat Methods Med Res. 2012 Feb 10;21(1):55–75. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0962280210386779&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21068053&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 22. 22.O’Hagan JJ, Lipsitch M, Hernán MA. Estimating the Per-Exposure Effect of Infectious Disease Interventions. Epidemiology. 2014 Jan;25(1):134–8. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/EDE.0000000000000003&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24240656&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) 23. 23.Halloran ME, Hudgens MG. Causal Inference for Vaccine Effects on Infectiousness. Int J Biostat. 2012 Jan 6;8(2):1–40. 24. 24.De-Leon H, Aran D. MAM: Flexible Monte-Carlo Agent based Model for Modelling COVID-19 Spread. medRxiv [Internet]. 2022 Jan 1;2022.09.11.22279815. Available from: [http://medrxiv.org/content/early/2022/09/12/2022.09.11.22279815.abstract](http://medrxiv.org/content/early/2022/09/12/2022.09.11.22279815.abstract) 25. 25.[https://data.gov.il/dataset/covid-19](https://data.gov.il/dataset/covid-19) [Internet]. [cited 2022 Sep 1]. Available from: [https://data.gov.il/dataset/covid-19](https://data.gov.il/dataset/covid-19) 26. 26.[https://www.cbs.gov.il/EN/Pages/default.aspx](https://www.cbs.gov.il/EN/Pages/default.aspx). 27. 27.[https://fs.knesset.gov.il/globaldocs/MMM/f9e58b56-5bbf-ea11-8107-00155d0aee38/2\_f9e58b56-5bbf-ea11-8107-00155d0aee38\_11\_15145.pdf](https://fs.knesset.gov.il/globaldocs/MMM/f9e58b56-5bbf-ea11-8107-00155d0aee38/2\_f9e58b56-5bbf-ea11-8107-00155d0aee38\_11\_15145.pdf). [cited 2023 Jan 24]; Available from: [https://github.com/hdeleon1/Over-and-under-estimation-/blob/main/2\_f9e58b56-5bbf-ea11-8107-00155d0aee38\_11\_15145.pdf](https://github.com/hdeleon1/Over-and-under-estimation-/blob/main/2_f9e58b56-5bbf-ea11-8107-00155d0aee38_11_15145.pdf) 28. 28.[https://fs.knesset.gov.il/globaldocs/MMM/f9e58b56-5bbf-ea11-8107-00155d0aee38/2\_f9e58b56-5bbf-ea11-8107-00155d0aee38\_11\_15145.pdf](https://fs.knesset.gov.il/globaldocs/MMM/f9e58b56-5bbf-ea11-8107-00155d0aee38/2_f9e58b56-5bbf-ea11-8107-00155d0aee38_11_15145.pdf). 29. 29.De-Leon H, Pederiva F. Particle modeling of the spreading of coronavirus disease (COVID-19). Physics of Fluids. 2020;32(8):087113. 30. 30.De-Leon H, Pederiva F. Statistical mechanics study of the introduction of a vaccine against covid-19 disease. Phys Rev E. 2021;104(1):014132. 31. 31.Ji C, Jiang D. Threshold behaviour of a stochastic SIR model. Appl Math Model. 2014;38(21–22):5067–79. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.apm.2014.03.037&link_type=DOI) 32. 32.Aron JL, Schwartz IB. Seasonality and period-doubling bifurcations in an epidemic model. J Theor Biol. 1984;110(4):665–79. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0022-5193(84)80150-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=6521486&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1984TT57500010&link_type=ISI) 33. 33.Kermack WO, McKendrick AG. A contribution to the mathematical theory of epidemics. Proceedings of the royal society of london Series A, Containing papers of a mathematical and physical character. 1927;115(772):700–21. 34. 34.De-Leon H, Calderon-Margalit R, Pederiva F, Ashkenazy Y, Gazit D. First indication of the effect of COVID-19 vaccinations on the course of the outbreak in Israel. medRxiv. 2021; 35. 35.Caspi G, Dayan A, Eshal Y, Liverant-Taub S, Twig G, Shalit U, et al. Socioeconomic disparities and COVID-19 vaccination acceptance: a nationwide ecologic study. Clinical Microbiology and Infection. 2021 Oct;27(10):1502–6. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F02%2F06%2F2022.01.24.22269737.atom) [1]: /embed/inline-graphic-1.gif [2]: /embed/graphic-1.gif [3]: /embed/inline-graphic-2.gif [4]: /embed/inline-graphic-3.gif [5]: /embed/inline-graphic-4.gif [6]: /embed/inline-graphic-5.gif [7]: /embed/inline-graphic-6.gif [8]: /embed/inline-graphic-7.gif [9]: F1/embed/inline-graphic-8.gif [10]: F1/embed/inline-graphic-9.gif [11]: /embed/inline-graphic-10.gif [12]: /embed/inline-graphic-11.gif [13]: /embed/inline-graphic-12.gif [14]: /embed/inline-graphic-13.gif [15]: /embed/inline-graphic-14.gif [16]: /embed/inline-graphic-15.gif [17]: /embed/inline-graphic-16.gif [18]: /embed/inline-graphic-17.gif [19]: /embed/inline-graphic-18.gif [20]: /embed/inline-graphic-19.gif [21]: F2/embed/inline-graphic-20.gif [22]: F2/embed/inline-graphic-21.gif [23]: F2/embed/inline-graphic-22.gif [24]: /embed/inline-graphic-23.gif [25]: /embed/inline-graphic-24.gif [26]: /embed/inline-graphic-25.gif [27]: /embed/inline-graphic-26.gif [28]: /embed/inline-graphic-27.gif