Abstract
Introduction A series of modelling reports that quantify the effect of non-pharmaceutical interventions (NPIs) on the spread of the SARS-CoV-2 virus have been made available prior to external scientific peer-review. The aim of this study was to investigate the method used by the Imperial College COVID-19 Research Team (ICCRT) for estimation of NPI effects from the system theoretical viewpoint of model identifiability.
Methods An input-sensitivity analysis was performed by running the original software code of the systems model that was devised to estimate the impact of NPIs on the reproduction number of the SARS-CoV-2 infection and presented online by ICCRT in Report 13 on March 30 2020. An empirical investigation was complemented by an analysis of practical parameter identifiability, using an estimation theoretical framework.
Results Despite being simplistic with few free parameters, the system model was found to suffer from severe input sensitivities. Our analysis indicated that the model lacks practical parameter identifiability from data. The analysis also showed that this limitation is fundamental, and not something readily resolved should the model be driven with data of higher reliability.
Discussion Reports based on system models have been instrumental to policymaking during the SARS-CoV-2 pandemic. With much at stake during all phases of a pandemic, we conclude that it is crucial to thoroughly scrutinise any SARS-CoV-2 effect analysis or prediction model prior to considering its use as decision support in policymaking. The enclosed example illustrates what such a review might reveal.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was partially supported by the ELLIIT Strategic Research Area, by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and by the Swedish Foundation for Strategic Research (SSF) via the project ASSEMBLE (contract number: RIT15-0012).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
N/A
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Manuscript writing and reviewing: Bo Bernhardsson, Örjan Dahlström, Joakim Ekberg, Joakim Jaldén, Anna Jöud, Thomas B. Schön, Kristian Soltesz, Armin Spreco, Toomas Timpka
Data Availability
All source code and data for full and completely transparent reproduction of all our results is available through https://github.com/albheim/covid19model
