ABSTRACT
Introduction Increasing concerns about integrity in medical research have prompted the development of tools to detect untrustworthy studies. Existing tools focus on evaluating aggregate or published data, though some trustworthiness issues may only be detected upon scrutiny of individual participant data (IPD). To address this, we developed the IPD Integrity Tool for detecting integrity issues in randomised controlled trials with IPD available. This manuscript describes the development of this tool.
Methods We conducted a literature review to collate and map existing integrity items. These were discussed with an expert advisory group, and agreed items were included in a standardised tool and automated where possible. We piloted this tool in two IPD meta-analyses, and conducted preliminary validation checks on 13 datasets with and without known integrity issues in a blinded manner.
Results The literature review identified 120 integrity items: 54 could be conducted at the publication or aggregate data (AD) level, 48 required IPD, and 18 were possible with aggregate data, but more comprehensive with IPD. Based on these items, an initial reduced tool was developed in a consensus process involving 13 advisors with different backgrounds (countries, profession, education). This initial tool included 11 items across four domains for AD, and 12 items across 8 domains requiring IPD. The tool was iteratively refined throughout piloting on two IPD meta-analyses including a total of 116 trials (73 with IPD, and 43 with only AD available), and preliminary validation using an additional 13 datasets. All five studies with known integrity issues were accurately identified during validation. The final version of the tool included seven domains with 13 items for AD and eight domains with 18 items requiring IPD.
Conclusions The quality of evidence informing health care relies on trustworthy data. This manuscript describes the development of a tool to enable researchers, editors, and other stakeholders to detect integrity issues in randomised trials using IPD. Detailed instructions on the application of this tool will be published subsequently.
Competing Interest Statement
The authors declare the following competing interests: KEH receives research funding support via two scholarships administered by the University of Sydney (Postgraduate Research Supplementary Scholarship in Methods Development (SC3504), and Research Training Program Stipend (SC3227)). RW is supported by an NHMRC Investigator grant (GNT2009767). WL is supported by an NHMRC Investigator grant (GNT2016729). BWM is supported by an NHMRC Investigator grant (GNT1176437), receives research funding and travel support from Merck KGaA, has stocks from ObsEva, and consults for Merck KGaA, Organon and Norgine. ALS Is supported by an NHMRC Investigator Grant (GNT2009432). All other authors (MA, JXS, JGW, JA, AB, NS, ACW) have no competing interests. The named funders had no role in the conceptualisation, design, data collection, analysis, decision to publish, or preparation of the manuscript.
Funding Statement
This study did not receive any funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
