Abstract
Background A thorough and in-depth examination of generative artificial intelligence (GAI) usage guidelines in medical journals will inform potential gaps and promote proper GAI usage in scholarly publishing. This study aims to examine the provision and specificity of GAI usage guidelines and their relationships with journal characteristics.
Methods From the SCImago Journal Rank (SJR) list for medicine in 2022, we selected 98 journals as top journals to represent highly indexed journals and 144 as whole-spectrum sample journals to represent all medical journals. We examined their GAI usage guidelines for scholarly publishing between December 2023 and January 2024.
Results Compared to whole-spectrum sample journals, the top journals were more likely to provide author guidelines (64.3% vs. 27.8%) and reviewer guidelines (11.2% vs. 0.0%) as well as refer to external guidelines (85.7% vs 74.3%). Probit models showed that SJR score or region was not associated with the provision of these guidelines among top journals. However, among whole-spectrum sample journals, SJR score was positively associated with the provision of author guidelines (0.85, 95% CI 0.49 to 1.25) and references to external guidelines (2.01, 95% CI 1.24 to 3.65). Liner models showed that SJR score was positively associated with the specificity level of author and reviewer guidelines among whole-spectrum sample journals (1.21, 95% CI 0.72 to 1.70), and no such pattern was observed among top journals.
Conclusions The provision of GAI usage guidelines is limited across medical journals, especially for reviewer guidelines. The lack of specificity and consistency in existing guidelines highlights areas deserving improvement. These findings suggest that immediate attention is needed to guide GAI usage in scholarly publishing in medical journals.
Question What are the provision and specificity of generative artificial intelligence (GAI) usage guidelines for scholarly publishing in top and whole-spectrum medical journals and their relationships with journal characteristics?
Findings Author guidelines were more abundant and specific in top journals than in whole-spectrum journals. However, reviewer guidelines were extremely scarce in both groups of journals. Journal ranking score was associated with both provision and specificity of GAI usage guidelines in whole-spectrum journals while no significant relationship was found in top journals.
Meaning The lack of provision and specificity as well as the inconsistencies in existing guidelines suggest that immediate attention is needed to guide GAI usage in scholarly publishing and safeguard integrity and trust in medical research.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
None
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data relevant to the study are included in the article or included as supplementary materials. Full data and analytical code are available from the corresponding author at chihuali{at}umich.edu.
