Abstract
Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians’ records, it has yet to be widely applied to pharmaceutical care records.
In this report, we investigated the feasibility of automatic extraction of patients’ diseases and symptoms from pharmaceutical care records. The verification was performed using MedNER-J, a Japanese disease-extraction system designed for physicians’ records.
MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F-measure.
The F-measure of NER for subjective, objective, assessment, and plan data was 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive–negative classification, the F-measure was 0.28, 0.39, 0.64, and 0.077, respectively. The F-measure of NER for objective and assessment data (F=0.70, 0.76) was higher than that for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F-measure of NER and positive-negative classification was high for assessment data alone (F=0.64), which was attributed to the similarity of its description format and contents to those of the training data.
MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.
Competing Interest Statement
I have read the journal's policy and the authors of this manuscript have the following competing interests: Author MM is an Audit & Supervisory Board Member(Outside) of Sumitomo Pharma. The other authors have declared that no competing interests exist.
Funding Statement
YO received a research grant from JST SPRING, Grant Number JPMJSP2123. For more information on JST SPRING, please visit https://www.jst.go.jp/jisedai/. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee of the Keio University School of Medicine gave ethical approval for this work
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Data of pharmaceutical care records cannot be shared publicly due to an arrangement with the Ethics Committee. Data requests may be sent to the corresponding author.
