Automatic quantitative stroke severity assessment from Chinese electronic health records based on advanced large language models

Gu, Zhanzhong

Automatic quantitative stroke severity assessment from Chinese electronic health records based on advanced large language models

Gu, Zhanzhong

Permalink

Publication Type:: Thesis
Issue Date:: 2024

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download thesisAdobe PDF (2.65 MB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Gu, Zhanzhong
dc.date.accessioned	2024-12-12T02:05:44Z
dc.date.available	2024-12-12T02:05:44Z
dc.date.issued	2024
dc.identifier.uri	http://hdl.handle.net/10453/182491
dc.description	University of Technology Sydney. Faculty of Engineering and Information Technology.	en_US.UTF-8
dc.description.abstract	Stroke is one of the leading causes of death worldwide. Accurate assessment of stroke severity plays a pivotal role in precise diagnosis, development of treatment plans, and efficient allocation of healthcare resources. The assessment of stroke in hospitals is usually conducted manually by clinicians. However, it is labor-intensive, time-consuming, and sometimes unreliable. With the continuous development of artificial intelligence (AI) techniques in recent years, applying them to automate clinical assessment in EHRs has attracted much interest. In this thesis, we outline an innovation pathway to advance stroke healthcare from ontology construction, clinical named entity recognition (CNER) with pre-training, to LLM-driven automatic quantitative stroke assessment. The journey begins with the development of “StrokePEO”, a stroke clinical ontology co-designed with specialists using advanced natural language processing (NLP) and deep learning techniques. StrokePEO successfully represents clinical terms and relationships in stroke assessment, demonstrating applicability in diverse medical contexts. Building on this foundation, we develop a deep learning-based framework to automatically assess stroke severity through Chinese CNER and domain-adaptive pre-training. We first construct a new dataset “Chinese Stroke Clinical Records” (CSCR) and pre-train a Chinese clinical embedding “CliRoberta” for CNER. Then, a dictionary-based mapping method is developed to map CNER results into quantitative scores. Comprehensive experiments demonstrate the effectiveness and reliability of the CNER model with our domain-adaptive pre-training. Ultimately, our automatic NIHSS scoring approach achieves excellent inter-rater agreement and intra-class consistency with the ground truth, with significantly improved efficiency. We further advance toward cutting-edge LLMs with a prompting paradigm “GAPrompt” to empower the generic LLMs to assess diagnostic notes and generate quantitative evaluation results. GAPrompt assesses the suitability of LLMs for specific tasks through prompting for LLM selection, facilitates their comprehension of task-specific knowledge derived from the constructed knowledge base, enhances the accuracy of knowledge retrieval and demonstration through summary-based generation-augmented retrieval (SGAR), improves LLM inference precision via hierarchical chain-of-thought (HCoT), strengthens generation robustness, and mitigates LLM hallucinations through ensembling. Experimental findings underscore the effectiveness of our approach in empowering LLMs to achieve automated stroke assessment based on EHRs. Collectively, these works contribute integrative and innovative AI-driven solutions for stroke healthcare, shifting from traditional methods to state-of-the-art LLM techniques, addressing knowledge representation, automated assessment, and quantitative analysis, with broad applications in medical research and practice.	en_US.UTF-8
dc.format	Thesis (PhD)
dc.language.iso	en_US	en_US.UTF-8
dc.relation	https://opus.lib.uts.edu.au/bitstream/10453/182491/1/thesis.pdf
dc.rights	info:eu-repo/semantics/openAccess
dc.rights	The author owns the copyright in this thesis including all reproduction and reuse rights for the work. The work may not be altered without the permission of the copyright owner. Attribution is essential when quoting or paraphrasing from this thesis.
dc.rights	© 2024 Zhanzhong Gu
dc.rights	au.edu.uts.lib/cph
dc.title	Automatic quantitative stroke severity assessment from Chinese electronic health records based on advanced large language models	en_US.UTF-8
dc.type	Thesis
utslib.copyright.status	open_access	*

Abstract:

Stroke is one of the leading causes of death worldwide. Accurate assessment of stroke severity plays a pivotal role in precise diagnosis, development of treatment plans, and efficient allocation of healthcare resources. The assessment of stroke in hospitals is usually conducted manually by clinicians. However, it is labor-intensive, time-consuming, and sometimes unreliable. With the continuous development of artificial intelligence (AI) techniques in recent years, applying them to automate clinical assessment in EHRs has attracted much interest. In this thesis, we outline an innovation pathway to advance stroke healthcare from ontology construction, clinical named entity recognition (CNER) with pre-training, to LLM-driven automatic quantitative stroke assessment. The journey begins with the development of “StrokePEO”, a stroke clinical ontology co-designed with specialists using advanced natural language processing (NLP) and deep learning techniques. StrokePEO successfully represents clinical terms and relationships in stroke assessment, demonstrating applicability in diverse medical contexts. Building on this foundation, we develop a deep learning-based framework to automatically assess stroke severity through Chinese CNER and domain-adaptive pre-training. We first construct a new dataset “Chinese Stroke Clinical Records” (CSCR) and pre-train a Chinese clinical embedding “CliRoberta” for CNER. Then, a dictionary-based mapping method is developed to map CNER results into quantitative scores. Comprehensive experiments demonstrate the effectiveness and reliability of the CNER model with our domain-adaptive pre-training. Ultimately, our automatic NIHSS scoring approach achieves excellent inter-rater agreement and intra-class consistency with the ground truth, with significantly improved efficiency. We further advance toward cutting-edge LLMs with a prompting paradigm “GAPrompt” to empower the generic LLMs to assess diagnostic notes and generate quantitative evaluation results. GAPrompt assesses the suitability of LLMs for specific tasks through prompting for LLM selection, facilitates their comprehension of task-specific knowledge derived from the constructed knowledge base, enhances the accuracy of knowledge retrieval and demonstration through summary-based generation-augmented retrieval (SGAR), improves LLM inference precision via hierarchical chain-of-thought (HCoT), strengthens generation robustness, and mitigates LLM hallucinations through ensembling. Experimental findings underscore the effectiveness of our approach in empowering LLMs to achieve automated stroke assessment based on EHRs. Collectively, these works contribute integrative and innovative AI-driven solutions for stroke healthcare, shifting from traditional methods to state-of-the-art LLM techniques, addressing knowledge representation, automated assessment, and quantitative analysis, with broad applications in medical research and practice.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/182491