Guided Activity Prediction for Minimally Invasive Surgery Safety Improvement in the Internet of Medical Things

Wang, H; Ding, S; Yang, S; Liu, C; Yu, S; Zheng, X

Guided Activity Prediction for Minimally Invasive Surgery Safety Improvement in the Internet of Medical Things

Wang, H Ding, S Yang, S Liu, C Yu, S

Zheng, X

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Internet of Things Journal, 2022, 9, (6), pp. 4758-4768
Issue Date:: 2022-03-15

Closed Access

	Filename	Description	Size
	Guided_Activity_Prediction_for_Minimally_Invasive_Surgery_Safety_Improvement_in_the_Internet_of_Medical_Things.pdf	Published version	2.04 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wang, H
dc.contributor.author	Ding, S
dc.contributor.author	Yang, S
dc.contributor.author	Liu, C
dc.contributor.author	Yu, S https://orcid.org/0000-0003-4485-6743
dc.contributor.author	Zheng, X
dc.date.accessioned	2023-04-05T04:42:34Z
dc.date.available	2023-04-05T04:42:34Z
dc.date.issued	2022-03-15
dc.identifier.citation	IEEE Internet of Things Journal, 2022, 9, (6), pp. 4758-4768
dc.identifier.issn	2327-4662
dc.identifier.issn	2327-4662
dc.identifier.uri	http://hdl.handle.net/10453/169191
dc.description.abstract	With the application of the Internet of Medical Things (IoMT) in minimally invasive surgery (MIS), surgeons now have a better chance at hard-to-treat cases by carrying out more complicated MIS workflows. However, a scheduled surgical workflow is often required to be updated based on the patient's internal tissue states. Perioperative complications could occur if in-time adjustments are lacking in the operating rooms when needed. To help manage the uncertainty of live surgical workflows in the IoMT environment, we propose a MIS safety improvement framework. It helps surgeons in predicting surgical workflows with limited MIS video frames by embedding our proposed model GuidedNet. To predict future surgical activities, we first build three isomorphic neural networks to capture the spatiotemporal information. Then, we establish a guidance fusion module to handle the contextual information. It guides the GuidedNet to recognize the surgical stage. Moreover, we build a novel joint loss function to train the GuidedNet to predict the future surgical stage. We evaluate the approach on a large data set that contains 80 cholecystectomy videos (Cholec-80) and compare it with the state of the art. Experiments show that the GuidedNet can assist surgeons in carrying out MIS as well as guide the next stage of surgery for improving surgical safety. Comparing to the state of the art, our approach can obtain better predict accuracy (up to 79%) with less computing resource consumption. The result also shows that our approach has a high application prospect in video classification in other Internet of Things scenarios.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Internet of Things Journal
dc.relation.isbasedon	10.1109/JIOT.2021.3108457
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0805 Distributed Computing, 1005 Communications Technologies
dc.title	Guided Activity Prediction for Minimally Invasive Surgery Safety Improvement in the Internet of Medical Things
dc.type	Journal Article
utslib.citation.volume	9
utslib.for	0805 Distributed Computing
utslib.for	1005 Communications Technologies
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2023-04-05T04:42:32Z
pubs.issue	6
pubs.publication-status	Published
pubs.volume	9
utslib.citation.issue	6

Abstract:

With the application of the Internet of Medical Things (IoMT) in minimally invasive surgery (MIS), surgeons now have a better chance at hard-to-treat cases by carrying out more complicated MIS workflows. However, a scheduled surgical workflow is often required to be updated based on the patient's internal tissue states. Perioperative complications could occur if in-time adjustments are lacking in the operating rooms when needed. To help manage the uncertainty of live surgical workflows in the IoMT environment, we propose a MIS safety improvement framework. It helps surgeons in predicting surgical workflows with limited MIS video frames by embedding our proposed model GuidedNet. To predict future surgical activities, we first build three isomorphic neural networks to capture the spatiotemporal information. Then, we establish a guidance fusion module to handle the contextual information. It guides the GuidedNet to recognize the surgical stage. Moreover, we build a novel joint loss function to train the GuidedNet to predict the future surgical stage. We evaluate the approach on a large data set that contains 80 cholecystectomy videos (Cholec-80) and compare it with the state of the art. Experiments show that the GuidedNet can assist surgeons in carrying out MIS as well as guide the next stage of surgery for improving surgical safety. Comparing to the state of the art, our approach can obtain better predict accuracy (up to 79%) with less computing resource consumption. The result also shows that our approach has a high application prospect in video classification in other Internet of Things scenarios.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/169191