Skip to main content
Kent Academic Repository

Augmented Reality Applications for Image-Guided Robotic Interventions using deep learning algorithms

Seetohul, Jenna, Shafiee, Mahmood, Sirlantzis, Konstantinos (2022) Augmented Reality Applications for Image-Guided Robotic Interventions using deep learning algorithms. In: The 3rd International Conference on Medical Image and Computer-Aided Diagnosis. . , UK (In press) (Access to this publication is currently restricted. You may be able to access a copy if URLs are provided) (KAR id:97255)

PDF Author's Accepted Manuscript
Language: English

Restricted to Repository staff only
Contact us about this Publication
[thumbnail of 358.pdf]

Abstract

A significant breakthrough in the field of surgery has seen the integration of augmented reality (AR) in standard robot operations, allowing anatomical objects to be digitalized and overlaid onto a real-life scenario during or pre-intervention. This paper provides an overview of the methodology used to reconstruct and register laparoscopic head and neck image sequences for an AR tool. Deep learning (DL) algorithms are designed to strategically place fiducial markers or labels in a dataset, hence enabling a virtual tool path to be set up for guiding the end effector of a robot. We introduce a dataset of 271 images of patients from four different clinics in Quebec with a proven history of head-and-neck cancer. We then propose a marker-based registration method for mapping a trajectory during surgery, utilizing an unsupervised neural network for computing the medical image transformations. During the training stage, we use an optimized convolutional neural network (CNN) that warps a set of labels from the moving image in contrast with the related counterparts in the fixed image. To this end, we compare the loss functions between warped moving labels and fixed labels with respect to the ground truth in the method. Next, we propose a UNet architecture where we measure the accuracies in label localization throughout the test sequences relative to the initial output results. Our experiments showed that the UNet outperformed the initial CNN architecture, with optimum performance outcomes in losses being closer to 1.0.

Item Type: Conference or workshop item (Paper)
Uncontrolled keywords: Augmented Reality, Image Registration, Path planning, Supervised Learning
Subjects: Q Science > Q Science (General) > Q335 Artificial intelligence
T Technology > TA Engineering (General). Civil engineering (General) > TA165 Engineering instruments, meters etc. Industrial instrumentation
T Technology > TJ Mechanical engineering and machinery
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts
Depositing User: Mahmood Shafiee
Date Deposited: 01 Oct 2022 16:49 UTC
Last Modified: 08 Jun 2023 09:02 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/97255 (The current URI for this page, for reference purposes)

University of Kent Author Information

Shafiee, Mahmood.

Creator's ORCID: https://orcid.org/0000-0002-6122-5719
CReDIT Contributor Roles:

Sirlantzis, Konstantinos.

Creator's ORCID: https://orcid.org/0000-0002-0847-8880
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.