Skip to main content
Kent Academic Repository

A review of visualisation-as-explanation techniques for convolutional neural networks and their evaluation

Mohamed, Elhassan, Sirlantzis, Konstantinos, Howells, Gareth (2022) A review of visualisation-as-explanation techniques for convolutional neural networks and their evaluation. Displays, . ISSN 0141-9382. (KAR id:95158)

Abstract

Visualisation techniques are powerful tools to understand the behaviour of Artificial Intelligence (AI) systems. They can be used to identify important features contributing to the network decisions, investigate biases in datasets, and find weaknesses in the system's structure (e.g., network architectures). Lawmakers and regulators may not allow the use of smart systems if these systems cannot explain the logic underlying a decision or action taken. These systems are required to offer a high level of 'transparency' to be approved for deployment. Model transparency is vital for safety-critical applications such as autonomous navigation and operation systems (e.g., autonomous trains or cars), where prediction errors may have serious implications. Thus, being highly accurate without explaining the basis of their performance is not enough to satisfy regulatory requirements. The lack of system interpretability is a major obstacle to the wider adoption of AI in safety-critical applications. Explainable Artificial Intelligence (XAI) techniques applied to intelligent systems to justify their decisions offers a possible solution. In this review, we present state-of-the-art explanation techniques in detail. We focus our presentation and critical discussion on visualisation methods for the most adopted architecture in use, the Convolutional Neural Networks (CNNs), applied to the domain of image classification. Further, we discuss the evaluation techniques for different explanation methods, which shows that some of the most visually appealing methods are unreliable and can be considered a simple feature or edge detector. In contrast, robust methods can give insights into the model behaviour, which helps to enhance the model performance and boost the confidence in the model's predictions. Besides, the applications of XAI techniques show their importance in many fields such as medicine and industry. We hope that this review proves a valuable contribution for researchers in the field of XAI.

Item Type: Article
Uncontrolled keywords: Activation heatmapsArchitecture understandingBlack-box representationsCNN visualisationConvolutional neural networksExplainable AIFeature visualisationInterpretable neural networksSaliency mapsXAI
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts
Depositing User: Elhassan Mohamed
Date Deposited: 23 May 2022 12:09 UTC
Last Modified: 05 Nov 2024 13:00 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/95158 (The current URI for this page, for reference purposes)

University of Kent Author Information

  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.