Mohamed, Elhassan, Sirlantzis, Konstantinos, Howells, Gareth (2022) A review of visualisation-as-explanation techniques for convolutional neural networks and their evaluation. Displays, . ISSN 0141-9382. (KAR id:95158)
PDF
Author's Accepted Manuscript
Language: English
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
|
|
Download this file (PDF/2MB) |
Preview |
Request a format suitable for use with assistive technology e.g. a screenreader | |
Official URL: https://www.sciencedirect.com/science/article/pii/... |
Abstract
Visualisation techniques are powerful tools to understand the behaviour of Artificial Intelligence (AI) systems. They can be used to identify important features contributing to the network decisions, investigate biases in datasets, and find weaknesses in the system's structure (e.g., network architectures). Lawmakers and regulators may not allow the use of smart systems if these systems cannot explain the logic underlying a decision or action taken. These systems are required to offer a high level of 'transparency' to be approved for deployment. Model transparency is vital for safety-critical applications such as autonomous navigation and operation systems (e.g., autonomous trains or cars), where prediction errors may have serious implications. Thus, being highly accurate without explaining the basis of their performance is not enough to satisfy regulatory requirements. The lack of system interpretability is a major obstacle to the wider adoption of AI in safety-critical applications. Explainable Artificial Intelligence (XAI) techniques applied to intelligent systems to justify their decisions offers a possible solution. In this review, we present state-of-the-art explanation techniques in detail. We focus our presentation and critical discussion on visualisation methods for the most adopted architecture in use, the Convolutional Neural Networks (CNNs), applied to the domain of image classification. Further, we discuss the evaluation techniques for different explanation methods, which shows that some of the most visually appealing methods are unreliable and can be considered a simple feature or edge detector. In contrast, robust methods can give insights into the model behaviour, which helps to enhance the model performance and boost the confidence in the model's predictions. Besides, the applications of XAI techniques show their importance in many fields such as medicine and industry. We hope that this review proves a valuable contribution for researchers in the field of XAI.
Item Type: | Article |
---|---|
Uncontrolled keywords: | Activation heatmapsArchitecture understandingBlack-box representationsCNN visualisationConvolutional neural networksExplainable AIFeature visualisationInterpretable neural networksSaliency mapsXAI |
Divisions: | Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts |
Depositing User: | Elhassan Mohamed |
Date Deposited: | 23 May 2022 12:09 UTC |
Last Modified: | 05 Nov 2024 13:00 UTC |
Resource URI: | https://kar.kent.ac.uk/id/eprint/95158 (The current URI for this page, for reference purposes) |
- Link to SensusAccess
- Export to:
- RefWorks
- EPrints3 XML
- BibTeX
- CSV
- Depositors only (login required):