Skip to main content
Kent Academic Repository

Information theoretical properties of a spiking neuron trained with Hebbian and STDP learning rules

Chu, Dominique (2023) Information theoretical properties of a spiking neuron trained with Hebbian and STDP learning rules. Natural Computing, . ISSN 1567-7818. E-ISSN 1572-9796. (doi:10.1007/s11047-022-09939-6) (KAR id:99451)

Abstract

Using formal methods complemented by large-scale simulations we investigate information theoretical properties of spiking neurons trained using Hebbian and STDP learning rules. It is shown that weight space contains meta-stable states, which are points where the average weight change under the learning rule vanishes. These points may capture the random walker transiently. The dwell time in the vicinity of the meta-stable state is either quasi-infinite or very short and depends on the level of noise in the system. Moreover, important information theoretic quantities, such as the amount of information the neuron transmits are determined by the meta-stable state. While the Hebbian learning rule reliably leads to meta-stable states, the STDP rule tends to be unstable in the sense that for most choices of hyper-parameters the weights are not captured by meta-stable states, except for a restricted set of choices. It emerges that stochastic fluctuations play an important role in determining which meta-stable state the neuron takes. To understand this, we model the trajectory of the neuron through weight space as an inhomogeneous Markovian random walk, where the transition probabilities between states are determined by the statistics of the input signal.

Item Type: Article
DOI/Identification number: 10.1007/s11047-022-09939-6
Additional information: For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
Uncontrolled keywords: Hebbian learning, Spike-timing dependent plasticity, Stochastic systems, Random walk
Subjects: Q Science > Q Science (General) > Q335 Artificial intelligence
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Funders: Engineering and Physical Sciences Research Council (https://ror.org/0439y7842)
Depositing User: Dominique Chu
Date Deposited: 09 Jan 2023 11:08 UTC
Last Modified: 27 Feb 2024 10:56 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/99451 (The current URI for this page, for reference purposes)

University of Kent Author Information

  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.