Multi-LRA: Multi logical residual architecture for spiking neural networks

Shen, H; Wang, H; Ma, Y; Li, L; Duan, S; Wen, S

Multi-LRA: Multi logical residual architecture for spiking neural networks

Shen, H Wang, H Ma, Y Li, L Duan, S Wen, S

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Information Sciences, 2024, 660, pp. 120136
Issue Date:: 2024-03-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0020025524000495-main.pdf	Published version	1.06 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Shen, H
dc.contributor.author	Wang, H
dc.contributor.author	Ma, Y
dc.contributor.author	Li, L
dc.contributor.author	Duan, S
dc.contributor.author	Wen, S https://orcid.org/0000-0001-8077-7001
dc.date.accessioned	2024-02-01T01:24:51Z
dc.date.available	2024-02-01T01:24:51Z
dc.date.issued	2024-03-01
dc.identifier.citation	Information Sciences, 2024, 660, pp. 120136
dc.identifier.issn	0020-0255
dc.identifier.uri	http://hdl.handle.net/10453/175186
dc.description.abstract	Recently, it has been noticed that certain current state-of-the-art (SOTA) spiking neural networks incorporated non-spike data by use of residual connections and count coding. However, these architectures may not be well-suited for the neuromorphic chips optimized primarily for processing binary inputs and floating-point weights. On the other hand, some pure-spike structures, such as SEW-AND and SEW-IAND, have been found to exhibit spike vanishing and limited capacity. To overcome these shortcomings, we analyze the energy consumption and expressive ability of binary and integer inputs on neuromorphic chips, and then propose a new Multi Logical Residual Architecture (Multi-LRA), which eliminates non-spike computation on neuromorphic chips and addresses the issues of spike vanishing and limited capacity in SEW-AND and SEW-IAND. Furthermore, we prove that the upper bound of conditional entropy of Multi-LRA is higher than that of SEW-AND and logical-OR, which means that Multi-LRA has better capacity and may avoid the spike vanishing. Finally, experiments on CIFAR-10 have shown that Multi-LRA can significantly reduce energy consumption because of its pure-spike computation and logical operation, where Multi-LRA-ResNet50 can reach the SOTA accuracy 96.27% in just 4 time steps. In addition, the effectiveness of Multi-LRA has been validated on ImageNet, CIFAR10-DVS and DVS128 Gesture.
dc.language	en
dc.publisher	Elsevier
dc.relation.ispartof	Information Sciences
dc.relation.isbasedon	10.1016/j.ins.2024.120136
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	01 Mathematical Sciences, 08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.subject.classification	49 Mathematical sciences
dc.title	Multi-LRA: Multi logical residual architecture for spiking neural networks
dc.type	Journal Article
utslib.citation.volume	660
utslib.for	01 Mathematical Sciences
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2024-02-01T01:24:50Z
pubs.publication-status	Accepted
pubs.volume	660

Abstract:

Recently, it has been noticed that certain current state-of-the-art (SOTA) spiking neural networks incorporated non-spike data by use of residual connections and count coding. However, these architectures may not be well-suited for the neuromorphic chips optimized primarily for processing binary inputs and floating-point weights. On the other hand, some pure-spike structures, such as SEW-AND and SEW-IAND, have been found to exhibit spike vanishing and limited capacity. To overcome these shortcomings, we analyze the energy consumption and expressive ability of binary and integer inputs on neuromorphic chips, and then propose a new Multi Logical Residual Architecture (Multi-LRA), which eliminates non-spike computation on neuromorphic chips and addresses the issues of spike vanishing and limited capacity in SEW-AND and SEW-IAND. Furthermore, we prove that the upper bound of conditional entropy of Multi-LRA is higher than that of SEW-AND and logical-OR, which means that Multi-LRA has better capacity and may avoid the spike vanishing. Finally, experiments on CIFAR-10 have shown that Multi-LRA can significantly reduce energy consumption because of its pure-spike computation and logical operation, where Multi-LRA-ResNet50 can reach the SOTA accuracy 96.27% in just 4 time steps. In addition, the effectiveness of Multi-LRA has been validated on ImageNet, CIFAR10-DVS and DVS128 Gesture.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/175186