Pruning and quantization algorithm with applications in memristor-based convolutional neural network

Guo, M; Sun, Y; Zhu, Y; Han, M; Dou, G; Wen, S

Pruning and quantization algorithm with applications in memristor-based convolutional neural network

Guo, M Sun, Y Zhu, Y Han, M Dou, G Wen, S

Permalink

Publisher:: Springer Nature
Publication Type:: Journal Article
Citation:: Cognitive Neurodynamics, 2023, pp. 1-13
Issue Date:: 2023-01-01

Closed Access

	Filename	Description	Size
	s11571-022-09927-7.pdf	Published version	2.58 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Guo, M
dc.contributor.author	Sun, Y
dc.contributor.author	Zhu, Y
dc.contributor.author	Han, M
dc.contributor.author	Dou, G
dc.contributor.author	Wen, S https://orcid.org/0000-0001-8077-7001
dc.date.accessioned	2023-03-13T12:51:58Z
dc.date.available	2023-03-13T12:51:58Z
dc.date.issued	2023-01-01
dc.identifier.citation	Cognitive Neurodynamics, 2023, pp. 1-13
dc.identifier.issn	1871-4080
dc.identifier.issn	1871-4099
dc.identifier.uri	http://hdl.handle.net/10453/167168
dc.description.abstract	The human brain’s ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.
dc.language	en
dc.publisher	Springer Nature
dc.relation.ispartof	Cognitive Neurodynamics
dc.relation.isbasedon	10.1007/s11571-022-09927-7
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0606 Physiology, 1116 Medical Physiology, 1702 Cognitive Sciences
dc.subject.classification	Experimental Psychology
dc.title	Pruning and quantization algorithm with applications in memristor-based convolutional neural network
dc.type	Journal Article
utslib.for	0606 Physiology
utslib.for	1116 Medical Physiology
utslib.for	1702 Cognitive Sciences
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	closed_access	*
dc.date.updated	2023-03-13T12:51:56Z
pubs.publication-status	Published

Abstract:

The human brain’s ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/167168