Memristor-Based Design of Sparse Compact Convolutional Neural Network

Wen, S; Wei, H; Yan, Z; Guo, Z; Yang, Y; Huang, T; Chen, Y

Memristor-Based Design of Sparse Compact Convolutional Neural Network

Wen, S

Wei, H Yan, Z

Guo, Z Yang, Y Huang, T Chen, Y

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Network Science and Engineering, 2020, 7, (3), pp. 1431-1440
Issue Date:: 2020-07-01

Closed Access

	Filename	Description	Size
	08798700.pdf	Published version	3.23 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Wen, S https://orcid.org/0000-0002-5048-0319
dc.contributor.author	Wei, H
dc.contributor.author	Yan, Z https://orcid.org/0000-0003-3368-2100
dc.contributor.author	Guo, Z
dc.contributor.author	Yang, Y
dc.contributor.author	Huang, T
dc.contributor.author	Chen, Y
dc.date.accessioned	2021-03-01T20:50:06Z
dc.date.available	2021-03-01T20:50:06Z
dc.date.issued	2020-07-01
dc.identifier.citation	IEEE Transactions on Network Science and Engineering, 2020, 7, (3), pp. 1431-1440
dc.identifier.issn	2334-329X
dc.identifier.issn	2327-4697
dc.identifier.uri	http://hdl.handle.net/10453/146585
dc.description.abstract	© 2013 IEEE. Memristor has been widely studied for hardware implementation of neural networks due to the advantages of nanometer size, low power consumption, fast switching speed and functional similarity to biological synapse. However, it is difficult to realize memristor-based deep neural networks for there exist a large number of network parameters in general structures such as LeNet, FCN, etc. To mitigate this problem, this paper aims to design a memristor-based sparse compact convolutional neural network (MSCCNN) to reduce the number of memristors. We firstly use an average pooling and 1× 1 convolutional layer to replace fully connected layers. Meanwhile, depthwise separation convolution is utilized to replace traditional convolution to further reduce the number of parameters. Furthermore, a network pruning method is adopted to remove the redundant memristor crossbars for depthwise separation convolutional layers. Therefore, a more compact network structure is obtained while the recognition accuracy remaining unchanged. Simulation results show that the designed model achieves superior accuracy rates while greatly reducing the scale of the hardware circuit. Compared with traditional designs of memristor-based CNN, our proposed model has smaller area and lower power consumption.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Network Science and Engineering
dc.relation.isbasedon	10.1109/TNSE.2019.2934357
dc.rights	© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.	en_US
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Memristor-Based Design of Sparse Compact Convolutional Neural Network
dc.type	Journal Article
utslib.citation.volume	7
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0805 Distributed Computing
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
pubs.organisational-group	/University of Technology Sydney
utslib.copyright.status	closed_access	*
dc.date.updated	2021-03-01T20:50:03Z
pubs.issue	3
pubs.publication-status	Published
pubs.volume	7
utslib.citation.issue	3

Abstract:

© 2013 IEEE. Memristor has been widely studied for hardware implementation of neural networks due to the advantages of nanometer size, low power consumption, fast switching speed and functional similarity to biological synapse. However, it is difficult to realize memristor-based deep neural networks for there exist a large number of network parameters in general structures such as LeNet, FCN, etc. To mitigate this problem, this paper aims to design a memristor-based sparse compact convolutional neural network (MSCCNN) to reduce the number of memristors. We firstly use an average pooling and 1× 1 convolutional layer to replace fully connected layers. Meanwhile, depthwise separation convolution is utilized to replace traditional convolution to further reduce the number of parameters. Furthermore, a network pruning method is adopted to remove the redundant memristor crossbars for depthwise separation convolutional layers. Therefore, a more compact network structure is obtained while the recognition accuracy remaining unchanged. Simulation results show that the designed model achieves superior accuracy rates while greatly reducing the scale of the hardware circuit. Compared with traditional designs of memristor-based CNN, our proposed model has smaller area and lower power consumption.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/146585