Efficient multi-objective neural architecture search framework via policy gradient algorithm

Lyu, B; Yang, Y; Cao, Y; Wang, P; Zhu, J; Chang, J; Wen, S

Efficient multi-objective neural architecture search framework via policy gradient algorithm

Lyu, B Yang, Y Cao, Y Wang, P Zhu, J Chang, J Wen, S

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Information Sciences, 2024, 661, pp. 120186
Issue Date:: 2024-03-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (1.48 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Lyu, B
dc.contributor.author	Yang, Y
dc.contributor.author	Cao, Y
dc.contributor.author	Wang, P
dc.contributor.author	Zhu, J
dc.contributor.author	Chang, J
dc.contributor.author	Wen, S https://orcid.org/0000-0001-8077-7001
dc.date.accessioned	2024-03-07T04:30:41Z
dc.date.available	2024-03-07T04:30:41Z
dc.date.issued	2024-03-01
dc.identifier.citation	Information Sciences, 2024, 661, pp. 120186
dc.identifier.issn	0020-0255
dc.identifier.uri	http://hdl.handle.net/10453/176264
dc.description.abstract	Differentiable architecture search plays a prominent role in Neural Architecture Search (NAS) and exhibits preferable efficiency than traditional heuristic NAS methods, including those based on evolutionary algorithms (EA) and reinforcement learning (RL). However, differentiable NAS methods encounter challenges when dealing with non-differentiable objectives like energy efficiency, resource constraints, and other non-differentiable metrics, especially under multi-objective search scenarios. While the multi-objective NAS research addresses these challenges, the individual training required for each candidate architecture demands significant computational resources. To bridge this gap, this work combines the efficiency of the differentiable NAS with metrics compatibility in multi-objective NAS. The architectures are discretely sampled by the architecture parameter α within the differentiable NAS framework, and α are directly optimised by the policy gradient algorithm. This approach eliminates the need for a sampling controller to be learned and enables the encompassment of non-differentiable metrics. We provide an efficient NAS framework that can be readily customized to address real-world multi-objective NAS (MNAS) scenarios, encompassing factors such as resource limitations and platform specialization. Notably, compared with other multi-objective NAS methods, our NAS framework effectively decreases the computational burden (accounting for just 1/6 of the NSGA-Net). This search framework is also compatible with the other efficiency and performance improvement strategies under the differentiable NAS framework.
dc.language	en
dc.publisher	Elsevier
dc.relation.ispartof	Information Sciences
dc.relation.isbasedon	10.1016/j.ins.2024.120186
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	01 Mathematical Sciences, 08 Information and Computing Sciences, 09 Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	40 Engineering
dc.subject.classification	46 Information and computing sciences
dc.subject.classification	49 Mathematical sciences
dc.title	Efficient multi-objective neural architecture search framework via policy gradient algorithm
dc.type	Journal Article
utslib.citation.volume	661
utslib.for	01 Mathematical Sciences
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	open_access	*
dc.date.updated	2024-03-07T04:30:40Z
pubs.publication-status	Published
pubs.volume	661

Abstract:

Differentiable architecture search plays a prominent role in Neural Architecture Search (NAS) and exhibits preferable efficiency than traditional heuristic NAS methods, including those based on evolutionary algorithms (EA) and reinforcement learning (RL). However, differentiable NAS methods encounter challenges when dealing with non-differentiable objectives like energy efficiency, resource constraints, and other non-differentiable metrics, especially under multi-objective search scenarios. While the multi-objective NAS research addresses these challenges, the individual training required for each candidate architecture demands significant computational resources. To bridge this gap, this work combines the efficiency of the differentiable NAS with metrics compatibility in multi-objective NAS. The architectures are discretely sampled by the architecture parameter α within the differentiable NAS framework, and α are directly optimised by the policy gradient algorithm. This approach eliminates the need for a sampling controller to be learned and enables the encompassment of non-differentiable metrics. We provide an efficient NAS framework that can be readily customized to address real-world multi-objective NAS (MNAS) scenarios, encompassing factors such as resource limitations and platform specialization. Notably, compared with other multi-objective NAS methods, our NAS framework effectively decreases the computational burden (accounting for just 1/6 of the NSGA-Net). This search framework is also compatible with the other efficiency and performance improvement strategies under the differentiable NAS framework.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/176264