AutoHAS: Efficient Hyperparameter and Architecture Search

Dong, X; Tan, M; Yu, AW; Peng, D; Gabrys, B; Le, QV

AutoHAS: Efficient Hyperparameter and Architecture Search

Dong, X

Tan, M Yu, AW Peng, D Gabrys, B

Le, QV

Permalink

Publication Type:: Journal Article
Citation:: 2020
Issue Date:: 2020-06-06

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Submitted Manuscript VersionAdobe PDF (422.1 kB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Dong, X https://orcid.org/0000-0001-9272-1590
dc.contributor.author	Tan, M
dc.contributor.author	Yu, AW
dc.contributor.author	Peng, D
dc.contributor.author	Gabrys, B https://orcid.org/0000-0002-0790-2846
dc.contributor.author	Le, QV
dc.date.accessioned	2021-05-04T01:46:52Z
dc.date.available	2021-05-04T01:46:52Z
dc.date.issued	2020-06-06
dc.identifier.citation	2020
dc.identifier.uri	http://hdl.handle.net/10453/148595
dc.description.abstract	Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unified pipeline, AutoHAS, to efficiently search for both architectures and hyperparameters. AutoHAS learns to alternately update the shared network weights and a reinforcement learning (RL) controller, which learns the probability distribution for the architecture candidates and HP candidates. A temporary weight is introduced to store the updated weight from the selected HPs (by the controller), and a validation accuracy based on this temporary weight serves as a reward to update the controller. In experiments, we show AutoHAS is efficient and generalizable to different search spaces, baselines and datasets. In particular, AutoHAS can improve the accuracy over popular network architectures, such as ResNet and EfficientNet, on CIFAR-10/100, ImageNet, and four more other datasets.
dc.language	en
dc.rights	info:eu-repo/semantics/openAccess
dc.title	AutoHAS: Efficient Hyperparameter and Architecture Search
dc.type	Journal Article
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - CHT - Health Technologies
pubs.organisational-group	/University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/A/DRsch The Data Science Institute
utslib.copyright.status	open_access	*
dc.date.updated	2021-05-04T01:46:50Z

Abstract:

Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unified pipeline, AutoHAS, to efficiently search for both architectures and hyperparameters. AutoHAS learns to alternately update the shared network weights and a reinforcement learning (RL) controller, which learns the probability distribution for the architecture candidates and HP candidates. A temporary weight is introduced to store the updated weight from the selected HPs (by the controller), and a validation accuracy based on this temporary weight serves as a reward to update the controller. In experiments, we show AutoHAS is efficient and generalizable to different search spaces, baselines and datasets. In particular, AutoHAS can improve the accuracy over popular network architectures, such as ResNet and EfficientNet, on CIFAR-10/100, ImageNet, and four more other datasets.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/148595