Automated Deep Learning: A Study on Neural Architecture Search
- Publication Type:
- Thesis
- Issue Date:
- 2021
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
Automated Deep Learning (AutoDL) aims to build a better deep learning model in a data-driven and automated manner, so that most practitioners in deep learning can also build a high-performance machine learning model, with being relieved from a labor-intensive and time-consuming neural network design process. AutoDL can bring new research ideas to deep neural networks, and lower the threshold of deep learning in various research areas through automated neural network design. This thesis focuses on the two specific research problems of neural architecture search (NAS) in the automated deep learning: one-shot NAS and differentiable NAS. In particular, this paper proposed a novelty driven sampling method and formulate the supernet training as a constrained continual learning optimization problem, to address the "rich-get-richer" problem and multi-model forgetting issue existing in one-shot NAS. As to the differentiable NAS, we leveraged a variational graph autoencoder to relieve the non-negligible incongruence, formulating the neural architecture search as a distribution learning problem to enhance exploration, and proposed the differentiable architecture search with stochastic implicit gradients to enable multi-step inner optimization.
Please use this identifier to cite or link to this item: