High dimensional Bayesian optimization via supervised dimension reduction

International Joint Conference on Artificial Intelligence
Publication Type:
Conference Proceeding
IJCAI International Joint Conference on Artificial Intelligence, 2019, 2019-August, pp. 4292-4298
Issue Date:
Full metadata record
© 2019 International Joint Conferences on Artificial Intelligence. All rights reserved. Bayesian optimization (BO) has been broadly applied to computational expensive problems, but it is still challenging to extend BO to high dimensions. Existing works are usually under strict assumption of an additive or a linear embedding structure for objective functions. This paper directly introduces a supervised dimension reduction method, Sliced Inverse Regression (SIR), to high dimensional Bayesian optimization, which could effectively learn the intrinsic sub-structure of objective function during the optimization. Furthermore, a kernel trick is developed to reduce computational complexity and learn nonlinear subset of the unknowing function when applying SIR to extremely high dimensional BO. We present several computational benefits and derive theoretical regret bounds of our algorithm. Extensive experiments on synthetic examples and two real applications demonstrate the superiority of our algorithms for high dimensional Bayesian optimization.
Please use this identifier to cite or link to this item: