Posted: 7th October 2013

Organizations working with big data often have to deal with big numbers of users, big software systems, and big libraries of data analysis tools. The level of complexity is typically so high that it becomes impossible to find enough domain experts in the market, or to find people in the organization capable of understanding the entire system behind a product. In these large domains, it becomes essential to automate so as to increase robustness, efficiency, product quality, and human productivity. Many of the automation challenges can be cast within the growing framework of Bayesian optimization. However, for Bayesian optimization to be feasible, it is essential to extend the technique to high-dimensions. This paper attacked this hard problem and presented an effective solution.

Title: Bayesian Optimization in High Dimensions via Random Embeddings

Authors: Ziyu Wang (Oxford CS PhD student), Masrour Zoghi, Frank Hutter, David Matheson and Nando de Freitas (Oxford CS professor)

Abstract: Bayesian optimization techniques have been successfully applied to robotics, web analytics, planning, automatic machine learning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach was restricted to problems of moderate dimension. Several scientific meetings on Bayesian optimization had identified its scaling to high dimensions as one of the holy grails of the field. In this paper, we introduced a novel random embedding idea to attack the problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple and applies to domains with both categorical and continuous variables. The experiments demonstrated that REMBO can effectively solve high-dimensional problems, such as automatic configuration of mixed integer programming solvers.