Abstract

Residual neural networks (ResNets) are among the state-of-the-art for image classification tasks. With the advent of automated machine learning (AutoML), automated hyperparameter optimization methods are by now routinely used for tuning various network types. However, in the thriving field of deep neural networks, this progress is not yet matched by equal progress on rigorous techniques that yield information beyond performance-optimizing hyperparameter settings. In this work, we aim to answer the following question: given a residual neural network architecture, what are generally (across datasets) its most important hyperparameters? In order to answer this question, we assembled a benchmark suite containing 10 image classification datasets. For each of these datasets, we analyze which of the hyperparameters were most influential using the functional ANOVA framework. This experiment both confirmed expected patterns, and revealed new insights. With these experimental results, we aim to form a more rigorous basis for experimentation that leads to better insight towards what hyperparameters are important to make neural networks perform well.

Short Bio:

Jan N. van Rijn completed his PhD in 2016 at Leiden University. Currently he has the post of Assistant Professor in Automated Machine Learning, Leiden University. In previous years he was working as Post-doc in Data Science Institute, Columbia University (New York, USA) and also in ML4AAD, Freiburg University (Germany). Jan van Rijn is one of the founders of OpenML.org, an experiment database for Machine Learning research. By storing results of earlier Machine Learning experiments, this site help to model which algorithms work well on which data. These meta-studies can be applied on many areas of Machine Learning, such as General Data Science, Data Stream Research and Subgroup Discovery.

About Talks@DCC 

The mission of the Talks@DCC seminars is to bring together researchers and students, to foster discussions and promote scientific awareness and collaboration.

Participate: attend and propose your talk.