Please visit the official website for more information.
Description: In the last few decades, much effort has been devoted to the development of first-order methods. These methods enjoy a low per-iteration cost and have optimal complexity, are easy to implement, and have proven to be effective for most machine learning applications. In contrast, higher-order methods, such as Newton, quasi-Newton and adaptive gradient descent methods, are extensively used in many scientific and engineering domains. At least in theory, these methods possess several nice features: they exploit local curvature information to mitigate the effects of ill-conditioning, they avoid or diminish the need for hyper-parameter tuning, and they have enough concurrency to take advantage of distributed computing environments. However, often higher-order methods are “undervalued.”
This workshop will attempt to shed light on this statement. Topics of interest include, but are not limited to, second-order methods, adaptive gradient descent methods, regularization techniques, as well as techniques based on higher-order derivatives. This workshop will bring machine learning and optimization researchers closer, in order to facilitate a discussion with regards to underlying questions such as the following:
- Why are they not omnipresent?
- Why are higher-order methods important in machine learning, and what advantages can they offer?
- What are their limitations and disadvantages?
- How should (or could) they be implemented in practice?
Call for Papers
We welcome submissions to the workshop under the general theme of “Beyond First-Order Optimization Methods in Machine Learning”. Topics of interest include, but are not limited to,
- Second-order methods
- Quasi-Newton methods
- Derivative-free methods
- Distributed methods beyond first-order
- Online methods beyond first-order
- Applications of methods beyond first-order to diverse applications (e.g., training deep neural networks, natural language processing, dictionary learning, etc)
We encourage submissions that are theoretical, empirical or both.