Master 2024/2025

Optimization in Machine Learning

Type: Elective course (Math of Machine Learning)

When: 1 year, 2 module

Open to: students of one campus

Language: English

Course Syllabus

Abstract

Most machine learning tasks are formally optimization problems.The first part of the course is an introduction. We will study/repeat classical concepts of optimization (convexity, optimality conditions, etc.) and optimization methods (gradient descent, conditional gradient method, conjugate gradient method, Newton's method, quasi-Newton methods). The second part of the course is devoted to the stochastic gradient method and its different variations, which are used in various learning problems. In the third part of the course we will talk about modern state-of-the-art methods for learning problems. First of all, we will discuss adaptivity and methods with momentum. In the fourth part of the course, we will focus on various distributed formulations of the optimization problems from cluster training to the now popular federated learning.