Optimization in Machine Learning

2024/2025

Academic Year

ENG

Instruction in English

Course type:

Elective course

When:

1 year, 2 module

Course Syllabus

Abstract

Most machine learning tasks are formally optimization problems.The first part of the course is an introduction. We will study/repeat classical concepts of optimization (convexity, optimality conditions, etc.) and optimization methods (gradient descent, conditional gradient method, conjugate gradient method, Newton's method, quasi-Newton methods). The second part of the course is devoted to the stochastic gradient method and its different variations, which are used in various learning problems. In the third part of the course we will talk about modern state-of-the-art methods for learning problems. First of all, we will discuss adaptivity and methods with momentum. In the fourth part of the course, we will focus on various distributed formulations of the optimization problems from cluster training to the now popular federated learning.