Delivered at:: Big Data and Information Retrieval School

Course type:: Compulsory course

When:: 3 year, 3, 4 module

Instructors

Bykov, Kirill

Abstract

The course is dedicated to studying deep learning, which is the most rapidly developing field of machine learning. The course attendees will learn what konds of machine learning tasks can be solved using neural networks and what types of neural networks are currently in use. The course has a clear practical focus, students will have to train neural networks on the various frameworks using the Python programming language. The course also covers tasks related to images and texts.

Learning Objectives

The course aims to help students develop an understanding of the principles, algorithms, and applications of deep learning.
The course aims to equip students with the necessary skills and knowledge to apply deep learning techniques to solve real-world problems.
The course focuses on developing a theoretical foundation in deep learning, as well as practical experience in implementing and experimenting with deep learning models.
The course aims to expose students to the latest advances and research in deep learning, and encourage critical thinking and problem-solving skills in the context of deep learning.
The aim of the course is to prepare students for any careers where deep learning techniques are increasingly being used.

Expected Learning Outcomes

Knows basic types tasks solved with using deep learning. Is developing architecture, implements, trains and produces optimization neural parameters networks. Solves applied tasks from various areas with using deep learning.
Understand the key computations underlying deep learning, use them to build and train deep neural networks, and apply it to computer vision.
Student are aware of basic concepts and can use Python for NLP deep lerning: recurrent neural networks, convolutional networks, pooling, attention mechanism, transformer.
Students are aware of basic concepts of deep learning: tensor, model weighs, layers, various activation functions, loss function and metrics, optimization methods, softmax and crossentropy, dropout, batches, stochastic gradient decent, epoch, batch normalization.
Learn the operation and training of neural networks, and their relation to deep learning
Learn the basic concepts and uses of reinforcement learning algorithms
Can create and use convolutional neural networks
Having completed the topic, students should be able to understand the basic concepts of Bayesian inference.
Having completed the topic, students should be able to identify its differences from Frequentist Inference in point and interval estimation, hypothesis testing and prediction.

Course Contents

Bayesian Inference
Artificial Neural Networks (ANN)
Linear Neural Networks for Regression
Linear Neural Networks for Classification
Multilayer Perceptrons
Convolutional Neural Networks
Modern Convolutional Neural Networks
Recurrent Neural Networks
Modern Recurrent Neural Networks
Attention Mechanisms and Transformers
Optimization Algorithms
Computer Vision
Natural Language Processing
Reinforcement Learning
Generative Adversarial Networks

Assessment Elements

Home assignments
Home assignments. The grade for the current category is calculated as cumulative from the beginning of the course.
Quizzes
The grade for the current category is calculated as cumulative from the beginning of the course.
Participation
The grade for the current category is calculated as cumulative from the beginning of the course.
Midterm Test
These are individualized, timed, (possibly) proctored and otherwise constrained tests to prevent cheating. In general, expect 60 questions in 60 minutes, some of which you may will have seen in quizzes. The assessment of the test is based on the marking scheme that comes with the exam assignment. Each problem and their sub parts are worth a certain number of points, the sum of these points is equal to 100, which is the maximum grade for the exam on the 100 point scale. The student is awarded the assigned number of points for the correct answer to each part of the question and partial credit may also be awarded.
Exam
These are individualized, timed, (possibly) proctored and otherwise constrained tests to prevent cheating. In general, expect 60 questions in 60 minutes, some of which you may will have seen in quizzes. The assessment of the exam is based on the marking scheme that comes with the exam assignment. Each problem and their sub parts are worth a certain number of points, the sum of these points is equal to 100, which is the maximum grade for the exam on the 100 point scale. The student is awarded the assigned number of points for the correct answer to each part of the question and partial credit may also be awarded.

Interim Assessment

2023/2024 4th module
0.2 * Exam + 0.3 * Home assignments + 0.2 * Midterm Test + 0.1 * Participation + 0.2 * Quizzes

Bibliography

Recommended Core Bibliography

Ian Goodfellow, Yoshua Bengio, & Aaron Courville. (2016). Deep Learning. The MIT Press.

Recommended Additional Bibliography

James, G. et al. An introduction to statistical learning. – Springer, 2013. – 426 pp.

Authors

Кононова Елизавета Дмитриевна
Галевская Софья Андреевна

Bachelor’s Programme 'Data Science and Business Analytics'

Deep Learning