Магистратура
2022/2023
Нейронные сети и глубокое обучение
Статус:
Курс по выбору (Бизнес-аналитика и системы больших данных)
Направление:
38.04.05. Бизнес-информатика
Кто читает:
Департамент бизнес-информатики
Где читается:
Высшая школа бизнеса
Когда читается:
2-й курс, 1, 2 модуль
Формат изучения:
с онлайн-курсом
Онлайн-часы:
16
Охват аудитории:
для всех кампусов НИУ ВШЭ
Преподаватели:
Калмыкова Надежда Сергеевна
Прогр. обучения:
Бизнес-аналитика и системы больших данных
Язык:
английский
Кредиты:
6
Контактные часы:
32
Course Syllabus
Abstract
Recently, deep learning has become a tool of choice for multiple problems in the fields of computer vision, robotics, biology, natural language processing, and many more. The technology is enabled by various milestones in computing and engineering, such as graphical processing units (GPU) and differentiable programming. Due to recent advances in open source software for deep learning, training a model has become a relatively simple exercise. However, the techniques used by researchers are fairly complicated. This course is focused on practical part of applying state-of-the-art techniques of deep learning on real-world problems. The course is based on popular and accessible MOOC called fast.ai and covers all the sections of the course with advice from machine learning practice.
Learning Objectives
- Learn to train models that achieve state-of-the-art results in: computer vision, natural language processing, tabular data, and collaborative filtering
- Learn approaches to turn models into web applications
- Learn to implement deep learning training loop from the scratch
- Acknowledge ethical implications of applying machine learning in practice
Expected Learning Outcomes
- Able to use embedding for tabular data and recommenders
- Able to use momentum and advanced optimizers for stochastic gradient descent
- Able to use residual blocks with neural networks
- Can construct a digit classifier using a deep learning model
- Can construct a neural network from scratch
- Can construct recurrent neural network from scratch
- Can solve multi-class and multi-label problems with deep learning
- Knows advanced neural networks such as U-Net and Siamese
- Knows and uses state-of-the-art approaches to train neural networks
- Knows the definitions of deep learning
- Understands approaches to put machine learning system into production
- Understands approaches to solve natural language processing problems
- Understands data ethics and able to detect ethical problems
- Understands the role of convolutions in image processing
Course Contents
- Introduction
- Machine learning in production
- Data ethics
- Training a digit classifier
- Image classification
- Other computer vision problems
- Reaching state-of-the-art
- Collaborative filtering
- Tabular data
- Natural language processing
- Software libraries for data processing
- A language model from scratch
- Convolutional neural networks
- Residual neural networks
- Neural architectures
- The training process of neural networks
- A neural net from scratch
- Class Activation Maps
- Introduction to deep learning
- Neural Networks Basics
- Shallow neural networks
- Deep Neural Networks
Assessment Elements
- Homework №1A student should provide a Jupyter notebook
- Homework №2A student should provide a Jupyter notebook
- ExamExam format: the exam is taken in a written format, remotely (online) on MS Teams platform. A student is expected to complete the assignment and provide Jupyter (.ipynb) notebook to the instructor (the URL for Google Forms to submit is provided in MS Teams). To participate in the exam a student should have access to a compute that is either able to run Jupyter or work with Google Colab page. The file should be submitted one hour before the end of the exam. A student is allowed to use any kind of information to complete the assignment. A student should not consult or re-use solutions by other fellow students.
Bibliography
Recommended Core Bibliography
- Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning : Data Mining, Inference, and Prediction (Vol. Second edition, corrected 7th printing). New York: Springer. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=277008
- Ian Goodfellow, Yoshua Bengio, & Aaron Courville. (2016). Deep Learning. The MIT Press.
- Jeremy Howard, & Sylvain Gugger. (2020). Deep Learning for Coders with Fastai and PyTorch. O’Reilly Media.
Recommended Additional Bibliography
- M Narasimha Murty, & V Susheela Devi. (2015). Introduction To Pattern Recognition And Machine Learning. World Scientific.