Bachelor
2023/2024
Introduction to neural network and machine translation
Category 'Best Course for New Knowledge and Skills'
Type:
Elective course (Fundamental and Applied Linguistics)
Area of studies:
Fundamental and Applied Linguistics
Delivered by:
Department of Applied Mathematics and Informatics
When:
4 year, 2, 3 module
Mode of studies:
distance learning
Online hours:
20
Open to:
students of one campus
Language:
English
ECTS credits:
8
Contact hours:
38
Course Syllabus
Abstract
The course introduces basic concepts of neural networks, deep learning and machine translation.
Learning Objectives
- The purpose of the ciyrse is to develop the ability to use neural network in their research and applied projects.
Expected Learning Outcomes
- Is able to use word embedding models
- Is able to use supervised learning
- Understands the advantages and disadvantages of neural networks
- Can create and use convolutional neural networks
- Can create and use recurrent neural networks
- Can create and use attention-based neural networks
- Can pretrain and fine-tune neural networks and their components
- Understands the principles of large language models and knows how to use them to solve applied problems
Course Contents
- Word embedding, word2vec model
- Supervised learning, logistic regression, multilayer perceptron
- Overfitting problem, regularization
- Convolutional neural networks
- Recurrent neural networks, Seq2seq modeling
- Attention-based models, Transformers
- Pretraining and fine-tuning, BERT, GPT
- Large language models, Prompt engineering, Chain-of-thought
Assessment Elements
- Lab 1
- Lab 2
- Lab 3
- Lab 4
- Project
- QuizzesTest tasks that are given at every lecture except the first.
Interim Assessment
- 2023/2024 3rd moduleScore = 0.15*Lab 1 + 0.15*Lab 2 + 0.2*Lab 3 + 0.3*Lab 4 + 0.1*Project + 0.1*Quizzes Final score = 0.8*Score + 0.2*Bonus Bonus: A bonus point is awarded if a student has voluntarily gone beyond the scope of the discipline. An example of such work is the application of a method considered as part of an individual project on other data, or the adaptation of a basic model from laboratory 4 with ideas from a recent research on neural networks. The work that can be evaluated for a bonus point is previously agreed with the teacher.A student gets a maximum score of 10 if he/she has successfully completed at least two works that deserve a bonus.
Bibliography
Recommended Core Bibliography
- Kelleher, J. D. (2019). Deep Learning. Cambridge: The MIT Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=2234376
Recommended Additional Bibliography
- Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning, 2016. URL: http://www.deeplearningbook.org
- Ian Goodfellow, Yoshua Bengio, & Aaron Courville. (2016). Deep Learning. The MIT Press.
- Глубокое обучение. - 978-5-4461-1537-2 - Николенко С., Кадурин А., Архангельская Е. - 2020 - Санкт-Петербург: Питер - https://ibooks.ru/bookshelf/377026 - 377026 - iBOOKS