• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
For visually-impairedFor visually-impairedUser profile (HSE staff only)Search
Advanced search
Menu
HSE University
User profile (HSE staff only)Search
Advanced search

Course Syllabus

Master 2023/2024

Distributed Data Processing

Category 'Best Course for New Knowledge and Skills'
Type: Compulsory course
Area of studies: Business Informatics
Delivered by: Department of Business Informatics
Where: Faculty of Economics
When: 1 year, 3, 4 module
Mode of studies: distance learning
Online hours: 20
Open to: students of all HSE University campuses
Instructors: Kirill Gomenyuk
Master’s programme: Business Analytics
Language: English
ECTS credits: 6
Contact hours: 36

Course Syllabus

Full Syllabus

Abstract

The main goal of this course is to provide students with an opportunity to acquire conceptual background and practical tools applicable to Big Data analytics and real time computation. The course will briefly review specific challenges of Big Data Analytics, such as problems of extracting, unifying, updating, and merging information and specific needs in processing data, which should be highly parallel and distributed. We will see how different mathematical approaches can be transformed to conform to the Big Data demands.
Learning Objectives

Learning Objectives

  • Learn fundamentals of data analysis using distributed data processing frameworks, setting the foundation for how to combine data with advanced analytics at scale and in production environments
Expected Learning Outcomes

Expected Learning Outcomes

  • Identify when a big data problem needs data integration
  • Describe the connections between data management operations and the big data processing patterns needed to utilize them in large-scale analytical applications
  • Retrieve data from example database and big data management systems
  • Execute simple big data integration and processing on Hadoop and Spark platforms
  • Write scalable Spark SQL code that executes against a cluster of machines
  • Inspect the Spark UI to analyze query performance and identify bottlenecks
Course Contents

Course Contents

  • Introduction to Distributed Data Processing
  • Retrieving Big Data
  • Big Data Integration
  • Processing Big Data
  • Big Data Analytics using Spark
Assessment Elements

Assessment Elements

  • non-blocking Homework
    Each seminar, students will receive homework on the materials they have passed
  • non-blocking Practice
    Students will be asked to prepare reports after practice lessons
  • non-blocking Exam
    Final test
  • non-blocking Test
    Short quiz at the beginning of the lectures
Interim Assessment

Interim Assessment

  • 2023/2024 4th module
    0.3 * Exam + 0.21 * Homework + 0.28 * Practice + 0.21 * Test
Bibliography

Bibliography

Recommended Core Bibliography

  • Hoger Khayrolla Omar, & Alaa Khalil Jumaa. (2019). Big Data Analysis Using Apache Spark MLlib and Hadoop HDFS with Scala and Java. https://doi.org/10.24017/science.2019.1.2
  • Jules S. Damji, Brooke Wenig, Tathagata Das, & Denny Lee. (2020). Learning Spark. O’Reilly Media.
  • Luu H. Beginning Apache Spark 2: With Resilient Distributed Datasets, Spark SQL, Structured Streaming and Spark Machine Learning Library. – Berkeley: Apress, 2018.
  • Romeo Kienzler, Md. Rezaul Karim, Sridhar Alla, Siamak Amirghodsi, Meenakshi Rajendran, Broderick Hall, & Shuen Mei. (2018). Apache Spark 2: Data Processing and Real-Time Analytics : Master Complex Big Data Processing, Stream Analytics, and Machine Learning with Apache Spark. Birmingham: Packt Publishing. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1991793

Recommended Additional Bibliography

  • Brajesh Mishra. (2020). Big Data Analysis Using Hadoop Map Reduce. https://doi.org/10.26562/irjcs.2020.v0705.005
  • Field, L., & Newcomb, O. (2012). Distributed Computing : Concepts, Architecture and Applications. Delhi: Academic Studio. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=446466
  • Kienzler, R. (2017). Mastering Apache Spark 2.x - Second Edition (Vol. 2nd ed). Birmingham: Packt Publishing. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1562681
  • Langewisch, R. P. (2016). Performance study of an implementation of the push-relabel maximum flow algorithm in Apache Spark’s GraphX, A.
  • Ryza, S., Laserson, U., Owen, S., & Wills, J. (2017). Advanced Analytics with Spark : Patterns for Learning From Data at Scale (Vol. Second edition). Sebastopol, CA: Reilly - O’Reilly Media. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1533378

Authors

  • Beklarian Armen Levonovich
  • Gomeniuk Kirill Sergeevich