Bachelor 2020/2021

Bayesian Statistics

Type: Elective course (Fundamental and Computational Linguistics)

Area of studies: Fundamental and Applied Linguistics

Delivered by: School of Linguistics

Where: Faculty of Humanities

When: 3 year, 3 module

Mode of studies: distance learning

Open to: students of one campus

Instructors: Yury Lander

Language: English

ECTS credits: 3

Contact hours: 2

Course Syllabus

Abstract

This course describes Bayesian statistics, in which one's inferences about parameters or hypotheses are updated as evidence accumulates. You will learn to use Bayes’ rule to transform prior probabilities into posterior probabilities, and be introduced to the underlying theory and perspective of the Bayesian paradigm. The course will apply Bayesian methods to several practical problems, to show end-to-end Bayesian analyses that move from framing the question to building models to eliciting prior probabilities to implementing in R (free statistical software) the final posterior distribution. Additionally, the course will introduce credible regions, Bayesian comparisons of means and proportions, Bayesian regression and inference using multiple models, and discussion of Bayesian prediction. We assume learners in this course have background knowledge equivalent to what is covered in the earlier three courses in this specialization: "Introduction to Probability and Data," "Inferential Statistics," and "Linear Regression and Modeling." https://www.coursera.org/learn/bayesian

Learning Objectives

• The goal of this course is to teach students about Bayesian Statistics, Bayesian Linear Regression, Bayesian Inference, R Programming, teach them to use Bayes’ rule to transform prior probabilities into posterior probabilities, and introduce them to the underlying theory and perspective of the Bayesian paradigm.

Expected Learning Outcomes

• Knows the basics of Bayesian statistics, understands conditional probabilities, Bayes' rule, Bayesian vs. frequentist definitions of probability, frequentist and Bayesian approaches to inference for a proportion, effect of sample size on the posterior, frequentist vs. Bayesian inference, can solve problems using Bayes' rule
• Can understand and define the concepts of prior, likelihood, and posterior probability and identify how they relate to one another, knows what Bayesian inference, non-conjugate priors, credible intervals and predictive inference module learning objectives are.
• Knows how to work with loss functions, how to minimize expected loss for hypothesis testing, how to work with mixtures of conjugate priors and MCMC, how to compare two paired means using Bayes' factors. Knows what posterior probabilities of hypotheses and Bayes factors, the Normal-Gamma Conjugate Family, predictive distributions are.
• Can implement Bayesian model averaging, interpret Bayesian multiple linear regression and understand its relationship to the frequentist linear regression approach.
• Knows how Bayesian methods can be used when working with big data, in biostatistics and public health.
• Can use the data set provided to complete and report on a data analysis question.

Course Contents

• About the Specialization and the Course
  This short module introduces basics about Coursera specializations and courses in general, this specialization: Statistics with R, and this course: Bayesian Statistics.
• The Basics of Bayesian Statistics
  In this module, we will work with conditional probabilities, which is the probability of event B given event A. Conditional probabilities are very important in medical decisions. By the end of the week, you will be able to solve problems using Bayes' rule, and update prior probabilities. The Basics of Bayesian statistics, conditional probabilities and Bayes' rule, Bayes' rule and diagnostic testing, Bayes updating, Bayesian vs. frequentist definitions of probability, inference for a proportion: frequentist approach, inference for a proportion: Bayesian approach, effect of sample size on the posterior, frequentist vs. Bayesian inference. Week 1 Lab Instructions (RStudio), Week 1 Lab Instructions (RStudio Cloud)
• Bayesian Inference
  In this week, we will discuss the continuous version of Bayes' rule and show you how to use it in a conjugate family, and discuss credible intervals. Bayesian inference, from the discrete to the continuous, elicitation, conjugacy, inference on a binomial proportion, the Gamma-Poisson conjugate families, the normal-normal conjugate families, non-conjugate priors, credible intervals, predictive inference module learning objectives, week 2 lab instructions (RStudio), week 2 lab instructions (RStudio Cloud).
• Decision Making
  In this module, we will discuss Bayesian decision making, hypothesis testing, and Bayesian testing. By the end of this week, you will be able to make optimal decisions based on Bayesian statistics and compare multiple hypotheses using Bayes Factors. Decision making, losses and decision making, working with loss functions, minimizing expected loss for hypothesis testing, posterior probabilities of hypotheses and Bayes factors, the Normal-Gamma Conjugate Family, inference via Monte Carlo sampling, predictive distributions and prior choice, reference priors, mixtures of conjugate priors and MCMC, hypothesis testing: normal mean with known variance, comparing two paired means using Bayes' factors, comparing two independent means: hypothesis testing, comparing two independent means: what to report, week 3 lab instructions (RStudio), week 3 lab instructions (RStudio Cloud).
• Bayesian Regression
  This week, we will look at Bayesian linear regressions and model averaging, which allows you to make inferences and predictions using several models. Bayesian regression, Bayesian simple linear regression, checking for outliers, Bayesian multiple regression, model selection criteria, Bayesian model uncertainty, Bayesian model averaging, stochastic exploration, priors for Bayesian model uncertainty, r demo: crime and punishment, decisions under model uncertainty, module learning objectives, week 4 lab instructions (Rstudio), week 4 lab instructions (RStudio Cloud).
• Perspectives on Bayesian Applications
  This week consists of interviews with statisticians on how they use Bayesian statistics in their work, as well as the final project in the course. Bayesian inference, Bayesian methods and big data, Bayesian methods in biostatistics and public health.
• Data Analysis Project
  In this module you will use the data set provided to complete and report on a data analysis question. Please read the background information, review the report template (downloaded from the link in Lesson Project Information), and then complete the peer review assignment.

Assessment Elements

• Online course
• Discussion with a HSE instructor

Interim Assessment

• Interim assessment (3 module)
  0.3 * Discussion with a HSE instructor + 0.7 * Online course

Bibliography

Recommended Core Bibliography

• Rohatgi, V. K., & Saleh, A. K. M. E. (2001). An Introduction to Probability and Statistics (Vol. 2nd ed. Vijay K. Rohatgi, A.K. Md. Ehsanes Saleh). New York: Wiley-Interscience. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=396326

Recommended Additional Bibliography

• Donovan, T. M., & Mickey, R. M. (2019). Bayesian Statistics for Beginners : A Step-by-step Approach. Oxford: OUP Oxford. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=2139683