Delivered at:: Department of Educational Programmes

Course type:: Compulsory course

When:: 1 year, 1 module

Instructor

Antipkina, Inna

Full Syllabus

Abstract

The course provides students with the basic concepts of psychological and educational measurement: theories of measurement, test and item development, reliability and validity. The course starts with a historical review of the development of measurement in psychology and shows how the basic concepts changed over time. The most important issues of these concepts are discussed. The students will learn the main steps they need to go through in order to design a test or questionnaire and to ensure their psychometric quality. The guidance we use is aligned with the latest editions of the Educational measurement and the Standards of psychological measurement by APA and NCME. During the course the students will apply their theoretical knowledge to the development of their own questionnaire. The students will learn what such item chatecteristics as difficulty and discrimination are, how to calculate them, and how to deal with errors of measurement. We will dedicate a significant amount of time to the interpretation of test scores. The social consequences of standardized testing will also be discussed. The students have to be ready for extensive reading of texts on general methodological issues of measurement in social science as well as for group work and discussion.

Learning Objectives

Understand the theory of quantitative psychology and psychometrics Interpret fundamental concepts (e.g. measurement, validity, scales) and its' criticism Design, conduct, analyze, interpret and report Classical Test Theory analyses Understand fundamentals of Generalizability Theory Communicate research findings effectively Ensure social justice in testing and assessment

Expected Learning Outcomes

A student knows the history of the discipline and subfields
Student aware of advantages and limitations of quantitative psychology
Students can correctly apply key terminology of the field.
Students meet basic issues of quantitative psychometrics and learn how to avoid (if possible) the issues in their real practice.
Students master how to create operationalization of a latent construct
Students face the issue of "arbitrary metrics" and will learn how to avoid the problem in their practice.
Students can appropriately use basic validity terms.
Students will enable to plan their own validity studies (criterion validity, content validity)
Students face the controversial framework of the validity concept and will be able to build well-grounded validity desing
Students will be able to critically consider basic assumptions in psychological measurement.
Students master how to design, conduct, analyze, interpret and report Classical Test Theory analyses
Students will capable to interpret basics of CTT
Students aware of g-theory as one of basic psychometric approaches.
Students capable to apply g-theory in thier own research
Students communicate research findings effectively
Students ensure equity and social justice in testing and assessment
Students capable to design the feedback to the respondents in their own research

Course Contents

History of quantitative psychology. The role of quantitative psychology in human decision making
Psychological constructs and operational definitions
Operationalization Issues and Arbitrary Metrics
Validity concept (1). Criterion and content validity.
Validity concept (2) Construct validity. Modern approaches to validity
Scales of Measurement and the Criticism of the Concept.
Introduction to Classical Test Theory. Reliability estimates in CTT. Item analysis.
Introduction to Generalizability theory
Interpreting Test Results

Assessment Elements

Individual and group homework
Two home task we be group projects (you will operationalize a construct on your choice and prepare a CTT report) and one home task will be individual (an essay, describing a good test validization design, based on an article you find).
Final exam
Your final test will consist of MC and open-ended questions and will cover all the main topics of the discipline.

Interim Assessment

2024/2025 1st module
0.4 * Final exam + 0.6 * Individual and group homework

Bibliography

Recommended Core Bibliography

Archuleta, K. (2009). Evidence of Construct-Related Validity for Assessment Centers: More Pieces of the Inferential Pie. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsndl&AN=edsndl.oai.union.ndltd.org.tamu.edu.oai.repository.tamu.edu.1969.1.ETD-TAMU-2009-05-579
Coulacoglou, C., & Saklofske, D. H. (2017). Psychometrics and Psychological Assessment : Principles and Applications. London: Academic Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1169380
Epskamp, S., Maris, G. K. J., Waldorp, L. J., & Borsboom, D. (2016). Network Psychometrics.
Gary L. Canivez, Timothy R. Konold, Jason M. Collins, & Greg Wilson. (n.d.). Construct Validity of the Wechsler Abbreviated Scale of Intelligence and Wide Range Intelligence Test: Convergent and Structural Validity.
Helena Espirito Santo. (2017). Editorial: Validity in assessment instruments. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.D179CC42
Introduction to classical and modern test theory, Crocker, L., 2008
James, A., Algina, J., & Swaminathan, H. (2015). Psychometrics: Classical Test Theory. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.7E3F0482
Joel Michell. (2000). Normal science, pathological science and psychometrics. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.858D34B6
Kane, M. (2000). Current Concerns in Validity Theory. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED446094
Keith A. Markus, & Denny Borsboom. (2013). Frontiers of Test Validity Theory : Measurement, Causation, and Meaning. Routledge.
Kline, P. (2014). The New Psychometrics : Science, Psychology and Measurement. Hoboken: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=687083
Leary, M., Kelly, K., Cottrell, C., & Schreindorfer, L. (2013). Construct Validity of the Need to Belong Scale: Mapping the Nomological Network. Journal of Personality Assessment, 95(6), 610–624. https://doi.org/10.1080/00223891.2013.819511
Measurement and evaluation in psychology and education, Thorndike, R. M., 2010
Nako, Z., & Barnard, A. (2012). Construct validity of competency dimensions in a leadership assessment and development centre. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.F9B61AE3
Prabhaker Mishra, C M Pandey, Uttam Singh, & Anshul Gupta. (2018). Scales of measurement and presentation of statistical data. Annals of Cardiac Anaesthesia, (4), 419. https://doi.org/10.4103/aca.ACA_131_18
Ronald K. Hambleton, & Russell W. Jones. (n.d.). ========================= = ITEMS. Instructional Topics in Educational Measurement ========================= An NCME Instructional Module on Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.46F2C9F0
Stephen N. Haynes, David C. S. Richard, & Edward S. Kubany. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.E05795A1
Wallis, T. (2004). Psychological tests are not really measuring what they claim to measure: A re-evaluation of the concept of construct validity. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.F42F5769
Психологическое измерение : учеб. пособие для вузов, Тюменева, Ю. А., 2007
Психологическое тестирование, Анастази, А., 2002
Психологическое тестирование, Анастази, А., 2003
Психологическое тестирование, Анастази, А., 2007

Recommended Additional Bibliography

Measurement and evaluation in psychology and education, Thorndike, R. M., 2014
Rust, J., & Golombok, S. (2014). Modern Psychometrics : The Science of Psychological Assessment (Vol. 3rd ed). London: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=812538

Authors

Antipkina Inna Veniaminovna

Master’s Programme 'Science of Learning and Assessment'

Methodology of Measurеment in Psychology and Education