June 2026 Live Online Short Course

Course Description

The CARMA Introduction to Multilevel Analysis short course provides both (1) the theoretical foundation and (2) the resources and skills necessary to conduct basic multilevel analyses. Emphasis will be placed on techniques for traditional, hierarchically nested data (e.g., students in classrooms; employees in teams). Part 1 of the course introduces issues related to multilevel theory (e.g., multilevel constructs, principles of multilevel theory building). Part 2 discusses issues related to multilevel measurement (e.g., data aggregation; estimating within-group agreement). Part 3 discusses the alignment of multilevel theory, analyses, and inferences (e.g., cross-level inferences; cross-level biases). Part 4 focuses on the basic 2-level model (e.g., students nested in classrooms; soldiers nested in platoons; employees nested in work teams) analyzed using multilevel regression (i.e., random coefficient regression; mixed effects models). The R software package will be used throughout this short course. Participants are encouraged to also bring datasets to the course and apply the principles to their specific areas of research. The course is best suited for faculty and graduate students who are familiar with traditional (i.e., single-level) multiple regression analysis, but have little (if any) expertise with multilevel analyses.

Specific topics will include:

Part 1: Multilevel Theory: Constructs, Inferences, & Composition Models

Part 2: Multilevel Measurement: Estimating Interrater Agreement & Reliability

Part 3: Multilevel Inferences: Data Aggregation, Cross-Level Biases, & Cross-Level Inference

Part 4: Multilevel Analysis: Introduction to the Basic, 2-Level, Multilevel Regression Model

Other topics (only if time permits) might include:

• Extension of the 2-level model to the study of growth and change (i.e., growth model)
• Different centering/scaling strategies (e.g., group-mean centering vs. grand-mean centering)

Required Software:

R ( https://www.r-project.org ), RStudio (https://posit.co/products/open-source/rstudio/?sid=1)

COURSE LEARNING OBJECTIVES

Part 1 – Multilevel Theory: Constructs, Inferences, & Composition Models

• What is “nesting” and how is it related to multilevel data structures?
• What are “top down” and “bottom up” processes, and what are examples of each?
• What is emergence, and how is it related to multilevel theory and measurement?
• What are the distinguishing features of composition and compilation models of emergence?

Part 2 – Multilevel Measurement: Agreement, Reliability, & Data Aggregation

• What is “interrater agreement,” and how is this concept related to data aggregation?
• What is “interrater reliability,” and how is the concept related to data aggregation?
• What statistics are used to estimate interrater agreement (or more generally, within-unit agreement)?
• What are rwg and rwg(j), and how are they used in multilevel research?
• What is ICC(1) and how is it used in multilevel research?
• What is ICC(2) and how is it used in multilevel research?

Part 3 – Multilevel Measurement & Analysis: Aggregation, Bias, & Cross-Level Inference

• What is a “cross-level inference” and how are such inferences related to “cross-level bias”?
• What are the “ecological fallacy” and “atomistic fallacy”? What are examples of each type of fallacy?
• What are the different ways that a researcher could analyze two-level nested data?
• How does one avoid drawing biased inferences?

Part 4 – Multilevel Analysis: Multilevel Regression for 2-Level Nested Models

• What types of hypotheses can be tested using multilevel regression models?
• What is the problem with using OLS with nested data? How does multilevel modeling overcome this problem?
• What are the four basic models that are typically tested using multilevel regression?
  
  • Identify & interpret all fixed effects.
  • Identify & interpret all random effects.
• How does scaling/centering impact the interpretation of random and fixed effects?