Machine Learning Meets PSM: Evaluating Proactive Advising with DiD

Session Abstract: How do we know if proactive advising truly impacts student persistence and success? Randomized trials are rarely possible in higher education, but institutional researchers can still estimate causal effects using advanced designs. This session demonstrates how to combine propensity score matching (PSM) with difference-in-differences (DiD) to evaluate a district-wide advising initiative. We will walk through defining treatment, building pre-treatment covariates, constructing outcomes, and checking balance and parallel trends. We will also show how machine learning can improve propensity estimation and highlight subgroup differences. Attendees will leave with practical tools to evaluate advising or other student success programs, and strategies for translating findings into actionable insights for leadership.

Keywords: Program Evaluation, Advising, Propensity Score Matching, Difference-in-Differences, Machine Learning