Session Abstract: Institutional researchers often use open-ended survey questions to capture the depth and nuance of the student experience, but analyzing this rich qualitative data at scale presents a significant challenge. This session directly addresses this challenge and compares three sentiment analysis models applied to survey responses: TextBlob (lexicon-based, the default for sentiment analysis in ChatGPT), VADER (rule-based), and RoBERTa (a state-of-the-art transformer-based deep-learning model). Evaluation against human-coded results and using standard machine learning metrics showed that the transformer-based deep learning model -- RoBERTa significantly outperformed the traditional models. This session helps IR professionals make informed choices about adopting AI-assisted methods that balance rigor and feasibility. Attendees will learn how to integrate sentiment analysis into their workflows, combining human coding with automated methods to more efficiently analyze student feedback.
