AI for IR: Sentiment Analysis of Survey Data

Session Abstract: Institutional researchers often use open-ended survey questions to capture the depth and nuance of the student experience, but analyzing this rich qualitative data at scale presents a significant challenge. This session directly addresses this challenge and compares three sentiment analysis models applied to survey responses: TextBlob (lexicon-based, the default for sentiment analysis in ChatGPT), VADER (rule-based), and RoBERTa (a state-of-the-art transformer-based deep-learning model). Evaluation against human-coded results and using standard machine learning metrics showed that the transformer-based deep learning model -- RoBERTa significantly outperformed the traditional models. This session helps IR professionals make informed choices about adopting AI-assisted methods that balance rigor and feasibility. Attendees will learn how to integrate sentiment analysis into their workflows, combining human coding with automated methods to more efficiently analyze student feedback.

Keywords: sentiment analysis, qualitative Data, machine learning, deep learning, artificial intelligence