ATL Standard

Georgia State University issued the following announcement on Mar. 21. 

Georgia State’s pioneering, artificial intelligence-enhanced chatbot “Pounce” is well established as an institutional tool for helping incoming students navigate the thorny world of finances, registration and just getting started in college. Now, Georgia State is showing student performance jumps when classes employ the chatbot to keep them connected.

Receiving direct text messages about their class assignments, academic supports and course content increased the likelihood students would earn a B or higher and, for first-generation students, increased their likelihood of passing the class. First-generation students receiving the messages earned final grades about 11 points higher than their peers.

“Eleven points is more than a full letter grade, and a full letter grade can be the difference between students holding onto their HOPE Scholarship and Pell Grant awards or not,” said Timothy M.  Renick, the founding executive director of the National Institute for Student Success at Georgia State. “These are exceptional results from a fairly light-touch initiative.”

An AI-enhanced text messaging tool in use at Georgia State since spring 2016, the institution-level Pounce chatbot is just one of myriad efforts by the university to support students as they progress through college. Georgia State’s student success efforts have made it a national leader in graduating students from diverse backgrounds and reduced the average time it takes to earn a degree by almost a full semester.

In 2021, the chatbot was integrated for the first time into course content. A group of 500 students in one of Georgia State’s largest courses — political science 1101 — was selected to participate in a randomized controlled trial led by Lindsay Page, the Annenberg Associate Professor of Education Policy at Brown University. Half of the students were selected to receive messages related to the class from the chatbot, and half did not receive the messages.

Page and colleagues Katharine Meyer of Brown and Tyler Walsh of Georgia State presented the findings last week, in a talk titled “Let’s Chat: Chatbot Nudging for Improved Course Performance,” during a conference of the Association for Education Finance and Policy in Denver.

The study was conducted in coordination with Michael Evans, senior lecturer in political science at Georgia State and the faculty coordinator for the course, taken by about 3,000 students each semester to satisfy degree requirements in American government. Students participating in the fall 2021 trial were taking the course fully online.

Evans said the course typically has a relatively high rate of what are known as DFWs (students receiving a grade of D, F or withdrawing) — around 15 percent prior to the COVID-19 pandemic and as high as 25 percent since the onset.

“It’s a challenge because a lot of times students are just learning how to be college students,” Evans said. “We get every type of student, in terms of their preparation.”

Students already get email reminders about assignments, quizzes and exams, but Evans said students receiving text messages from the “PolsPounce” chatbot were getting an extra, direct reminder as well as the opportunity to text questions back, through PolsPounce, to their professor. Some reminders were targeted to students who didn’t have much time left to complete a reading assignment by the deadline. Students were also reminded when an exam was coming up and given an opportunity to take a short practice quiz to gauge their preparedness.

“Clearly this was having a bigger impact than just the emails, and part of that was because they were receiving the message,” Evans said. “I did not expect we’d get this overwhelming evidence that the chatbot is this effective.”

According to Page’s study results, students who received the PolsPounce messages earned grades of B or above at a rate 16 percent higher than those not on the chatbot.

The average final grade in the class for first-generation students in the control group was 64, while similar students receiving the messages earned a final grade an average of 11 points higher.

Among students who were not in their first year of college, those who received the chatbot messages were more likely to pass the class and less likely to drop their other Georgia State classes. They also did better in their other classes, earning fall term grade point averages 0.28 points higher.

Students participating in the study who displayed lower academic performance in high school were more likely to attend supplemental instruction, more likely to earn a B or higher in the class and complete more credits for the semester if they received the chatbot messages.

“While we’ve seen that students from all kinds of different demographic groups benefit, what we’ve seen consistently is that students who struggle the most overall benefit the most from the chatbot intervention,” Renick said. “Students who are low income, the first in their families to go to college or from underrepresented minorities often lack a support system at home or familiarity with how to get help in college. They are the ones to benefit the most by having this tool in their hand 24/7.”

Original source can be found here.

Source: Georgia State University