CICS 'Smart Tutor' Research Aims to Boost Educational Success
CICS researcher Beverly Woolf and colleagues have received a four-year, $2 million grant from the National Science Foundation to develop a "smart tutor" that will use a computer's camera and visual recognition software to detect student feelings, known as affect. The goal is to sense when the learner has become frustrated, confused or bored, for example, so the computer can adjust lessons to bring the student back on track.
Woolf and colleagues had earlier designed a responsive tutoring system that sensed affect through movement plus a computational model. She and colleagues now plan to teach computers to recognize visible signs of mental persistence, or "grit," and to support student users in avoiding or overcoming feelings of failure.
Woolf says, "The system will be taught to recognize facial expressions such as a frown or other sign of distress, identify what students do as they learn, and continuously personalize the learning experience for them while there's still time, before they give up."
For this grant, UMass Amherst's College of Information and Computer Sciences (CICS) will receive about $1 million of the total, led by Woolf and her co-investigator, research fellow Tom Murray. The remainder will go to collaborators including Ivon Arroyo, assistant professor at Worcester Polytechnic Institute; Margrit Betke, expert in video-based interfaces at Boston University; Stan Sclaroff of the video image and computing group and John Baillieul, expert in robotics and control theory, both at Boston University; and John Magee, computer vision researcher at Clark University in Worcester.
They will enlist more than 1,600 New England elementary and high school students who will work in their own classrooms on MathSpring software developed earlier by Woolf and Arroyo as the testbed for these smart tutor experiments. The MathSpring software teaches and assesses student mathematics skills in grades 5-9.
Woolf explains that people who have "grit" typically know how to set goals, manage their emotions and recognize their waning motivation. For example, grit has been instrumental in defining whether students will complete courses or whether military recruits will complete basic training. "Though grit has alternative definitions," Woolf says, "we'll take it as axiomatic that grit includes these positive traits based on passion for long term goals."
She and colleagues will teach a computer to detect facial expressions and other visible evidence of student perseverance and toughness. They will use artificial intelligence and machine learning techniques, along with existing log-file data of every student and tutor action recorded during the learning process: for example, number of seconds before the student responds, number of hints requested, difficulty of the problem and an estimate of the student's apparent mathematical skill.
She adds, "The evidence so far for detecting and measuring grit has been obtained in psychological studies while interviewing people before and after classes. Ours will be the first studies that take measurements during student learning with computers that dynamically evaluate grit, which can change during an activity or unit and require fine-grained assessment techniques to evaluate."
Because most personal computers now come with built-in cameras, the researchers hope to develop real-time software that responds dynamically to students while they learn online through existing instructional systems, with no extra equipment required.
It is increasingly common for schools to assign computer-based lessons and courses, Woolf says, which may save money needed for extra help or textbooks, but the quality is uneven. "This is one of the problems in education today. There is no way a teacher in front of 30 students can offer personalized education. We believe we can improve learning by teaching the computer to recognize when students need more help, to identify students in distress or lacking motivation and to know when students are about to give up."
"We'll test interventions to determine which ones work for students by age, gender, achievement level and other variables, to make education really personalized. We hope that after four years, we will improve face and gesture recognition by computer, teach computers to recognize grit and persistence and ultimately improve learning and outcomes for a broad range students."