Khanmigo doesn’t answer students’ questions directly, but starts with questions of its own, such as asking if the student has any ideas on how to find an answer. It then guides them step by step to a solution, with hints and encouragement.
Despite Khan’s expansive vision of “awesome” personal tutors for every student on the planet, DiCerbo assigns Khanmigo a more limited teaching role. When students are working independently on a skill or concept but are stuck or stuck in a cognitive rut, she says, “we want to help students get unstuck.”
About 100,000 students and teachers tested Khanmigo in schools across the country last school year, allowing them to detect hallucinations from the bot and allowing DiCerbo and her team to analyze a large number of conversations between students and the bot.
“We pay attention to things like summarizing, giving hints and encouragement,” she explains.
The extent to which Khanmigo has closed the AI engagement gap is not yet known. Khan Academy plans to release some summary data on student-bot interactions later this summer, DiCerbo said. Plans for outside researchers to assess the tutor’s impact on learning will take longer.
AI feedback works both ways
Since 2021, the nonprofit Saga Education has also been experimenting with AI feedback to help teachers better engage and motivate students. In 2023, the Saga team, along with researchers from the University of Memphis and the University of Colorado, fed transcripts of their math tutoring sessions into an AI model trained to recognize when the teacher prompted students to explain their reasoning, refine their answers, or engage in deeper discussion. The AI analyzed how often each teacher took these steps.
They monitored approximately 2,300 tutoring sessions over several weeks and found that tutors whose coaches used AI feedback were significantly more likely to include these types of prompts in their sessions to encourage student engagement.
While Saga is exploring the possibility of AI providing feedback directly to teachers, the company is proceeding cautiously. According to Brent Milne, vice president of product research and development at Saga Education, “having a human coach in the group is really valuable to us.”
Experts expect AI’s role in education to grow, with interactions becoming more human-like. Earlier this year, OpenAI and startup Hume AI separately launched “emotionally intelligent” AI that analyzes voice tone and facial expressions to infer a user’s mood and respond with calibrated “empathy.” But even emotionally intelligent AI will likely fall short when it comes to student engagement, said Michael Littman, a Brown University computer science professor and the National Science Foundation’s division director for information and intelligent systems.
No matter how human the conversation, he says, students understand at a fundamental level that AI doesn’t really care about them, what they have to say in their writing, whether they pass or fail classes. In turn, students will never really care about the bot and what it thinks. A June study in the journal Learning and Instruction found that AI can already provide decent feedback on student essays. What’s not clear is whether student writers will put in the care and effort, rather than handing the task off to a bot, if AI becomes the primary audience for their work.
“There's incredible value in the human relationship part of learning,” Littman says, “and if you just take people out of the equation, something gets lost.”
This story about AI tutors was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.
