New AI Tutor Achieves 0.71-1.30 SD Effect Size in Dartmouth Course

Researchers at Dartmouth deployed an AI tutor in an introductory computer science course and measured effect sizes of 0.71 to 1.30 standard deviations on learning outcomes. The paper, presented at the 2026 InTextbooks workshop, compares the AI tutor to standard instruction. The results suggest that LLM-based tutoring can substantially outperform traditional methods in controlled classroom settings.

Key Findings

• Effect size range: 0.71 to 1.30 SD across different assessment types
• Study conducted in a Dartmouth introductory CS course
• AI tutor likely leverages Socratic-style hints and code feedback via LLM
• Control group received standard instruction without the AI tutor

While the exact architecture is not fully detailed in the PDF snippet, the effect sizes are large enough to be practically significant. An effect size of 1.0 SD typically corresponds to moving an average student from the 50th to about the 84th percentile.

Who This Matters For

Developers building educational agents or tutoring systems for coding. Also relevant for AI researchers evaluating real-world LLM impact.