American Economic Review published RCT findings for Saga Education's high-dosage math tutoring of low-income 9th & 10th graders in Chicago. Quick take: High-quality RCTs find sizable effect on math achievement that persists 1-2 years after tutoring ends.
Program:
-
Saga provided math tutoring to 9th & 10th graders in low-income high schools - 50 min/day, 5 days/week for 1 schoolyear. Tutors were recent college grads, trained & paid a modest stipend. Tutors worked with 2 students at a time. Cost was approximately $4800/tutored student.
Study Design:
-
2 RCTs evaluated Saga at scale in 15 high-poverty high schools, with a pooled sample of 5,343 students randomly assigned to treatment vs control (>95% Black or Hispanic, 87% low-income).
-
Only about 40% of treatment group (T) students received Saga tutoring due to scheduling & related issues, so the studies estimated impacts for both the full T group (“intent to treat”) & those T students who received at least 1 tutoring session (“treatment on treated”).
Pooled Findings of the 2 RCTs:
-
The RCTs found sizable effects on math achievement at end of the tutoring year, but what's remarkable is that effects persisted. At end of 11th grade — 1-2 yrs after tutoring ended — the effect on district test scores was 0.23 for tutored students (treatment on treated) & 0.10 for the full T group including untutored students (intent to treat).
-
The gain of 0.23 for tutored students represents a 2.6-fold improvement over the annual gain in math otherwise expected for 11th graders. Saga also increased tutored students' 11th grade math GPA by a quarter point. These effects were statistically significant, p<0.01.
-
The RCTs found no discernible effects on behavior, absenteeism or high school graduation rates. The first RCT also tested a cognitive-behavioral program for males (Becoming a Man) in 2 treatment arms, & found no discernible effect on academic outcomes, suspensions, or arrests.
Comment:
-
Based on careful review, these were well-conducted RCTs (e.g., excellent baseline balance, moderate sample attrition but not differential, valid analyses). Overall, I think the findings are very encouraging.