In a series of recent blog posts, Jay Greene sounds a note of caution for school choice advocates looking to create not just more options, but better ones.
Regulation, he argues, might not be the best way to ensure school quality — especially if regulators relay too heavily on test scores to make weighty decisions like shutting down schools. Test scores, he writes, aren’t always perfect proxy for school performance.
There is no question that growth in student learning provides us with some useful information. The problem is that school quality is much broader than just test score results. I always understood that achievement tests were only a partial and imperfect indicator of school quality, but I used to believe that other aspects of school quality not captured by achievement tests were largely correlated with those test results. That is, I used to think that if a school raised scores it probably meant that students were safer, more students would graduate, more students would learn productive values, and more students would go on to become successful adults.
Unfortunately, the evidence is increasingly clear that test scores are only weakly correlated with all of these other desirable outcomes from schools. All you have to do is look at yesterday’s post. Schools that produce the largest achievement test gains are not necessarily the ones that produce higher graduation, or college-attendance rates. And sometimes schools with unimpressive achievement gains make significant contributions to attainment and annual earnings when students join the workforce. I used to think that this couldn’t be possible. All of these happy outcomes had to be aligned. They just aren’t.
Matt Barnum responded with a review of the research for the Thomas B. Fordham Institute. He finds some evidence schools that are good at raising test scores can also improve other outcomes for those students, like helping more of them reach college. But like Greene, he finds the correlation between test score improvement and other outcomes is generally weak.
This might call for regulatory humility. If it’s hard to judge a school’s performance by test scores alone, regulators should take a light touch, focusing on things like student safety and truth-in-labeling. Schools will close if parents and students don’t think they’re being well-served.
Or it might call for a more nuanced approach to regulation. Test scores and test-score gains do provide some useful information. But charter school authorizers or portfolio managers should look at a wider range of information before deciding whether to shut down a school.
Around the country, there are still just a few large-scale experiments in school choice, from New Orleans’ heavily regulated, nearly all-charter school system to Nevada’s loosely regulated, free-market approach to education savings accounts. It would help if more cities and states adopted major school choice programs, and if the impact of those programs were judged not just by their ability to help students raise test scores, but to help students excel later in life.
