Last week Diane Ravitch warned her readers not to trust the findings of the latest Center for Research on Education Outcomes (CREDO) report highlighting student achievement in charter schools. To make her case, she cited a review of the CREDO study written by the National Education Policy Center (NEPC). Diane Ravitch citing NEPC struck me as humorous for a very good reason.
As the CREDO study results changed direction in favor of charter schools, both Ravitch and NECP took a u-turn of their own.
Now there is nothing wrong with changing your mind. In fact, I applaud people who review research and reassess their previous held beliefs, because it takes considerable courage to change your mind or admit you were wrong. However, the CREDO research methods didn’t change and neither did the concerns raised by NEPC. The only thing that did change was the CREDO results, and it no longer supported Ravitch’s or NEPC’s professional opinion.
Back in 2009, the CREDO report concluded, “Despite promising results in a number of states and within certain subgroups, the overall findings of this report indicate a disturbing — and far reaching — subset of poorly performing charter schools.”
When Ravitch accepted the National Education Association’s “Friend of Education” award in 2010, she cited CREDO findings stating, “five out of six charters will get no different results or worse results than the regular public schools.” Ravitch was still highlighting CREDO findings as late as the summer of 2011, but today she attacks CREDO as part of a corporate education reform plot to privatize education.
Like Ravitch, the NEPC had nice things to say about CREDO back in 2009. Despite very little change in NEPC’s own concerns, or even in the CREDO methodology, the organization’s opinion turned decidedly negative in 2013 when the CREDO results shifted in favor of charter schools.
In 2009, NEPC wrote that the “CREDO study strengthens the well-established, broader body of evidence showing average charter performance to be equal to, or perhaps lower than, the performance of traditional schools …” [emphasis added] NEPC did find areas to critique and described these problems as “some weaknesses” and “commonplace limitations.” But it rated the study as pretty solid overall: “The relative strength and comprehensiveness of the data set used for this study, as well as the solid analytic approaches of the researchers, makes this report a useful contribution to the charter school research base.”
In 2013, with the results more favorable to charter schools, NEPC’s largely similar criticisms were suddenly stated with more forcefulness. They wrote, “There are significant reasons for caution in interpreting the results.” [emphasis added] Regarding the now-positive impacts of attending a charter school, NEPC writes “these effects are so small as to be regarded, without hyperbole, as trivial.” Of the research methodology, NEPC concluded: “This review has noted reasons for caution when making inferences to a true causal effect of charter schools.”
To be clear, the CREDO methodology did not change in any significant way in the intervening years and neither did the size of the observed effects. The results and effect sizes were no less “trivial” in 2009, though the NEPC concluded charter schools could be performing worse than public schools. But instead of agreeing with CREDO’s observations, like it did in 2009, NEPC concludes that charter schools are “essentially indistinguishable from public schools in terms of their effects on academic performance.”
To be fair to the NEPC, the critiques in 2009 and 2013 were written by different authors. But consistency of language and tone matters. The contradictory attitudes coincide with a change in results more favorable to charter schools, and that might tarnish a reasonable person’s opinion of the NEPC.
I think Diane Ravitch is wrong on this one, a point I brought up on her blog was that we can’t just like info when it supports our positions. That we should be striving for the right answer not to just make sure our answer is right. Though I wonder why you didn’t mention that much of the improvement credited to charter schools is because low performing ones have been closing (after taking public money and providing poor educations to many of our children).
In fact I think we should go with the CREDO, which says by the way, despite huge advantages, that charter schools students are lagging behind their public school peers in Florida.
I submit since charter schools benefit from selection bias, the ability to counsel out under achievers and discipline problems, educate fewer numbers of ESOL, ESE, and kids on free and reduced lunch and the ability to put requirements on parents, they are failures even if they do as well as public schools and most aren’t. Charter schools should be killing public schools but they are not and that should be troubling to everybody.
Hi Chris,
Thanks for the response. To the first statement some, not all, of the growth was due to closing down bad charter schools. However, these closed charter schools constituted only 8 percent of the charter schools observed. Bad schools – of all types – should be shut down or completely reorganized.
To your last points, that is where the random assignment studies come in handy. Students are randomly selected for a treatment (charter school) or control (traditional public school). In that way we can control for potential biases like the ones you’ve mentioned. I do believe most random assignment charter school study shows benefits for at least one subgroup.
This suggests the alleged selection bias is weak at best but most likely non existent. Don’t forget, public schools don’t have to teach every kid – about 1 percent of public school students are expelled every year and another half percent are sent to segregated specialty schools for troubled kids.
Also take into account that charter schools get about half the funds of a public school and, according to a recent NYT article, the average charter teacher has about 5 years worth of teaching experience compared to public schools with 14.
Finally, charter schools serving mostly ESL and FRL students are the ones that do better than their public school counterparts serving a similar population. Even CREDO said as much back in 2009.
About money, nobody is forced open a charter school, so operators, many of who are profiting nicely, who complain they have less money than public schools are a bit disingenuous, furthermore the upkeep of an entire school system is admittedly more than the upkeep of a school.
As for the CREDO, I wish the schools in Wyoming and New Hampshire and the rest of the country well but I have to concern myself with Florida and the CREDO says the charter schools here, which have fewer esol, ese and free and reduced lunch students aren’t doing so well. I think I would also win on the expulsion argument; more and more reports are coming out how charter schools suspend or expulse out poor performers and kids with discipline problem.
I get it though, you love charter schools but even for people like you who love them wouldn’t slowing down and making sure every charter school is a financially stable, high achieving school be preferable to the mess of corporate raiders, hedge fund managers, Chinese millionaires looking for green cards, cutthroats looking for a quick buck and the few legitimate charter schools we have now? Over 230 have failed in Florida alone. How many lost millions and truncated lives is that?
I am not saying get rid of charter schools. I am saying get it right.
Would you hold the same standards for closing poor performing public schools?
Charter school market share in Florida is less than 7 percent they spend about 25 percent less per pupil. The 2013 CREDO study shows good positive impact growth for charter schools since 2009 though very little difference with traditional public schools today. We should also point out that Florida’s school system as a whole improved since 1996 and charter schools played a role in pushing that overall achievement growth.
A final note in regards to the population of special needs and low-income students. Florida is unusual in that low-income and special needs kids can get scholarships to attend private schools. This most likely has an impact on charter school enrollment. Indeed over in Pinellas County we see very few charter schools in the urban core of St. Pete. Historically, and nationwide, the urban core has been the bread and butter for charter school success. Charter schools don’t usually compete very well with white affluent public schools (which often have more resources than anyone) and that may be what we are seeing in Pinellas and beyond.
By poor performing public schools? Do you mean schools that do poorly on standardized tests, because the answer would be no. There is a relationship between poverty and performance on standardized tests and I think putting in measures that mitigate poverty is the way to go. Charter schools divert resources often from those schools thta can least afford to loose them. Now if there were some public school that lost institutional control then that would be something to think about but that doesn’t really happen in public schools. Sure you hear about the occasional book keeper who pocketed some club dues but that’s nothing compared to what the news has reported about charter school operators gone wild..
You are drawing a correlation between the first charter school and positive growth. What about the class size amendment? Couldn’t that be the reason the state of Florida has improved, if you believe the assertion that our schools have improved. I know people point to graduation rates going up, but they went up nationwide and that probably at least to a degree had something to do with the economy and changes in the job market. Plus as somebody who spent 5 years at public HS I can say it was hard to fail, you really had to work at it. Then I would look at what our ACT college ready rates and how many kids have to take remedial classes in college and those numbers are not very good. A good article would look at those rates historically.
I could also say the same thing about kids who receive vouchers as I do about those that attend charters too. I talked to the guy who did Florida’s study a few weeks back and he said that kids who get them don’t experience (as a group) better education outcomes despite private schools being able to pick who they take and having the advantages that charter schools do. He also said for the most part only those private schools affiliated with churches are doing well. I could tell you stories about private schools having classes run by teachers without degrees.
The answer is to fix the problems in public schools not to replace them with substandard options. Vouchers and charters should only have a limited and defined role as a supplement to public education.
I apologize, I’m confused. You support closing charter schools based on standardized test data but not traditional public schools?
I’m also confused about “diverting resources away from schools” because I thought the purpose of public education was to educate a child, not fund a particular type of school.
As far as the class-size amendment is concerned, the answer is it is possible that it had a positive impact but more likely it had little to no impact. It also cost $20 billion – more than charter schools, tax credit scholarships and McKay scholarships combined. https://www.sciencedirect.com/science/article/pii/S0272775712000271
Class size reduction is not an effective means of improving student achievement and there is a large body of research going back about 40 years on the subject. At best it provides very little bang for the buck.
Please note, fraud in charter schools is about as rare as in public schools. It happens but don’t think public school officials are just taking pocket change. Here is a recent story about a three public school district leaders taking millions in bribes and extra pay. Combined the three had to pay back $5.9 million to the school district and they face 3 to 11 years in prison.
https://www.fbi.gov/newark/press-releases/2012/former-superintendent-of-toms-river-regional-school-district-and-insurance-broker-ordered-to-pay-4.3-million-in-restitution
All three of NEPC’s reviews of CREDO charter school studies have raised the same set of strengths and weaknesses. The more recent reviews even point to the 2009 review in making some of the critiques. The CREDO studies are relatively strong (as charter school research goes) but have noteworthy weaknesses that we should all understand. Most importantly, they have been spun every which-way, by CREDO itself and by advocates on all sides.
The reviews themselves are available online:
2009 National: https://nepc.colorado.edu/thinktank/review-multiple-choice
2013 Michigan: https://nepc.colorado.edu/thinktank/review-charter-performance-michigan
2013 National: https://nepc.colorado.edu/thinktank/review-credo-2013
Thanks for responding and sharing your research.
Yes, NEPC raised the same concerns in 2009 and 13 as I mentioned. The tone and criticisms were mild in 2009 but more harshly phrased in 2013. When the effects were slightly negative NEPC mentioned they were still small but admitted it was possible charter schools could be no different or worse than public schools. When the results were positive, and no less small than in 2009, there was no admission of the possibility that they could now be no different than or slightly better than traditional public schools.
Overall, NEPC praised CREDO and considered the limitations of the report to be “commonplace” but by 2013 those same commonplace limitations suddenly became “significant reasons for caution.” NEPC readers were given no such warning in 2009 which might be why it was so often repeated by people, including Diane Ravitch, opposing charter schools as “proof” that charters were no better than or possibly worse than public schools.
I agree, highlighting weaknesses in reports is important – as they all have them and certainly some have considerably more problems than others.
Hello Mr. Gibbons. I think the question of the consistency of NEPC’s reviews is legitimate. I also invite readers to visit/read all three reviews and think they will find that — notwithstanding different expert reviewer teams — they are indeed very consistent. In a day or two, I (Kevin Welner) will write up something more detailed. But for now, I will just stress that the main theme of the NEPC reviews is that the overall body of research — and the CREDO reports/studies as part of that research — shows the test-score outcomes of the charter school sector to be essentially indistinguishable from the test-score outcomes of the conventional public school sector. That’s what the 2009 report stressed, and that’s what we’ve been stressing since then.
Hmm, Jeb Bush who sent his children to a 28k a year private school with small classes might disagree with you. Ahh who am I kidding he hates public school teachers. Also isn’t the reason we have charter schools is so children get better education outcomes? To save them from failing public schools? If that’s not the case why do we have them again?
We may have looked at different research because i have read plenty of reports that has said smaller classes are better, also I am a better teacher with 30 kids than I am with 22 or 18 said no teacher ever.
I spoke to former commissioner Gerard Robinson and he said the class size amendment has as much to do with Florida’s success as Bush’s reforms.
As for charter school shenanigans, I have a proposition, I will site 5 charter schools who have had financial hi-jinks for every one public school you can. I will check by to see if you take me up on it.