Are Better Schools All We Need?

Learning from the Harlem Children's Zone

Roland Fryer (second from left) found that most of the HCZ's success in student performance can be chalked up to its charter schools, and not to the social programs championed by HCZ's founder, Geoffrey Canada (far right).

A foundational belief at the Harlem Children’s Zone is that erasing the achievement gap between black and white kids requires a comprehensive, resource-intensive approach. The idea is that middle-class white kids don’t just have better schools, but the kind of home lives that lend themselves to better school performance. HCZ is essentially trying to fill in all those gaps with a  robust array of social services. More health screenings. More early childhood programs for parents and children. Students at HCZ’s charters have longer school days and shorter summers.

But what if the better schools were enough to erase the achievement gap? A new study by Harvard’s Roland Fryer and Will Dobbie found that the Harlem Children’s Zone’s successes in relation to student performance comes almost entirely from its schools, and that the benefits from the social programs are pretty insignificant.

Both lottery and instrumental variable identification strategies suggest that the effects of attending an HCZ middle school are enough to close the black-white achievement gap in mathematics. The effects in elementary school are large enough to close the racial achievement gap in both mathematics and ELA. We conclude with evidence that suggests high-quality schools are enough to significantly increase academic achievement among the poor. Community programs appear neither necessary nor sufficient.

In a weird way, this might actually help propagate some of HCZ’s principles. The big dilemma with their kitchen-sink approach, if proven successful (and this Harvard study certainly won’t be the last word on whether that is or isn’t the case) is that it would be hard, if not impossible, to scale up to a whole district. Very few cities would have the political will to do so, and few cities that had the will would have the means. (HCZ’s operating budget in 2010: $84 million.)  Nor would they have Geoffrey Canada, HCZ’s influential leader, to guide their implementation. But they could replicate, somewhat, the way HCZ’s schools themselves function. (The Obama administration’s efforts to copy the HCZ neighborhood-wide model have been relatively small bore.)

But there are plenty of arguments for HCZ to keep going ahead with all of their social services, scalability or no. That asthma screening program has no effect on classroom performance, but it’s still a worthwhile endeavor. (The CDC took an early look at the asthma initiative a few years ago — Harlem’s asthma rate is five times the national average —  and found that the rate of school absences and emergency or unscheduled doctor’s visits dropped dramatically for kids enrolled in the asthma program.) You’d think that less absences would mean better school performance, but even if it doesn’t, there are still a bunch of kids in Harlem who are much less sick.

I’m ambivalent about HCZ’s profile in national policymaking relative to its modest successes, but that seems worthy of some applause.



Gene "G.D." Demby is the founder and editor of PostBourgie. In his day job, he blogs and reports on race and ethnicity for NPR's Code Switch team.
  • -k-

    I appreciate your argument for keeping social services. I’m worried about the way this data will be interpreted in policy contexts- it’s absolutely asinine to suggest that medical benefits, nutrition programs, and home/school/community connections–e.g., things we know to be of tremendous value when it comes to improving kids’ educational experiences–should be thought of as ‘perks’, but these conclusions could very easily be reformulated and repackaged as making just that argument. It frightens me that we think it’s okay to parse out educational outcomes (which, let’s be real, amount to performance on high-stakes tests) from children’s overall well-being, the implication being that it’s acceptable to provide support for just one piece.

    The other thing that I think is easily overlook here is that these schools aren’t just better because they have, like, new textbooks. They’re hugely different on many counts, and my suspicion is that they’ll be difficult to recreate even if we’re just funneling the budget into school (vs. community) programs. You’re gonna see a lot of articles in the next few years where folks who are working with HCZ data try to tease out which factors are most important. Roland Fryer is obviously one of them, and when I saw him talk last year, he was oddly dismissive of leadership and school culture. He was particularly insistent that you don’t need a bunch of Geoff Canadas to replicate what’s going on in these schools–I get why he’d want to say that, just as I get why it could be beneficial to identify parts of HCZ that may not be ‘necessary’, but man, it makes me uneasy.

  • Jeremy

    A number of important things to consider:

    The only evidence the paper provides to argue that the community programs have no effect are twofold: 1) Students do not benefit more from attending an HCZ school if they live closer to the zone (where services are offered); and 2) that non-HCZ schooled siblings of HCZ-schooled students, presumably having access to the services, do not display improved scores. That’s it; that’s all the evidence and analysis.

    We have no ‘measure’ of the quality, extent, or capacity of any of the social service programs that HCZ provides. Maybe community programs have no effect because they aren’t managed as well as the schools. Perhaps better managed or better funded community programs would produce different impacts. We have no way to know this by design of the study.

    We also have no way to know, by the design of this study, whether or not high performing students are involved with community programs not affiliated with HCZ. New York is a big city; there are many social services offered as the nonprofit industry continues to grow.

    We also do not know, given the study’s design, whether the programs had any impact on Harlem’s local social structure outside of individually measured student achievement in Math and ELA. Perhaps things in Harlem would be a lot worse off without the 20 HCZ programs. In other words, what is our counterfactual? Maybe Harlem is better off with the programs present; we don’t know from this study.

    Nor, to get to the heart of the matter, should we have any reason to believe that the closer one lives to the zone, the greater access they have to the social services therein. Perhaps high achieving students that live further away stick around after school to participate in the programs, and their household distance doesn’t change the fact that they go to school everyday where the programs are located. This assumption effectively drives the entire paper’s conclusions, and should be treated with some healthy skepticism.

    • Thanks for raising these methodological issues, Jeremy — I had a similar reaction when I read the study but wondered if I’d missed something. At most, the results show students don’t need the specific services offered by HCZ (and as you note, maybe don’t show even that) — the design doesn’t control for whether students had access to substitute (equivalent or even superior) health care, social services, etc. from other providers.

      I’m also wary of treating test scores as a sole proxy of educational achievement anyway, but I grant that’s a different battle. As the study repeatedly notes, its findings only hold if you take the scores at “face value” (and see pp. 175-76). Also, if I’m reading correctly (around p. 172), the scores improved most for kids who came in with higher scores — and declined for kids who came in with below-median scores (which could, I’d imagine, itself be a proxy for poorer health, attendance, etc.)