$1,000 can raise a class’s test scores by as much as cutting class size by a third.
An emergency situation that turned out to be mostly a false alarm led a lot of schools in Los Angeles to install air filters, and something strange happened: Test scores went up. By a lot. And the gains were sustained in the subsequent year rather than fading away.
That’s what NYU’s Michael Gilraine finds in a new working paper titled “Air Filters, Pollution, and Student Achievement” that looks at the surprising consequences of the Aliso Canyon gas leak in 2015.
The impact of the air filters is strikingly large given what a simple change we’re talking about. The school district didn’t reengineer the school buildings or make dramatic education reforms; they just installed $700 commercially available filters that you could plug into any room in the country. But it’s consistent with a growing literature on the cognitive impact of air pollution, which finds that everyone from chess players to baseball umpires to workers in a pear-packing factory suffer deteriorations in performance when the air is more polluted.
If Gilraine’s result holds up to further scrutiny, he will have identified what’s probably the single most cost-effective education policy intervention — one that should have particularly large benefits for low-income children.
And while it’s too hasty to draw sweeping conclusions on the basis of one study, it would be incredibly cheap to have a few cities experiment with installing air filters in some of their schools to get more data and draw clearer conclusions about exactly how much of a difference this makes.
The Aliso Canyon gas leak, explained
Back on October 23, 2015, employees of the Southern California Gas Company discovered a massive leak in the Aliso Canyon natural gas storage facility near Porter Ranch in the San Fernando Valley. Significant for the larger purposes of the study, the Porter Ranch area is known for having “some of the cleanest air in the Valley year-round.”
The gas leak was a huge catastrophe from the standpoint of greenhouse gas emissions, but also naturally raised concerns in the local community about the immediate impact on public health.
Facing political pressure from concerned parents and teachers, Gilraine writes, “the Los Angeles Unified School District (LAUSD) and the owner of the gas well, the Southern California Gas Company, placed air filters in every classroom, office and common area in all schools within five miles of the gas leak at the end of January 2016.”
Strikingly, however, air testing conducted around the time of the installation of the filters shows that the schools didn’t actually have abnormally high levels of the kinds of pollution that are normally associated with natural gas. Methane is lighter than air, and by the time the filters were installed — nearly three months after the leak — the extra pollution caused was all the way up in the sky and not affecting school buildings.
Consequently, the installation of the filters served not to remove extra contamination caused by the leak, but simply to clean up the normal amount of background indoor air pollution present in the Valley. That lets Gilraine estimate the difference in student performance for schools just inside the boundary compared to those just outside.
He finds that math scores went up by 0.20 standard deviations and English scores by 0.18 standard deviations, and the results hold up even when you control for “detailed student demographics, including residential ZIP Code fixed effects that help control for a student’s exposure to pollution at home.”
For context, this is comparable in scale to some of the most optimistic studies on the potential benefits of smaller class sizes, with Alan Krueger finding that cutting class size by a third leads to a 0.22 standard deviation improvement in academic performance. Other studies find smaller or even negative effects (because adding teachers means bringing in less experienced or less effective ones), but even accepting the positive findings, it costs much more than $700 per classroom to achieve class size reductions of that scale.
This is a big, but not implausible, number
The effect Gilraine finds is strikingly large given that it’s a seemingly trivial intervention.
But Sefi Roth of the London School of Economics studied university students’ test performance relative to air pollution levels on the day of the test alone. He found that taking a test in a filtered rather than unfiltered room would raise test scores by 0.09 standard deviations. That’s about half the impact Gilraine found, just based on day-of-test air quality. In Gilraine’s natural experiment, students benefited from cleaner air for about four months. Given that context, it’s not incredibly surprising that you could see an impact that’s about twice as large.
What’s natural to ask — though unknowable from the study before us — is how much more change we could see if students benefited from an entire school year of clean air. Or perhaps an entire school career, from pre-K through high school graduation, of clean air.
One striking thing about this is the government has long been aware that indoor air pollution is a potential problem. But according to currently prevailing Indoor Air Quality standards, there was nothing wrong with the air in the schools. Filters were installed because of an essentially unwarranted panic about natural gas.
And while Los Angeles is a fairly high-pollution part of the country, outdoor particulate levels are higher in many areas — including New York, Chicago, and Houston — than they were in the impacted neighborhood. In other words, there’s no reason to think the impacted schools were unusually deficient in their air quality. They just happen to be the ones that installed filters.
A cheap, scalable initiative
For a sense of scale, Mathematica Policy Research’s best evidence on the effectiveness of the highly touted KIPP charter school network finds that after three years at KIPP there is significant improvement on three out of four test metrics — up 0.25 standard deviations on one English test, 0.22 standard deviations on another, and 0.28 standard deviations on one of two math tests.
Those are big gains, and they help explain why there is so much enthusiasm about KIPP in some quarters, even as charter schools remain politically controversial and charters in general seem to produce roughly average results.
This is bigger than the impact of letting kids benefit from clean air for four months. But installing the full suite of air filters costs about $1,000 per classroom, and continuing to operate them beyond the first year is cheaper than that. And best of all, unlike totally reworking school operations, it could be scaled up very quickly.
It would be almost trivially easy to get a variety of school districts all around the country to randomly select schools for the installation of air filters. That would rapidly generate a ton of additional data, and if the results continued to be promising, the initiative could be made universal very quickly.
The benefits, on their face, would be extremely large at a relatively low cost. And since air pollution is generally worse in lower-income communities, you would not only raise test scores nationally, but make progress on the big socioeconomic gaps in student achievement that have proven very difficult to remedy.