New Report Offers Clearest Picture Yet Of Pandemic Impact On Student Learning

Dec 1, 2020
Originally published on December 1, 2020 9:13 am

A sweeping new review of national test data suggests the pandemic-driven jump to online learning has had little impact on children's reading growth and has only somewhat slowed gains in math. That positive news comes from the testing nonprofit NWEA and covers nearly 4.4 million U.S. students in grades three through eight. But the report also includes a worrying caveat: Many of the nation's most vulnerable students are missing from the data.

"Preliminary fall data suggests that, on average, students are faring better than we had feared," says Beth Tarasawa, head of research at NWEA, in a news release accompanying the report.

"While there's some good news here, we want to stress that not all students are represented in the data, especially from our most marginalized communities."

Until now, estimates of learning loss have been just that — estimates or projections, based on the kind of academic backsliding schools see after a long summer. This report offers the clearest picture yet of the impact that the past eight months of disruption have had on student learning.

The MAP Growth test

The data at the heart of NWEA's report come from what's known to teachers and children alike as the MAP Growth test — a check-in assessment used to measure kids' math and reading skills that's generally given three times a year, in fall, winter and spring.

While millions of students took these MAP tests in the winter of 2020, few took them again in the spring as schools raced (and many struggled) to provide learning online. But this fall, nearly 4.4 million children did take the test, either from home or back in a classroom. And the results give researchers a vital new data point: a measure of where students are right now.

Tarasawa and her research team studied the data a few different ways. First, they compared students' performance this fall — in, say, third-grade reading — with the performance of a different group of students who took third-grade reading in the fall of 2019, before the coronavirus pandemic.

Tarasawa tells NPR that with this method of comparison, the results in reading were "relatively optimistic" because "kids on average are performing similarly to how [other children] did pre-pandemic." In math, the current pandemic class of students performed about 5 to 10 percentile points lower than the pre-pandemic comparison group — what Tarasawa describes as a "moderate" drop.

In addition to comparing two different groups of students, researchers also studied students' individual growth over time, looking at where they were when they took the MAP test in the winter of 2020 and comparing it with where they are now, in the fall of 2020.

"We saw, on average, students showed growth in both math and reading across the grade levels in almost all grades," says Tarasawa. "Most students made some learning gains in both reading and math since COVID started."

In short, students kept learning when schools shifted online; they just didn't learn quite as much in math as they likely would have if there had never been a pandemic.

Mitigating the learning loss that is happening will still require patience and a thoughtful approach, says Aaliyah Samuel, NWEA's executive vice president of government affairs and partnerships.

"Addressing the unfinished learning is going to be a matter of time. We really need to be thinking about the supports and interventions for kids over at least a two- or three-year runway."

Depending on the depth of learning lost, school districts could consider a range of options, including extending the school year or even enlisting a volunteer tutoring corps.

Roughly a quarter of students missing

The "good" news (and the not-so-good news) in this report also comes with an important and worrying red flag.

In an effort to be sure their 4.4 million-student sample, albeit large, was also representative of America's classrooms, NWEA researchers dug into the demographics of this new data set and compared it with the earlier fall 2019 test data — a sample of nearly 5.2 million children.

What they found, Tarasawa says, is that roughly a quarter of students were missing — meaning they didn't take the MAP test this fall — and that these children are "more likely to be black and brown, more likely to be from high-poverty schools and more likely to have lower performance in the first place."

The researchers cite a host of possible reasons these students weren't able to take the latest test, including a lack of technology or Internet access at home as well as the possibility that some children have disengaged from school more broadly.

"This is screaming that we have to be very cautious," says Tarasawa, about interpreting the relatively optimistic results in reading and even math as evidence that the kids are all right.

"It's just like any time you get a new puzzle," Samuel says. "The first thing you do is ... you start to look for the corners because those are usually the easiest to put together first."

That's where we're at now, she explains: building the edges of the puzzle.

Copyright 2020 NPR. To see more, visit https://www.npr.org.

RACHEL MARTIN, HOST:

Roughly 56 million children in this country were projected to attend school this fall. Some have returned to their classrooms, although many others are going to classes online from home. A new report out today measures how much kids are actually learning during all this pandemic disruption. For a majority of them, the study says things are actually not as bad as feared. That is not true, though, for the most vulnerable. NPR education correspondent Cory Turner is here. Good morning, Cory.

CORY TURNER, BYLINE: Good morning, Rachel.

MARTIN: First off, how do you even measure something like how much kids have lost, how much learning they have lost because of the pandemic?

TURNER: Yeah, well, you do it with this thing called the MAP Growth test. It's a low-stakes test that millions of kids take a few times every year in fall, winter and spring. My kids take it. It is the academic equivalent of a thermometer, basically. It's a way school districts can take their kids' temperature when it comes to learning so you can figure out who's doing well, who's struggling and even see, at least in terms of reading and math, what's tripping them up. Now, most kids did not take the MAP last spring because of the pandemic, obviously, but millions did take it this fall, not that long ago. And it's those results now that give us the first real temperature check we've had for kids mid-pandemic.

MARTIN: And what are the results here?

TURNER: Yeah. So let's start with reading. Researchers compared kids' skills this fall in, let's say, third-grade reading to the performance of a different group of kids who took third-grade reading before the pandemic. And what they found really surprised them. So looking at their sample of more than 4 million kids, the pandemic had little effect on reading skill. According to the report, kids on average are performing similarly to how other kids did pre-pandemic. Beth Tarasawa is head of research at NWEA. That's the not-for-profit behind the MAP test. And here she is describing her initial reaction to that.

BETH TARASAWA: Wow, this wasn't as bad as we predicted and others had, particularly in reading. All things considered, given the craziness that we are in, there was some optimism.

TURNER: Now, in math, Rachel, the news wasn't quite as optimistic. The current pandemic class of students performed about five to 10 percentile points lower than the pre-pandemic comparison group. But that's still what Tarasawa describes as a moderate drop. It's also important to note when comparing kids to their own performance on these tests, they're not literally falling backwards. They're still making gains in reading and math both, just not at the same pace in math that they were before the pandemic.

MARTIN: Interesting. So, yeah, some reason for optimism, but, I mean, this is an important thing to talk about here. You have done a lot of reporting on really vulnerable students, Cory, who are still struggling to even just log on each day, I mean, even if they have access to a computer, right? I mean, how could they even take a test like this?

TURNER: Yeah, this is one of the big questions I had when I first got this report. And the short answer is we're not sure because many of our most vulnerable kids clearly didn't take this test. Beth Tarasawa, again, the head of research at NWEA, told me roughly a quarter of students were missing from their sample. And she said that these children are, in her words, more likely to be Black and brown, more likely to be from high-poverty schools and more likely to have lower performance in the first place. And so she told me this fact is screaming that we have to be very cautious about making too much of this good news. And, you know, Rachel, it could be a warning sign of what we already know about this pandemic, that it hurts most those who have the least.

MARTIN: NPR's Cory Turner, thank you.

TURNER: You're welcome. Transcript provided by NPR, Copyright NPR.