Which Is Valid, SOR Story or Scholarly Criticism?: Checking for the “Science” in the “Science of Reading”

From late November of 2022 through late February 2023, I have (or will have) presented at 6 major literacy conferences, both national and state level.

Two dominant literary issues have been curriculum/book bans and the “science of reading” (SOR) movement. A few important patterns occurred with the latter.

Many teachers are overwhelmed and discouraged about the heavily negative messaging around SOR, but I also interacted with teachers not fully aware of the magnitude of this movement and remain puzzled about the controversy.

Further, the media, public, and political story around reading and teaching reading is the primary message reaching both educators and the public. The robust scholarly criticism of SOR [1] is often welcomed by teachers and administrators, but unless they are attending conferences, these critique goes unnoticed.

Scholars and educators have been backed into a corner since the SOR story is grounded in a great deal of blame, hyperbole, misinformation, and melodrama.

The media SOR story is simple to the point of being false, but simple in a way that is very compelling for people outside the field of literacy.

Here, I want to put some pieces together, and offer a place to hold the SOR movement/story to the same standards demanded by advocates of SOR (specifically The Reading League).

First, let’s start with the core of the scholarly critiques of mainstream media’s story:

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

While scholarly critiques (see note 1) are far more nuanced and substantive that this central point, this is a manageable way to interrogate whether or not the SOR story is valid based on the standards the movement itself established.

The debate, then, is well represented by conflicting evaluations of SOR and SOR criticism on social media: a literacy scholar and co-author of an SOR reading program called the scholarly criticism “stupid,” and a policy scholar not in literacy noted that the media story is “facile.”

To determine which is valid—the SOR story or the scholarly criticism—that story must be checked against the standards for science established by the movement itself, here from The Reading League:

Finally, the components of the SOR story must be identified in order to check the science behind the claims and the anecdotes; consider Aukerman’s overview:

And my overview in my policy brief:

Below, I outline the SOR story and identify current scientific research, or lack thereof, limiting the evidence to TRL’s guidelines (experimental/ quasi-experimental, published in peer-reviewed journals).

1

First, for the rest of the SOR story to hold up to scientific scrutiny, we must establish whether or not there is a unique reading crisis in the last 10-20 years in which students are failing to learn to read at acceptable rates; this must be true for the blame aspects of the SOR movement to be true.

What is the status of scientific research supporting this claim? There is no current scientific research to support this claim; most scholars have identified that NAEP [2] (and other measures of reading achievement) have remained flat and achievement gaps have remained steady as well for many decades predating the key elements blamed for reading failures.

Note that the age 9 longitudinal data including actual scores appears mostly flat with some fluctuation. (Source)
NAEP age 9 reading appears mostly flat after A Nation at Risk, mostly flat with some trending upward during the 1990s BL era, increasing after the federal pressure of NCLB in 2001, and then flat since the rise of SOR legislation around 2013.

Notably the long-term NAEP data during the recent SOR era for 9 and 13 year olds is relatively flat or unchanged except for lowest performing students:

2

Next, the SOR story claims teachers are not well prepared to teach reading and teacher educators either fail to teach evidence-based methods or willfully ignore the science.

What is the status of scientific research supporting this claim? There is no current scientific research to support this claim [3] although scholars have demonstrated that credible research is available on teacher knowledge of reading and teacher education:

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

3

The media story claims the current settled reading science is the “simple view” of reading.

What is the status of scientific research supporting this claim? Scientific research refutes this claim:

Duke, N.K. & Cartwright, K.B. (2021). The science of reading progresses: Communicating advances beyond the simple view of reading. Reading Research Quarterly, 56(S1), S25-S44. https://doi.org/10.1002/rrq.411

[UPDATE]

Burns, M. K., Duke, N. K., & Cartwright, K. B. (2023). Evaluating components of the active view of reading as intervention targets: Implications for social justice. School Psychology, 38(1), 30–41. https://doi.org/10.1037/spq0000519

4

The SOR story centers a claim that systematic phonics instruction is superior to all other approaches to teaching beginning readers and thus necessary for all students.

What is the status of scientific research supporting this claim? Current scientific research refutes this claims, showing that systematic phonics is no more effective than other approaches (balanced literacy, whole language) and confirming that systematic phonics can increase early pronunciation advantages but without any gains in comprehension and with that advantage disappearing over time:

Bowers, J.S. (2020). Reconsidering the evidence that systematic phonics is more effective than alternative methods of reading instruction. Educational Psychology Review, 32(2020), 681-705. https://link.springer.com/article/10.1007%2Fs10648-019-09515-y

Wyse, D., & Bradbury, A. (2022). Reading wars or reading reconciliation? A critical examination of robust research evidence, curriculum policy and teachers’ practices for teaching phonics and reading. Review of Education10(1), e3314. https://doi.org/10.1002/rev3.3314

Testing the impact of a systematic and rigorous phonics programme on early readers and also those that have fallen behind at the end of Key Stage 2. (2022, October). Education Endowment Foundation. https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/read-write-inc-and-fresh-start

Bowers, J.S. Yes Children Need to Learn Their GPCs but There Really Is Little or No Evidence that Systematic or Explicit Phonics Is Effective: a response to Fletcher, Savage, and Sharon (2020). Educ Psychol Rev 33, 1965–1979 (2021). https://doi.org/10.1007/s10648-021-09602-z

Legislating Phonics: Settled Science or Political Polemics? David Reinking, George G. Hruby, and Victoria J. Risko

5

Mississippi has been heralded in the SOR story as a key example of the success of SOR reading policy, based on MS 2019 grade 4 reading scores.

What is the status of scientific research supporting this claim? There is no current scientific research to support this claim, and the SOR story omits that MS has had steady grade 4 reading improvement since the early 1990s (well before SOR) and that MS grade 8 have remained low, suggesting the grade 4 gains are inflated (see also other states with the grade 4 to 8 drop).

6

The source of low reading proficiency, the SOR story claims, is the dominance of balanced literacy and a core of popular reading programs.

What is the status of scientific research supporting this claim? There is no current scientific research to support this claim. In fact, some of the most criticized programs are only adopted in about 1 in 4 schools suggesting that the variety of programs and practices make these claims overly simplistic at best. Journalists also often misidentify reading programs as balanced literacy that explicitly do not claim that label.

7

Often the SOR story includes a focus on dyslexia, claiming that multi-sensory approaches (such as Orton Gillingham) are necessary for all students identified as dyslexic (and often that all students would benefit from that approach).

What is the status of scientific research supporting this claim? Current scientific research refutes this claims:

Johnston, P., & Scanlon, D. (2021). An examination of dyslexia research and instruction with policy implications. Literacy Research: Theory, Method, and Practice70(1), 107. Retrieved July 26, 2022, from https://doi.org/10.1177/23813377211024625

International Literacy Association. (2016). Research advisory: Dyslexiahttps://www.literacyworldwide.org/docs/default-source/where-we-stand/ila-dyslexia-research-advisory.pdf

Socioeconomic dissociations in the neural and cognitive bases of reading disorders, Rachel R. Romeo, Tyler K. Perrachione, Halie A. Olson, Kelly K. Halverson, John D. E. Gabrieli, and Joanna A. Christodoulou

Stevens, E. A., Austin, C., Moore, C., Scammacca, N., Boucher, A. N., & Vaughn, S. (2021). Current state of the evidence: Examining the effects of Orton-Gillingham reading interventions for students with or at risk for word-level reading disabilities. Exceptional Children87(4), 397–417. https://doi.org/10.1177/0014402921993406

Hall, C., et al. (2022, September 13). Forty years of reading intervention research for elementary students with or at risk for dyslexia: A systematic review and meta-analysis. Reading Research Quarterly. Retrieved October 17, 2022, from https://doi.org/10.1002/rrq.477


As of this post, the claims made in the SOR story are not supported by scientific research, and the criticisms offered by scholars appear valid.

The media story is overstated and oversimplified even though nearly all literacy educators and scholars agree that too many marginalized students (minoritized students, special needs students, impoverished students, MLLs) are being underserved (which is a regrettably historical fact of US education).

The SOR movement has created a predicament for the media story in that the standards being required for teachers and reading policy is an incredibly high and narrow threshold that (as I have shown above) the movement itself has not reached.

Again, scholarly criticism of the SOR story is nuanced and substantive, but at its core, that criticism is best represented by demonstrating that SOR advocates, especially the media, cannot meet the standard they propose for the field of teaching reading.

Simply put, US reading achievement is not uniquely worse now than at nearly any point in the last 80 years, and therefore, blaming balanced literacy as well as popular reading programs proves to be a straw man fallacy.

Reading instruction and achievement, of course, can and should be better. But the current SOR story is mostly anecdote, oversimplified and unsupported claims, and fodder for the education marketplace.

Media is failing students far more so than educators by perpetuating a simplistic blame-game that fuels the education market place.


[1] Media Coverage of SOR [access materials HERE]

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

MacPhee, D., Handsfield, L.J., & Paugh, P. (2021). Conflict or conversation? Media portrayals of the science of reading. Reading Research Quarterly, 55(S1), S145-S155. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.384

Cryonics Phonics: Inequality’s Little Helper – New Politics

[UPDATE]

The Science of Reading and the Media: Is Reporting Biased?, Maren Aukerman, The University of Calgary

The Science of Reading and the Media: Does the Media Draw on High-Quality Reading Research?, Maren Aukerman

The Science of Reading and the Media: How Do Current Reporting Patterns Cause Damage?, Maren Aukerman

[Aukerman three posts as PDF]

Making sense of reading’s forever wars, Leah Durán and Michiko Hikida

[2] Note that NAEP is not grade level, however:

NAEP does not report the percentage of students performing at grade level.  NAEP reports the percentage of students reaching a “proficient” level of performance.  Here’s the problem. That’s not grade level. 

In this post, I hope to convince readers of two things:

1.  Proficient on NAEP does not mean grade level performance.  It’s significantly above that.
2.  Using NAEP’s proficient level as a basis for education policy is a bad idea

The NAEP proficiency myth

[3] Media tend to cite reports from NCTQ that do not meet the standard for “scientific” established by TRL:

NEPC Review: 2020 Teacher Prep Review: Clinical Practice and Classroom Management (National Council on Teacher Quality, October 2020)

NEPC Review: 2018 Teacher Prep Review (National Council on Teacher Quality, April 2018)

NEPC Review: Learning about Learning: What Every New Teacher Needs to Know (National Council on Teacher Quality [NCTQ])