Category Archives: Educational Research

Dyslexia, Education, education reform, Educational Research, NCTQ, No Excuses Reform, NPR, reading, Science of Reading

How Media Misinformation Became “Holy Text”: The Anatomy of the SOR Movement

[Header Photo by Nathan Dumlao on Unsplash]

[NOTE: A PDF of this post as a presentation can be accessed HERE. See also a slightly revised presentation HERE. Please do not edit and please acknowledge this is my work if you use for instructional or public purposes.]

On the Social Media Platform Formerly Known as “Twitter,” Mark Weber posed a simple but powerful question:

The answer is the anatomy of how media misinformation in 2018 wrapped in sensationalistic anecdotes has been replicated uncritically by dozens and dozens of journalists, resulting in that misinformation becoming “holy text,” or in other words, sacrosanct Truth.

Here, I offer the template that “Hard Words” created, and unlike journalists, I include links to research showing why the claims throughout the piece (and in its cousin, “Sold a Story”) are both false and shoddy journalism.

I.

The article begins with the Big Lie, one of the three biggest lies (along with citing the NRP report and NCTQ reports) in the “science of reading” (SOR) movement:

This is a fundamental misunderstanding of NAEP data. NAEP “proficient” is well above grade level, but “basic” is approximately what most states consider “grade level,” and thus, if anything, about 60-65% of students for several decades have been at or above grade level. That isn’t sensational enough for reporters, however.

The Evidence:

From NAEP:

NAEP student achievement levels are performance standards that describe what students should know and be able to do. Results are reported as percentages of students performing at or above three NAEP achievement levels (NAEP Basic, NAEP Proficient, and NAEP Advanced). Students performing at or above the NAEP Proficient level on NAEP assessments demonstrate solid academic performance and competency over challenging subject matter. It should be noted that the NAEP Proficient achievement level does not represent grade level proficiency as determined by other assessment standards (e.g., state or district assessments). See short descriptions of NAEP achievement levels for each assessment subject.

Scale Scores and NAEP Achievement Levels

Loveless, T. (2016, June 13). The NAEP proficiency myth. Brookings. https://www.brookings.edu/blog/brown-center-chalkboard/2016/06/13/the-naep-proficiency-myth/

Loveless, T. (2023, June 11). Literacy and NAEP proficient (Web log). https://tomloveless.com/posts/literacy-and-naep-proficient/

II.

Next, the SOR movement is squarely grounded in the conservative education reform movement, specifically framing poverty as an “excuse”:

The Evidence:

Read carefully, Coles unpacks “Hard Words” in the context of false claims about poverty: Cryonics Phonics: Inequality’s Little Helper, Gerald Coles

See also: The Crumbling Facade of “No Excuses” and Educational Racism

III.

As I have pointed out about NCTQ (see more below on NCTQ and LETRS/Moats), much of the SOR advocacy has a market interest behind it and the SOR movement is grounded in the myth of the bad teacher, attacking classroom teachers and teacher educators:

The Evidence:

The High Cost of Marketing Educational Crisis [UPDATED]

The Myth of the Bad Teacher: 2023

IV.

Here and throughout mainstream media, including “Sold a Story,” the SOR movement relies on anecdotes, regardless of how well those stories reflect accurate claims:

The Evidence:

Claims of miracles in Pennsylvania (similar to those made about Mississippi) fall apart once the full picture is examined. Inflated gains at early grades routinely disappear in later grades; this score increases are mirages, not miracle, and ironically, the NRP report showed that reality despite SOR advocates ignoring that fact; see again: Cryonics Phonics: Inequality’s Little Helper, Gerald Coles.

V.

A persistent set of lies in the SOR media campaign concerns misrepresenting “guessing” and three cueing:

The Evidence:

Understanding MSV: The Types of Information Available to Readers

Is Reading a “Guessing Game”?: Reading Theory as a Debate, Not Settled Science

Compton-Lilly, C.F., Mitra, A., Guay, M., & Spence, L.K. (2020). A confluence of complexity: Intersections among reading theory, neuroscience, and observations of young readers. Reading Research Quarterly, 55(S1), S185-S195. https://doi.org/10.1002/rrq.348 [access HERE]

Mora, J.K. (2023, July 3). To cue or not to cue: Is that the question? Language Magazine. https://www.languagemagazine.com/2023/07/03/to-cue-or-not-to-cue-is-that-the-question/

VI.

The SOR misinformation campaign relies on making false claims and definitions about balanced literacy (and whole language, see below):

The Evidence:

Spiegel, D. (1998). Silver bullets, babies, and bath water: Literature response groups in a balanced literacy program. The Reading Teacher, 52(2), 114-124. www.jstor.org/stable/20202025

VII.

The misrepresentation of whole language also has a marketing element; Moats markets SOR-branded materials and thus has a financial interest in discrediting BL and WL:

The Evidence:

Krashen, S. (2002). Defending whole language: The limits of phonics instruction and the efficacy of whole language instruction. Reading Improvement, 39(1), 32-42. http://www.sdkrashen.com/content/articles/2002_defending_whole_language.pdf

Semingson, P. & Kerns, W. (2021). Where is the evidence? Looking back to Jeanne Chall and enduring debates about the science of reading. Reading Research Quarterly, 56(S1), S157-S169. https://doi.org/10.1002/rrq.405.

VIII.

Misrepresenting WL/BL is solidly linked to a complete misreading of the NRP reports (another Big Lie):

The Evidence:

Krashen, S. (2002). Whole language and the great plummet of 1987-92. Phi Delta Kappan, 83(10), 748-753.

McQuillan, J. (1998). The literary crisis: False claims, real solutions. Heinemann.

From Cryonics Phonics: Inequality’s Little Helper, Gerald Coles:

See a wealth of evidence that the NRP is regularly misrepresented by SOR advocacy:

IX.

As many scholars have noted, the SOR movement including “Sold a Story” is driven by sensationalistic anecdotes, stories:

The Evidence:

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. https://doi.org/10.1002/rrq.353

X.

SOR advocacy regularly demands only a narrow use of “scientific” in reading instruction while also endorsing practices and programs not supported by that same rigor, such as LETRS:

The Evidence:

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. https://doi.org/10.1002/rrq.353

Research Roundup: LETRS 

XI.

Much of the sensationalistic media coverage is self-contradictory, placing overstated claims of “scientific” beside “impossible to know”:

See also Hanford’s coverage of Mississippi:

XII.

A third Big Lie is using unscientific and discredited reports from the conservative think tank NCTQ to claim that teacher educators are incompetent and/or willfully misleading teacher candidates.

The Evidence:

Thomas, P.L. (2023, September). NEPC review: Teacher prep review: Strengthening elementary reading instruction. Boulder, CO: National Education Policy Center. https://nepc.colorado.edu/review/teacher-prep

Dudley-Marling, C., Stevens, L. P., & Gurn, A. (2007, April). A critical policy analysis and response to the report of the National Council on Teacher Quality (NCTQ). NCTE. https://ncte.org/resources/reports/critical-policy-analysis-response-nctq-report/

Benner, S. M. (2012). Quality in student teaching: Flawed research leads to unsound recommendations. Boulder, CO: National Education Policy Center. https://nepc.colorado.edu/thinktank/review-student-teaching

Fuller, E. J. (2014). Shaky methods, shaky motives: A critique of the National Council of Teacher Quality’s review of teacher preparation programs. Journal of Teacher Education, 65(1), 63-77. https://journals.sagepub.com/doi/10.1177/0022487113503872

Cochran-Smith, M., Stern, R., Sánchez, J.G., Miller, A., Keefe, E.S., Fernández, M.B., Chang, W., Carney, M.C., Burton, S., & Baker, M. (2016). Holding teacher preparation accountable: A review of claims and evidence. Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/publication/teacher-prep

Cochran-Smith, M., Keefe, E.S., Chang, W.C., & Carney, M.C. (2018). NEPC Review: “2018 Teacher Prep Review.” Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/thinktank/review-teacher-prep-2018

Burke, K. J., & DeLeon, A. (2020). Wooden dolls and disarray: Rethinking United States’ teacher education to the side of quantification. Critical Studies in Education, 61(4), 480-495. https://www.tandfonline.com/doi/abs/10.1080/17508487.2018.1506351

Stillman, J., & Schultz, K. (2021). NEPC Review: “2020 Teacher Prep Review: Clinical Practice and Classroom Management.” Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/thinktank/teacher-prep

Thomas, P.L., & Goering, C.Z. (2016). Review of “Learning about Learning: What Every New Teacher Needs to Know.” Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/thinktank/review-teacher-education

XIII.

One of the most damning aspects of the SOR movement has been the embracing of and rise in grade retention policies; grade retention is not supported by research and both creates false test score gains while harming children:

The Evidence:

Mississippi Miracle, Mirage, or Political Lie?: 2019 NAEP Reading Scores Prompt Questions, Not Answers [Update 7 December 2022]

A Critical Examination of Grade Retention as Reading Policy (OEA)

Grade Retention Advocacy Fails by Omission

Gaming the System with Grade Retention: The Politics of Reading Crisis Pt. 3

Beware Grade-Level Reading and the Cult of Proficiency

What Do We Really Know about Reading Proficiency in the US?

Understanding and Reforming the Reading Proficiency Trap

XIV.

The SOR movement grossly overstates brain science as well as the essential nature of science:

The Evidence:

Seidenberg, M.S., Cooper Borkenhagen, M., & Kearns, D.M. (2020). Lost in translation? Challenges in connecting reading science and educational practice. Reading Research Quarterly, 55(S1), S119-S130. https://doi.org/10.1002/rrq.341

Yaden, D.B., Reinking, D., & Smagorinsky, P. (2021). The trouble with binaries: A perspective on the science of reading. Reading Research Quarterly, 56(S1), S119-S129. https://doi.org/10.1002/rrq.402

XV.

The SOR movement has hyper-focused on dyslexia, but again, mostly offering misinformation:

The Evidence:

Johnston, P., & Scanlon, D. (2021). An examination of dyslexia research and instruction with policy implications. Literacy Research: Theory, Method, and Practice, 70(1), 107-128. https://doi.org/10.1177/23813377211024625

International Literacy Association. (2016). Research advisory: Dyslexia. https://www.literacyworldwide.org/docs/default-source/where-we-stand/ila-dyslexia-research-advisory.pdf

Socioeconomic dissociations in the neural and cognitive bases of reading disorders, Rachel R. Romeo, Tyler K. Perrachione, Halie A. Olson, Kelly K. Halverson, John D. E. Gabrieli, and Joanna A. Christodoulou

Stevens, E. A., Austin, C., Moore, C., Scammacca, N., Boucher, A. N., & Vaughn, S. (2021). Current state of the evidence: Examining the effects of Orton-Gillingham reading interventions for students with or at risk for word-level reading disabilities. Exceptional Children, 87(4), 397–417. https://doi.org/10.1177/0014402921993406

Hall, C., et al. (2022, September 13). Forty years of reading intervention research for elementary students with or at risk for dyslexia: A systematic review and meta-analysis. Reading Research Quarterly. https://doi.org/10.1002/rrq.477

Odegard, T. N., Farris, E. A., Middleton, A. E., Oslund, E., & Rimrodt-Frierson, S. (2020). Characteristics of Students Identified With Dyslexia Within the Context of State Legislation. Journal of Learning Disabilities, 53(5), 366–379. https://doi.org/10.1177/0022219420914551

“Sold a Story” became a “holy text” because dozens of journalists and politicians repeated the misinformation and lies begun in “Hard Words,” identified above.

This is not good journalism, but it does prove that sensationalistic stories will ultimately trump evidence, even the “science” SOR advocates are so apt to reference.

Recommended

ILEC Response: Mainstream media coverage of reading proficiency, teachers of reading, NAEP scores, and teacher preparation

Dyslexia, Education, education reform, Educational Research, reading, Science of Reading

The Zombie Politics of Marketing Phonics: “There Is a Sucker Born Every Minute”

Consider the following claims about reading proficiency in students and the teaching of reading in the US:

  • No one teaches phonics.
  • There is a phonemic awareness crisis.
  • Direct, systematic, and sequential phonics is the only way to go.
  • Decodable texts are important.

I suspect that most people concerned about education and reading who pay even a modicum of attention to mainstream media will find these claims not only applicable to the current state of reading but also true.

However, there is a problem, which prompted this post from Rachael Gabriel:

As Richard Allington details, these claims are simply not scientific, ironically, even as advocates of the “science of reading” repeat claims that have been standard but misleading arguments for decades.

Since at least the 1940s, these phonics-centered claims have been compelling for the media, the public, and more recently political leaders; yet, “there is no indisputable evidence of a national crisis in reading, and even if there were a crisis, there is no evidence that the amount of phonics in classrooms is necessarily the cause or the solution,” as shown by David Reinking, George G. Hruby, and Victoria J. Risko.

At the core of the phonics frenzy is market, and as Allington noted in the late 1990s, “There is a sucker born every minute.”

I recommend reading Allington’s piece in full and the following reader for context and much more complex and accurate understanding of reading proficiency and the teaching of reading:

Education, education reform, Educational Research, grade retention, reading, Science of Reading

LNL: The Debate Over Grade-Level Reading

LNL: The Debate Over Grade-Level Reading

See Also

A Critical Examination of Grade Retention as Reading Policy (OEA)

Grade Retention Advocacy Fails by Omission

Gaming the System with Grade Retention: The Politics of Reading Crisis Pt. 3

Grade Retention Harms Children, Corrupts Test Data, But Not a Miracle: Mississippi Edition

Beware Grade-Level Reading and the Cult of Proficiency

What Do We Really Know about Reading Proficiency in the US?

Understanding and Reforming the Reading Proficiency Trap

Education, education reform, Educational Research, Lou LaBrant, NCTE, reading, Science of Reading

Deja Vu All Over Again: The Never Ending Pursuit of “Scientific” Instruction

Writing in NCTE’s Elementary English (known as Language Arts since 1975), Lou LaBrant offered a bold proclamation that resonates still today: “This is not the time for the teacher of any language to follow the line of least resistance, to teach without the fullest possible knowledge of the implications of his medium” (1947, p. 94).

LaBrant entered the classroom in 1906, and after experiencing forced retirement in her 60s, she found ways to remain in the field at historically Black colleges, finally retiring fully in 1971 from Dillard University. This impressively long career sits at the center of an impressively long life, living until she was 102 after writing her memoir at 100.

The embodiment of Deweyian Progressivism, LaBrant was equally demanding of herself as she was of others—particularly educators. Her high standards and blunt speaking and writing style make her appealing and often intimidating.

Her piece from 1947 also includes other statements I have repeated in my public and scholarly work:

A brief consideration will indicate reasons for the considerable gap between the research currently available and the utilization of that research in school programs and methods…. (p. 87)

It is not strange, in view of the extensive literature on language, that the teacher tends to fall back upon the textbook as authority, unmindful of the fact that the writer of the text may himself be ignorant of the basis for his study. (pp. 88-89)

LaBrant, L. (1947, January). Research in language. Elementary English, 24(1), 86-94. http://www.jstor.org/stable/41383425

Seventy-six years later, LaBrant could just as easily be speaking into the current “science of reading” (SOR) debate that centers research (“science”) and the imbalance of authority often conceded to reading programs.

Some, in fact, may be compelled to assume LaBrant would be an outspoken advocate for SOR. However, LaBrant’s scholarship and practice offer a window into why the SOR movement is misguided and misleading, specifically about the central role of pursuing “scientific” instruction.

To understand that the current SOR is a misuse of the term “scientific” we should reach back a bit farther in LaBrant’s career to 1931:

The cause for my wrath is not new or single. It is of slow growth and has many characteristics. It is known to many as a variation of the project method; to me, as the soap performance. With the project, neatly defined by theorizing educators as “a purposeful activity carried to a successful conclusion,” I know better than to be at war. With what passes for purposeful activity and is unfortunately carried to a conclusion because it will kill time, I have much to complain. To be, for a moment, coherent: I am disturbed by the practice, much more common than our publications would indicate, of using the carving of little toy boats and castles, the dressing of quaint dolls, the pasting of advertising pictures, and the manipulation of clay and soap as the teaching of English literature. (p. 245)

LaBrant, L. (1931, March). Masquerading. The English Journal, 20(3), 244-246. http://www.jstor.org/stable/803664

In the first couple decades of the 1900s, John Dewey practiced and developed a progressive approach to teaching and learning that was grounded in his call for scientific instruction and holistic approaches to education. Many associate Dewey with “learning by doing,” a relatively fair summary but one that is ripe for misapplication.

Similar to what has been repeated in educational practice for at least a century, William Heard Kilpatrick seized onto Dewey’s concept but packaged it as the Project Method, the source of LaBrant’s “wrath” in 1931.

Dewey’s progressive education philosophy has a very odd history that includes progressivism routinely being blamed for educational failure even though public education in the US being historically and currently deeply traditional and conservative (read Kohn on this paradox).

Two dynamics are at play.

First, formal public education in the US has mostly grounded practice in efficiency since the 1920s—packing as many students per teacher into the classroom as possible and structuring curriculum and instruction around commercial programs and standardized testing.

Second, progressive “scientific” is much more complex and nuanced than current and narrow uses of “scientific” in the SOR movement.

Dewey and LaBrant were advocates for teacher autonomy and authority, which rested on the expectation that teachers know the current evidence base (the “science”) of their filed of literacy but in the context of their day-to-day classroom practice. Both, for example, would strongly reject teaching reading through a commercial reading program of any kind.

Dewey’s progressivism, then, is tethered to the real world in front of the teacher—student behaviors and classroom dynamics.

Philosophy and theory (based on evidence, some of which is generated by the scientific process) provide the teacher with a place to start instruction; however, the evidence in front of the teacher during the act of teaching perpetually shapes practice.

Dewey advocated for “scientific” teaching as an ongoing experiment, not teaching grounded to a template derived from a narrow body of experimental and quasi-experimental research.

If LaBrant were alive today, she would be writing pieces very similar to her 1931 diatribe about the project method, but targeting the SOR movement and the deeply unscientific legislation and practices that movement has spawned: testing students with nonsense words, grade retention, scripted reading programs, one-size-fits-all systematic phonics, LETRS training, NAEP data, “miracle” claims, and more.

Yes, as LaBrant lamented in 1947, public education has a long history of a “considerable gap” between research (“science”) and classroom practice, but another problem sitting between better instruction and greater learning by students is the never ending pursuit of “scientific” instruction that weaponizes “science” and fails to acknowledge the most powerful messages of Dewey’s progressivism—teaching and learning must be focused on the real students sitting in front of teachers daily.

Those unique and diverse students are best served by teachers who teach as scientists perform science—starting with informed hypotheses, implementing instructional practices, developing temporal and unique theories for each student, and adjusting practice based on that evidence for the benefit of each student.

Progressive ideas of “science” are ways to navigate the world in informed and practical ways; conversely, the SOR movement has once again reduced “scientific” to an ideological and political baseball bat used to batter anyone not conforming to their misinformation.

Although LaBrant left us over three decades ago, I can feel her wrath for the SOR movement growing somewhere in the universe, and regret we do not have her voice still to guide us—but we do have her words: “This is not the time for the teacher of any language to follow the line of least resistance, to teach without the fullest possible knowledge of the implications of his medium” (1947, p. 94).

Censorship, Education, education reform, Educational Research, literacy, literature, NCTE, reading, Science of Reading

Schedule: Fall 2023 – Winter/Spring 2024

Below I will keep an updated listing of presentations and other public work for Fall 2023 through Spring 2024.

I am available for webinars, podcasts, presentations, white papers, blog posts, etc., on a number of education and literacy topics (browse my blog posts for topics):

  • Censorship, CRT/Curriculum Bans
  • Reading Legislation/Policy, “Science of Reading”
  • Writing
  • Education Reform
  • Politics and Education
  • NCTQ

New York State Reading Association

Leadership Workshop: Making Sense of the Science of Reading

August 5, 2023, 12:45 – 1:45

Furman University/ Cultural Life Program

Title: Censorship in the Palmetto State: A Panel Discussion

Date: October 5

Time: 6:30 PM

Location: McEachern Lecture Hall – Furman Hall 214

Description: For years, we have witnessed increased attacks on books centered around LGBTQIA, race, offensive language, and more. While public and school librarians have received much backlash from the complaints, librarians, politicians, and community advocates have partnered in solidarity to help remove access barriers. Join our panel to discuss the harm of banned books, learn how community members can support librarians in their fight for intellectual freedom, and discuss the importance of standing against censorship to promote literacy to everyone who seeks to expand their knowledge. We encourage you to bring any questions you may have.  

Title: Libraries are Worthwhile: Why We Need Them and How We Will Keep Them

Date: October 10

Time: 7:00 PM

Location: Hartness Pavilion 

Description: Emily Drabinski, interim chief librarian at The Graduate Center, City University of New York and the 2023-2024 president of the American Library Association (ALA) will give a talk on the importance of libraries and librarians and how we can protect them in the face of ongoing censorship attempts.

NCTE Annual Conference

Conexiones 2023

Columbus, OH – November 16-19, 2023

Keep on Reading for a Free World: Reconnecting through Literacy and Literature (Roundtable) – 11/17/2023 12:30 – 1:45; Aminah Robinson Grand Ballroom B [Reading Wars and Censorship: A Long and Shared History click for PDF]

Connecting Teachers with their Professional Autonomy in the “Science of Reading” Era click for PDF (Presentation) – 11/18/2023 – 11:00 – 12:15; A-214/215

LitCon 2024

Columbus, OH – January 27-30, 2024

Sessions

Featured Speaker

Where Are We Going, Where Have We Been?: Prioritizing Teacher Autonomy in the SOR Era

Download PP HERE

Over the last decade, states have passed new or revised reading legislation, often grounded in the “science of reading” (SOR) movement. The SOR movement has perpetuated many oversimplified and misleading stories that portray teachers negatively. This featured session will prioritize teacher autonomy by exploring the following topics: reading crisis, NAEP reading data, reading programs, teacher training and LETRS, dyslexia, and the complicated full body of reading research.

Sunday, January 28, 3:15 pm – 4:15 pm

Monday, January 29 4:15 – 5:45 pm

SCCTE 2024

West Beach Conference Center at Kiawah Island Resort, Kiawah, SC from Friday-Saturday, February 2-3, 2024

February 2, 2024, 9:30-10:30

Which Is Valid, SOR Story or Scholarly Criticism?: Checking for the “Science” in the “Science of Reading”

P.L. Thomas, Professor of Education, Furman University

Download PP HERE

The “science of reading” (SOR) movement has shifted from media stories to state legislation and instructional policy. This workshop invites teachers to critically examine media claims about reading, teachers of reading, and teacher educators against the full body of reading science. The topics will include history of reading crises, the simple view of reading, NAEP, the Mississippi “miracle,” balanced literacy and reading programs, dyslexia, three cueing, brain science, and an overview of reading science.

2024 COE Winter Education Forum

6:30 – 8:00 EST

Buyer Beware: Avoiding the Unintended (But Predictable) Consequences of SOR Legislation [access PDF here]

2024 Illinois Reading Council Conference

March 14-15, 2024 – Springfield, Illinois

Program

Everything You Know Is Wrong: SOR Edition

[Access PDF HERE]

Friday March 15 8:30-9:30

The “science of reading” movement has perpetuated several compelling and highly influential stories about reading; however, much of those claims are misleading or even completely false. This session will examine some of those stories and claims in the context of the full body of evidence. Topics include NAEP reading data, grade retention, the Mississippi “miracle,” phonics research, dyslexia, teacher education (NCTQ), multiple cueing, and reading programs and theories (balanced literacy).

Reclaiming Teacher Authority and Autonomy in the SOR Era: When Structured Literacy Becomes a Script

[Access PDF HERE]

Friday March 15 9:45-10:45; 2:15-3:15

Increasingly since 2013, states have adopted reading legislation identified as the “science of reading.” Since curriculum and instruction should be driven by classroom teachers, not media narratives, parental advocacy, or political mandate, this session examines key reading topics framed with current research to support teacher authority and autonomy.

BustED Pencils LIVE – Monday, March 25th, 2024

USOS: The Politics and Reality of the “Science” of Reading

Education, education reform, Educational Research, miracle schools, No Excuses Reform, Poverty, race, racism, Social Context Reform, Teaching, Testing

Moving Beyond the Cult of Pedagogy in Education Reform

As a teacher for forty years and a teacher educator for more than half of that career, I have always struggled with the tendency to oversell teacher quality and instructional practice.

Does teacher quality matter? Of course.

Does instructional practice matter? Again, of course.

But both teacher quality and instruction (pedagogy) are dwarfed by teaching and learning conditions within schools and more significantly by the conditions of any child’s life.

As I have noted recently, the peak era of focusing on teacher quality, the value-added movement (VAM) occurring mostly under the Obama administration, instead of identifying high-quality teachers as a driver for improving student achievement found out something much different than intended:

VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.

ASA Statement on Using Value-Added Models for Educational Assessment (2014)

Teacher quality necessarily includes two types of knowledge by a teacher—content knowledge and pedagogical knowledge.

Yet the VAM experiment revealed something we have known for decades—standardized tests of student learning mostly reflect the student’s relative privilege or inequity outside of school.

Despite the refrain of Secretary Duncan under Obama, schools have never in fact been “game changers.”

While neoliberal/conservative education reforms leveraged the “soft bigotry of low expectations” and unsubstantiated claims that the Left uses poverty as an excuse, people all along the ideological spectrum are over-focused on instructional practices. And that overemphasis is used to keep everyone looking at teachers, students, and instruction instead of those more impactful out-of-school (OOS) influences on student learning.

A companion to the cult of pedagogy in education reform is the “miracle” school claim, but “miracle” schools rarely (almost never) exist once the claim is interrogated, and even if a “miracle” school exists, it is by definition an outlier and essentially offers no guidance for scaling outward or upward.

The paradox of the cult of pedagogy in education reform is that until will directly address OOS factor we will never have the context for better teasing out the importance of teacher quality and instructional practices.

The current education reform trapped in the cult of pedagogy is the “science of reading” (SOR) movement which oversells the blame for student reading achievement as well as oversells the solutions in the form of different reading programs, reading instructional practice, and teacher preparation and professional development.

The “miracle” of the day in the SOR propaganda is Mississippi, which is very likely a mirage based on manipulating the age of students being tested at grade level and not on teacher quality and instructional practices.

Not a single education reform promise since the 1980s has succeeded, and the US remains in a constant cycle of crisis and reform promises.

Yet, the evidence is overwhelming that many OOS factors impact negatively student learning and that social reform would pay huge dividends in educational outcomes if we simply would move beyond the cult of pedagogy in education reform.

For example, see the following:

My entire career has existed within the neoliberal accountability era of education reform that oversells education as a “game changer” and oversells teacher quality and instructional practices.

Like time-share frauds, we are being duped, and teachers and students need us to move beyond the cult of pedagogy in education reform and focus on the much larger influences on students being able to learn and teachers being able to show that their quality and instruction can matter.

Education, education reform, Educational Research, teacher preparation, Teaching

Critically Reconsidering Teacher Education (and NCTQ’s Shoddy Reports): A Reader

In 2018, a simplistic but compelling story was established: Teachers do not know how to teach children to read (60+% are not proficient readers!) because teacher educators have failed to teach the “science of reading” (SOR) in teacher prep programs.

These false narratives about teacher ed, NAEP data, and reading have gained momentum and now drive reading policy and legislation in practically every state in the US.

There is an ironic truism—“a lie can travel halfway around the world while the truth is still putting on its shoes”—most often misattributed to Mark Twain that certainly describes the misguided SOR movement’s central claims wrapped up in the initial mantra that SOR is both simple and settled.

Here are two complicated counter-points that are supported by the full body of evidence:

  • Reading instruction can and should be significantly reformed (in the context of addressing wider systemic inequities), but the SOR version of causes and solutions are false.
  • Teacher education can and should be significantly reformed , but the SOR version of causes and solutions are false.

I have been a strong advocate for education reform, beginning with my entering the field in 1984, and subsequently a strong advocate for teacher education reform, starting with entering higher education and teacher education in 2002.

In “Of Rocks and Hard Places—The Challenge of Maxine Greene’s
Mystification in Teacher Education,” I wrote about teacher education: “As teacher educators, we are trapped between the expectations of a traditional and mechanistic field and the contrasting expectations of best practice guided by critical pedagogy.”

Below, then, I offer a reader about critically reconsidering teacher education and why the use of NCTQ “reports” are misguided and fail the test of scientific evidence.

Teacher Education

NCTQ

Education, education reform, Educational Research, NAEP, reading, SAT, Science of Reading, Statistics, Testing

SAT Lessons Never Learned: NAEP Edition

Yesterday, I spent an hour on the phone with the producer of a national news series.

I realized afterward that much of the conversation reminded me of dozens of similar conversations with journalists throughout my 40-year career as an educator because I had to carefully and repeatedly clarify what standardized tests do and mean.

Annually for more than the first half of my career, I had to watch as the US slipped into Education Crisis mode when SAT scores were released.

Throughout the past five decades, I have been strongly anti-testing and anti-grades, but most of my public and scholarly work challenging testing addressed the many problems with the SAT—and notably how the media, public, and politicians misunderstand and misuse SAT data.

See these for example:

Over many years of critically analyzing SAT data as well as the media/public/political responses to the college entrance exam, many key lessons emerged that include the following:

  • Lesson: Populations being tested impact data drawn from tests. The SAT originally served the needs of elite students, often those seeking Ivey League educations. However, over the twentieth century, increasingly many students began taking the SAT for a variety of reasons (scholarships and athletics, for example). The shift in population of students being tested from an elite subset (the upper end of the normal curve) to a more statistically “normal” population necessarily drove the average down (a statistical fact that has nothing to do with school or student quality). While statistically valid, dropping SAT scores because of population shifts created media problems (see below); therefore, the College Board recentered the scoring of the SAT.
  • Lesson: Ranking by test data must account for population differences among students tested. Reporting in the media of average SAT scores for the nation and by states created a misleading narrative about school quality. Part of that messaging was grounded in the SAT reporting average SAT scores by ranking states, and then, media reporting SAT average scores as a valid assessment of state educational quality. The College Board eventually issued a caution: “Educators, the media and others should…not rank or rate teachers, educational institutions, districts or states solely on the basis of aggregate scores derived from tests that are intended primarily as a measure of individual students.” However, the media continued to rank states using SAT average scores. SAT data has always been strongly correlated with parental income, parental level of education, and characteristics of students such as gender and race. But a significant driver of average SAT scores also included rates of participation among states. See for example a comparison I did among SC, NC, and MS (the latter having a higher poverty rate and higher average SAT because of a much lower participation rate, including mostly elite students):
  • Lesson: Conclusions drawn from test data must acknowledge purpose of test being used (see Gerald Bracey). The SAT has one very narrow purpose—predicting first-year college grades; and the SAT has primarily one use—a data point for college admission based on its sole purpose. However, historically, media/public/political responses to the SAT have used the data to evaluate state educational quality and the longitudinal progress of US students in general. In short, SAT data has been routinely misused because most people misunderstand its purpose.

Recently, the significance of the SAT has declined, students taking the ACT at a higher rate and more colleges going test-optional, but the nation has shifted to panicking over NAEP data instead.

The rise in significance of NAEP includes the focus on “proficiency” included in NCLB mandates (which required all states to have 100% student proficiency by 2014).

The problem now is that media/public/political responses to NAEP mimic the exact mistakes during the hyper-focus on the SAT.

NAEP, like the SAT, then, needs a moment of reckoning also.

Instead of helping public and political messaging about education and education reform, NAEP has perpetuated the very worst stories about educational crisis. That is in part because there is no standard for “proficiency” and because NAEP was designed to provide a check against state assessments that could set cut scores and levels of achievement as they wanted:

Since states have different content standards and use different tests and different methods for setting cut scores, obviously the meaning of proficient varies among the states. Under NCLB, states are free to set their own standards for proficiency, which is one reason why AYP school failure rates vary so widely across the states. It’s a lot harder for students to achieve proficiency in a state that has set that standard at a high level than it is in a state that has set it lower. Indeed, even if students in two schools in two different states have exactly the same achievement, one school could find itself on a failed-AYP list simply because it is located in the state whose standard for proficient is higher than the other state’s….

Under NCLB all states must administer NAEP every other year in reading and mathematics in grades 4 and 8, starting in 2003. The idea is to use NAEP as a “check” on states’ assessment results under NCLB or as a benchmark for judging states’ definitions of proficient. If, for example, a state reports a very high percentage of proficient students on its state math test but its performance on math NAEP reveals a low percentage of proficient students, the inference would be that this state has set a relatively easy standard for math proficiency and is trying to “game” NCLB.

What’s Proficient?: The No Child Left Behind Act and the Many Meanings of Proficiency

In other words, NAEP was designed as a federal oversight of state assessments and not an evaluation tool to standardize “proficient” or to support education reform, instruction, or learning.

As a result, NAEP, as the SAT/ACT has done for years, feeds a constant education crisis cycle that also fuels concurrent cycles of education reform and education legislation that has become increasingly authoritarian (mandating specific practices and programs as well as banning practices and programs).

With the lessons from the SAT above, then, NAEP reform should include the following:

  • Standardizing “proficient” and shifting from grade-level to age-level metrics.
  • Ending state rankings and comparisons based on NAEP average scores.
  • Changing testing population of students by age level instead of grade level (addressing impact of grade retention, which is a form of state’s “gaming the system” that NAEP sought to correct). NAEP testing should include children in an annual band of birth months/years regardless of grade level.
  • Providing better explanations and guidance for reporting and understanding NAEP scores in the context of longitudinal data.
  • Developing a collaborative relationship between federal and state education departments and among state education departments.

While I remain a strong skeptic of the value of standardized testing, and I recognize that we over-test students in the US, I urge NAEP reform and that we have a NAEP reckoning for the sake of students, teachers, and public education.

Recommended

Literacy and NAEP Proficient, Tom Loveless

The NAEP proficiency myth, Tom Loveless