Category Archives: Science of Reading

Dyslexia, Education, education reform, reading, Science of Reading

What Do We Really Know about Reading Proficiency in the US?

A data-rich but disappointing report on reading legislation in the US from 2019-2022 has been released by the Shanker Institute.

The report concedes “legislative efforts have at times been criticized widely,” but chooses to applaud the “science of reading” (SOR) movement without considering the considerable scholarly criticism raising cautions about claims of a reading crisis and mandates in that legislation.

Further, the report ignores how the SOR movement fits into decades of political education reform since the 1980s, reforms that have repeatedly failed to produce positive outcomes for students or teachers.

While the report lacks critical grounding, it also offers a couple key points to consider. First:

There are no quick fixes: The path to improvement will require time, consistent investment and a holistic approach to reform. The magnitude of the task should motivate us to persevere and collaborate more effectively. Yet, we are concerned about the polarizing rhetoric surrounding reading and hope that this review can foster a more measured dialogue about the strengths and limitations of state efforts and reading improvement more broadly.

Reading Reform Across America

The emphasis on avoiding one-size-fits all solutions is important and supported by many critics of the SOR movement. And certainly the “polarizing rhetoric” of the SOR debate is harmful; yet, this report’s positive spin on harmful legislation is certain to trigger, not ameliorate that caustic debate.

Valid criticism isn’t any more “polarizing” than idealistic endorsements.

Next, and more importantly for this post:

Whether we see the current state of American students’ reading achievement as a new crisis or as part of a stable trend, the truth remains that more than one-third (37 percent) of the nation’s fourth-graders performed below the National Assessment of Educational Progress (NAEP) “Basic” level in 2022. Because there is no achievement-level description for below “Basic,” it is difficult to make full sense of this statistic.

Reading Reform Across America

Here is the central problem with the SOR movement as well as nine decades of reading wars: The truth is that we know very little empirically about reading proficiency in the US because we have no stable or unified metric or assessment to understand what proficiency is or how well students are developing as readers.

There simply has never been a single day in the US since at least the 1940s that the media, public, and political leaders have declared reading proficiency adequate.

What does it mean to have been in a continual reading crisis in the US for almost a century and yet the country has experienced no major or catastrophic decline?

What does it mean to have been in a continual reading crisis in the US for almost a century because we claim reading is essential for student and societal success and yet the dooms day messaging never materializes?

That leads us to this: What do we really know about reading proficiency in the US?

As the report notes, one aspect of reading proficiency in the US is quite clear and easy to document with multiple data points: Reading proficiency data expose a significant inequity among marginalized groups of students—notably Black and brown students, students in poverty, multi-lingual learners (whom the report advocates for admirably), and special needs students.

Yet this fact about reading is replicated in all other educational measurements, and thus, is not a unique reality about reading proficiency, suggesting something other than reading legislation (or any educational legislation) is needed in the US.

Also, it seems fair and supported by the evidence that we have to note that reading progress by students (how well any students gains reading proficiency in relationship with their peers) is a strong marker for educational progress in general.

While over-emphasizing reading proficiency at grade 3 is problematic, no one suggests that early reading progress should be ignored. Yet, many states persist in adopting harmful grade 3 retention that has been shown to correlate strongly with negative consequences.

The report does concede about grade retention: “Consequently, there are reasons to be cautious about the policy.”

Beyond these two points, however, claims about reading proficiency are at best speculation and at worst ideological assertions without empirical support.

The latter, regretfully, is the crux of most reading wars for decades.

So here is what we don’t have but urgently need in order to address reading in ways that are supportive of students and teachers and avoids the “polarizing rhetoric” with which the report seems deeply concerned:

  • A standardized definition of “proficiency” that is age-based and not grade-based.
  • A comprehensive documentation of reading programs and instructional practices implemented in the US over the last decade.
  • A set of diverse assessments grounded in a standardized definition of “proficiency.”
  • Patience and a willingness to admit that human behaviors occur on a spectrum; not all students learn at the same rates.
  • Reading legislation that neither mandates nor bans practices or policies, but provides a funding framework that supports educators as autonomous professionals.

The polarization in public and political debates about reading is in part driven by all that we do not know and do not have regarding reading proficiency, allowing too many people (some without good intentions) to make melodramatic claims that reinforce political, media, and market interests, not student achievement or teacher/teaching quality.

Ultimately, this current trend in reading legislation is far more dangerous than promising since the decisions being made for teachers and students are not grounded evidence-based claims.

The inequity exposed in data on reading achievement is itself enough to justify that we do something, but continuing to do the same thing over and over while expecting different results is a tremendous political and educational mistake.

We simply do not know what we need to know about reading proficiency, but we do know that reading achievement is not uniquely inequitable; and thus, education reform broadly has failed for decades, and we are far past time to re-evaluate political educational reform.

This report eagerly endorsing more of the same political educational reform; therefore, it fails in its central mission.

A Nation at Risk, Capitalism, Disaster capitalism, Education, education reform, Media, NAEP, Neoliberalism, reading, School choice, Science of Reading

Neoliberal Education Reform: “Science of Reading” Edition

[Header Photo by Mike Erskine on Unsplash]

Cliches become cliches often because they do capture a truth, and “fish don’t know they are in water” may sound trite, but the saying captures well our five decades of education reform in the US.

Since A Nation at Risk under Ronald Reagan and then reinforced and expanded under George W. Bush (with Rod Paige and Margaret Spelling as Secretaries of Education), education reform in the US has been grounded in neoliberal ideology, the foundational beliefs of Republicans and conservatives.

“Neoliberalism” is a challenging term. First, it is hard to define, and second, the use of the word “liberal” has two contrasting meanings in the US—”liberal” as in “classic liberalism” is “conservative” or politically “right,” yet in common usage “liberal” is typically associate with “progressive” or politically “left.”

However, to simplify, in education reform, we can fairly interchange “neoliberal” with “conservative” and “Republican”—even though, as I want to discuss here, it is incredibly important to understand that neoliberal education reform is embraced and perpetuated by both Republicans and Democrats.

Look at the education reform landscape since the 1980s to understand.

A Nation at Risk established the neoliberal education reform playbook: manufacture an education crisis; declare that students, teachers, and public schools are failing; and mandate accountability policies to “fix” students, teachers, and schools (in-school reform only).

Insiders exposed that Reagan gave marching orders to the committee that created A Nation at Risk; Reagan wanted the US to embrace school choice (neoliberalism is a market ideology) and to “put prayer back in schools” (although voluntary prayer has always been allowed in public education, Reagan and Republicans depended on culture wars).

A key component of neoliberal education reform is the buy-in of the media. Until decades later, after numerous scholars discredited the report as a “manufactured crisis,” the media uncritically declared US education—teachers and students—failures.

And thus we set out on several cycles of the same accountability reform grounded in new standards, new tests, and new political mandates.

Governors scrambled to show they took education seriously, and George W. Bush in Texas turned his role as education reform governor into a launching pad for the White House.

Here is another key element.

Although Bush claimed a Texas “miracle,” again as with A Nation at Risk, after the political success and media as well as public buy-in, scholars showed that the “miracle” was a “mirage” (or better yet, a lie).

None the less, Bush took Paige into his administration and No Child Left Behind (NCLB) was modeled in the Texas “miracle”/”mirage”—and just as Democrats rushed to embraced Reagan’s lie, Democrats joyfully made NCLB one of the most prominent federal bi-partisan accomplishments in recent US history.

Few things show how pervasive neoliberal (Republican/conservative) education reform has become the water to the fish (education) than the Barack Obama/Arne Duncan education era.

Instead of ushering in a progressive or critical response to the Bush education policy, Obama/Duncan doubled down—fueling the draconian value-added method era of teacher evaluation, launching the deceptive and austere education career of Michelle Rhee, and supercharging the charter school movement (a “school-choice lite” movement that fulfills the market beliefs of neoliberalism).

At 40 years since A Nation at Risk, all we have to show for the constant reform in education is a series of claims of “crisis” and a smattering of “miracles”—both of which are always manufactured.

But if we reach back further, into the 1940s, we see that neoliberalism also depends on sparking culture wars. For example, the reading wars have always been about attacking progressive/liberal ideologies—Dewey in the early to mid-1900s, whole language in the 1990s, and now, balanced literacy.

So now we come to the “science of reading” (SOR).

SOR has its roots firmly in NCLB and the National Reading Panel (SOR cites the NRP report as much or more than any other evidence)—the peak of neoliberal education reform.

SOR was also fueled throughout the 2000s by the Florida model, which depends heavily on grade retention and laser-focusing on grade 3 reading.

Around 2013, states began to revisit or reimagine reading legislation, but in 2018, the media supercharged the SOR movement, echoing the “manufactured crisis” approach of A Nation at Risk.

Notably, the “manufactured crisis” of the SOR movement is firmly grounded in NAEP testing; first, the media misrepresents NAEP data, and second, NAEP is purposefully designed (the test is a neoliberal tool) to create the veneer of failure by students, teachers, and schools.

NAEP allows media and political leaders to shout that 2/3 of students are not proficient in reading even though that claim isn’t what most people think.

Therefore, at its core, the SOR movement is another neoliberal education reform movement, a tool of Republican/conservative ideology and politics.

SOR has the student/teacher/school failure rhetoric, the “miracle” that is a “mirage,” the eager and uncritical compliance of the media, and the compelling use of standardized tests data (NAEP). But most importantly to understand how SOR is neoliberal education reform, the policies are repackaging Jeb Bush’s Florida model, emphasizing punitive reading policies such as grade retention.

However just like all the other neoliberal education reform since the 1980s, it will not work because it isn’t designed to work.

We are only 20 years since NCLB/NRP which mandated scientifically based reading instruction, yet there is a reading crisis?

Here is the dirty little secret about neoliberal education reform: It is a distraction for political gain.

Neoliberalism keeps the public’s gaze on individuals (students, teachers) and away from systemic forces; SOR wants people to believe that a couple reading programs are to blame for reading failures instead of poverty and inequity.

And the neoliberal attacks in SOR on people are yet another swipe at progressive and critical educators.

Like fish, many educators cannot see they are willing participants in neoliberal education reform; almost all Democrats cannot see they are willing participants in neoliberal education reform.

Fish don’t know they are in water, but with the SOR movement (and whatever crisis comes next), the better analogy may be lobsters in a slowing boiling pot.

Education, education reform, grade retention, Science of Reading

Gaming the System with Grade Retention: The Politics of Reading Crisis Pt. 3

Michael Hiltzik’s How Mississippi gamed its national reading test scores to produce ‘miracle’ gains is the first national media to challenge the Mississippi “miracle,” recently perpetuated again by Nicholas Kristof in the NYT.

Below is a reader that links the key components of the politics of reading crisis:

Note that the connections include policy in Florida (often called the “Florida Model” and anchored by third grade retention) and Mississippi and the influence of Jeb Bush’s ExcelinEd (which funded the report on retention in MS that Kristof cited).

Here are some of the ugly numbers concerning grade retention and its disproportionate impact on Black students:

The short response to all this must be that grade retention is gaming the system and it is harmful to children, disproportionately harmful to Black, brown, and poor children.

Childhood, Education, reading, Science of Reading

Open Letter to Teachers of Young Children: Dr. Diane Stephens

Open Letter to Teachers of Young Children

Dr. Diane Stephens

June 23, 2023

I have become increasingly concerned about states (and, in the past, the federal government) making decisions/passing laws about the practices associated with teaching reading and writing. I believe that teachers should step up and assert their right to be treated as professionals. To accomplish this, teachers need to

  • make a life-long commitment to broadening and deepening their knowledge base so their curricular decisions are consistently based on current peer-reviewed research which appears in top-tier reading journals, their own experiences, and their knowledge of each child in their classroom 
  • keep track of legislative bills and laws that attempt to curtail their curricular decision-making and 
  • take action (write, call, protest) so that their rights as teachers as not dictated by legislation.

In so doing, we will honor our responsibility to ensure that, in turn, children have rights as readers and writer. We are the only ones who can do this.

We need to stand up and demand that decision-makers at the local, state and national level resist what has been a long-established practice of telling one professional group, teachers, what to do while honoring the right of every other profession to establish their own standards and scientifically based practices.  

Below I have drafted a list of the rights of children as readers and writers. If you have classroom footage to go with #2, #3, #4, #5, and/or #11 and consent from parents to use that footage for educational purposes, please send the videos and copy of the consent forms and I will select one for each of those rights. Also please weigh in on your thoughts about books to name for #10. You can contact me at stephens.diane@gmail.com.

The Rights of Children as Readers and Writers 

in Pre-K, K and 1st grade Classrooms

1.  Children have the right to fall in love with books (if they haven’t already) and know that books make sense, so teachers read books to and with children (this is called an Interactive Read-Aloud).  The teacher chooses books that are easy for the children to understand. This is referred to as their Listening Comprehension.  Via Read Aloud, children also learn that Reading is a Meaning-Making Process. To see an Interactive Read Aloud in Brooke Bridges’ Kindergarten classroom, see Additional Video #2: Interactive Read-Alouds https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos.  The password is learners.

2. Children have the right to understand how books work so teachers read large-sized versions of books which allow every child see the pages clearly and then teachers read the books to them, pointing to words as they read. This is called Interactive Shared Reading. This helps children learn that books in English are read top to bottom and right to left. This is referred to as Book Knowledge. It also helps the children understand that there is a relationship between what the teacher “says” and what is written on the page. This is often referred to as Print-to-Speech Matching.  

3.  Children have the right to understand that oral and written language can be segmented and blended so teachers teach them songs, rhymes, and word games – oral and written. This is referred to as Phonemic Awareness. 

4.  Children have the right to understand how language works e.g., that some sound/symbol relationships are constant. Teachers help young children learn this through alphabet cards with pictures of objects the children have brought in and pictures of each other under the first letter of their names, through songs and rhymes and large group discussion of Morning Message, and via word hunts for words that contain consistent patterns, e.g., /an/, /am/, /at/ and also for words in which two letters make the one sound like /th/, /sh/, /ch/. Children also learn about this by reading and writing. This particularly understanding is referred to as Phonics.  

5. Children have the right to understand that written language is as predictable as the oral language they hear around them, so teachers read and provide access to books that sound like the language they know. This reinforces the idea that Reading is a Meaning-Making Process and it helps children develop Fluency – the ability to read smoothly and meaningfully, in thought units. 

6. Children have the right to understand that writing (and therefore reading) are ways of communicating, so teachers encourage children to use their emergent understanding of sound/symbol relationships to write labels, letters, and books. This allows students to understand that Writing is a Meaning-Making Process. To see kindergarten teacher Brooke Bridges introduce and carry out book-making early in the third month of school, see Additional Video #9 – Creating Books with Children https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos. The password is learners.

7. Children have the right to believe in their ability to make sense of text, so teachers provide books with which they will be successful and, as children’s skills and strategies develop, teachers ensure that those books are matched to children’s evolving strengths. This helps children develop Agency – a belief that they are capable of making sense of print.  Children without a sense of agency often stop trying and claim they do not “like” reading. These students all too often eventually drop out of school. To see kindergartners reading together in Resi Suehiro’s Kindergarten classroom, see Additional Video #1 – Buddy Reading in Kindergarten https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos. The password is learners.

8.  Children have the right to choose books during an independent reading time.  This increases their interest in books and in their Motivation to read.  To see how Nicole Bishop helps her first graders choose books, see Additional Video #5 – Look, Think, Pass https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos. The password is learners.

9.  Children have the right to have ample time to read because volume of reading is directly related to Reading Achievement. To see Independent Reading in Brooke Bridges’ Kindergarten classroom, see Additional Video #18 – Independent Reading in Kindergarten https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos. Password is learners.

10.  Children have the right to have their uniqueness recognized, so their teachers provide whole group support based on the strengths and needs of the whole group, flexible small group support for children with similar strengths and one-on-one support. This means not subjecting children to one-size-fits-all instruction. This insures Authenticity of Instructional Support to each child as opposed to fidelity to a program that may help only a few children.

To get an idea of the diversity of one kindergarten classroom in which there seems to be little ethnic diversity, listen to this intro by Brooke Bridges about the characteristics of her students during academic year 2018-2019:  https://drive.google.com/file/d/1FyZpkm4tAg8glex7khLzPnK5U_VYY8Rs/view?usp=sharing.

To see how Ms. Bridges responses to children vary (a) during independent reading, see Additional Video #18 – Independent Reading in Kindergarten https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos and (b) during independent writing,  see Additional Video #9 – Creating Books with Children https://www.scholastic.com/content/educators/en/pro/readingrevealed.html#additionalvideos. 

Both of these videos show how she supports children based on her knowledge of them. 

The password for the last two videos is learners.

11. Children have the right to learn problem-solving skills and strategies to figure out unfamiliar words, see, for example, Scanlon and Anderson’s (2010) Interactive Strategies List (below).  This fosters Reading Independence. 

    Interactive Strategies:

  • Check the pictures
  • Think about the sounds in the word 
  • Think of words that might make sense 
  • Look for word families or other parts you know 
  • Read past the puzzling word
  • Go back to the beginning of the sentence and start again
  • Try different pronunciations of some of the letters, particularly the vowels. 
  • Break the word into smaller parts 

It would be great if legislators (and some publishers of reading materials for pre-K to 3), already understood that they should be stepping back from mandating or selling curriculum to teachers – that they should instead be encouraging teachers to make their own informed curricular decisions and to choose materials based on their knowledge of the broad field of research on reading and writing and on their knowledge of children in their classrooms. 

But that’s not going to happen spontaneously. It is only going to happen if informed teachers get themselves involved in the decision-making process by writing letters, making phone calls, and scheduling appointments with decision-makers.

I realize that taking political action is not comfortable. If it helps just think of it as having a conversation (through the mail, on the phone, in an office) with someone who does not yet know enough about teaching reading and writing. 

Think of legislators as learners who need our help.

It is our responsibility to ensure that children have the at least the eleven rights outlined in this letter. If a law limiting these rights has already been passed in your state, learn the process for submitting amendments and propose them. If a bill is in process (see, for example South Carolina Senate Bill 518), write, call, visit your legislator and the members of the House and Senate Education Committees. And be sure to be in contact with the legislative aide for both Committees. Those individuals are lawyers who put pen to paper. And, in my experience, they really listen. 

Please, step up for your rights as professionals and for the rights of the children you serve.  If enough of us stand up, there is no limit to how much we can improve our own lives and the lives of children.

Thanks.

Diane Stephens, Ph.D. 

Distinguished Professor Emerita

John E. Swearingen, Sr. Professor Emerita in Education

University of South Carolina                      

Reference

D.M. Scanlon and K.L. Anderson (2010). “Using the Interactive Strategies Approach to Prevent Reading Difficulties in an RTI Context” (p. 49). In M.Y. Lipson and K.K. Wixson (Eds.), Successful Approaches to RTI: Collaborative Practices for Improving K–12 Literacy, Newark, DE: International Reading Association.

Education, NAEP, reading, Science of Reading, Testing

NAEP LTT 2023: A Different Story about Reading

“The Nation’s Report Card” has released NAEP Long-Term Trend Assessment Results: Reading and Mathematics for 2023.

The LTT is different than regularly reported NAEP testing, as explained here:

As I will highlight below, it is important to emphasize LTT is age-based and NAEP is grade-based.

LTT assesses reading for 13-year-old students, and by 2023, these students have experienced school solidly in the “science of reading” (SOR)-legislation era, which can be traced to 2013 (30+ states enacting SOR legislation and growing to almost every state within the last couple years [1]).

Being age-based (and not impacted by grade retention), the trends tell a much different story than the popular and misleading SOR movement.

Consider the following [2]:

Here is the different story:

  • There is no reading crisis.
  • Test-based gains in grades 3 and 4 are likely mirages, grounded in harmful policies and practices such as grade retention.
  • Age 13 students were improving at every percentile when media and politicians began crying “crisis,” but have declined in SOR era, notably the lowest performing students declining the most.
  • Reading for fun and by choice have declined significantly in the SOR era (a serious concern since reading by choice is strongly supported by research as key for literacy growth).

Here are suggested readings reinforced by the LTT data:

The US has been sold a story about reading that is false, but it drives media clicks, sells reading programs and materials, and serves the rhetorical needs of political leaders.

Students, on the other hand, pay the price for false stories.

[1] Documenting SOR/grade-three-intensive reading legislation, connected to FL as early as 2002, but commonly associated with 2013 as rise of SOR-labeled legislation (notably in MS):

Olson, L. (2023, June). The reading revolution: How states are scaling literacy reform. FutureEd. Retrieved June 22, 2023, from https://www.future-ed.org/teaching-children-to-read-one-state-at-a-time/

Cummings, A. (2021). Making early literacy policy work in Kentucky: Three considerations for policymakers on the “Read to Succeed” act. Boulder, CO: National Education PolicyCenter. Retrieved May 18, 2022, from https://nepc.colorado.edu/publication/literacy

Cummings, A., Strunk, K.O., & De Voto, C. (2021). “A lot of states were doing it”: The development of Michigan’s Read by Grade Three law. Journal of Educational Change. Retrieved April 28, 2022, from https://link.springer.com/article/10.1007/s10833-021-09438-y

Collet, V.S., Penaflorida, J., French, S., Allred, J., Greiner, A., & Chen, J. (2021). Red flags, red herrings, and common ground: An expert study in response to state reading policy. Educational Considerations, 47(1). Retrieved July 26, 2022, from https://doi.org/10.4148/0146-9282.2241

Reinking, D., Hruby, G.G., & Risko, V.J. (2023). Legislating phonics: Settle science of political polemic? Teachers College Record. https://doi.org/10.1177/01614681231155688

Schwartz, S. (2022, July 20). Which states have passed “science of reading” laws? What’s in them? Education Week. Retrieved July 25, 2022, from https://www.edweek.org/teaching-learning/which-states-have-passed- science-of-reading-laws-whats-in-them/2022/07

Thomas, P.L. (2022). The Science of Reading movement: The never-ending debate and the need for a different approach to reading instruction. Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/publication/science-of-reading

[2] Despite claims of a “miracle” MS grade 8 NAEP in reading remains at the bottom after a decade of SOR legislation:

Education, education reform, Educational Research, NAEP, reading, SAT, Science of Reading, Statistics, Testing

SAT Lessons Never Learned: NAEP Edition

Yesterday, I spent an hour on the phone with the producer of a national news series.

I realized afterward that much of the conversation reminded me of dozens of similar conversations with journalists throughout my 40-year career as an educator because I had to carefully and repeatedly clarify what standardized tests do and mean.

Annually for more than the first half of my career, I had to watch as the US slipped into Education Crisis mode when SAT scores were released.

Throughout the past five decades, I have been strongly anti-testing and anti-grades, but most of my public and scholarly work challenging testing addressed the many problems with the SAT—and notably how the media, public, and politicians misunderstand and misuse SAT data.

See these for example:

Over many years of critically analyzing SAT data as well as the media/public/political responses to the college entrance exam, many key lessons emerged that include the following:

  • Lesson: Populations being tested impact data drawn from tests. The SAT originally served the needs of elite students, often those seeking Ivey League educations. However, over the twentieth century, increasingly many students began taking the SAT for a variety of reasons (scholarships and athletics, for example). The shift in population of students being tested from an elite subset (the upper end of the normal curve) to a more statistically “normal” population necessarily drove the average down (a statistical fact that has nothing to do with school or student quality). While statistically valid, dropping SAT scores because of population shifts created media problems (see below); therefore, the College Board recentered the scoring of the SAT.
  • Lesson: Ranking by test data must account for population differences among students tested. Reporting in the media of average SAT scores for the nation and by states created a misleading narrative about school quality. Part of that messaging was grounded in the SAT reporting average SAT scores by ranking states, and then, media reporting SAT average scores as a valid assessment of state educational quality. The College Board eventually issued a caution: “Educators, the media and others should…not rank or rate teachers, educational institutions, districts or states solely on the basis of aggregate scores derived from tests that are intended primarily as a measure of individual students.” However, the media continued to rank states using SAT average scores. SAT data has always been strongly correlated with parental income, parental level of education, and characteristics of students such as gender and race. But a significant driver of average SAT scores also included rates of participation among states. See for example a comparison I did among SC, NC, and MS (the latter having a higher poverty rate and higher average SAT because of a much lower participation rate, including mostly elite students):
  • Lesson: Conclusions drawn from test data must acknowledge purpose of test being used (see Gerald Bracey). The SAT has one very narrow purpose—predicting first-year college grades; and the SAT has primarily one use—a data point for college admission based on its sole purpose. However, historically, media/public/political responses to the SAT have used the data to evaluate state educational quality and the longitudinal progress of US students in general. In short, SAT data has been routinely misused because most people misunderstand its purpose.

Recently, the significance of the SAT has declined, students taking the ACT at a higher rate and more colleges going test-optional, but the nation has shifted to panicking over NAEP data instead.

The rise in significance of NAEP includes the focus on “proficiency” included in NCLB mandates (which required all states to have 100% student proficiency by 2014).

The problem now is that media/public/political responses to NAEP mimic the exact mistakes during the hyper-focus on the SAT.

NAEP, like the SAT, then, needs a moment of reckoning also.

Instead of helping public and political messaging about education and education reform, NAEP has perpetuated the very worst stories about educational crisis. That is in part because there is no standard for “proficiency” and because NAEP was designed to provide a check against state assessments that could set cut scores and levels of achievement as they wanted:

Since states have different content standards and use different tests and different methods for setting cut scores, obviously the meaning of proficient varies among the states. Under NCLB, states are free to set their own standards for proficiency, which is one reason why AYP school failure rates vary so widely across the states. It’s a lot harder for students to achieve proficiency in a state that has set that standard at a high level than it is in a state that has set it lower. Indeed, even if students in two schools in two different states have exactly the same achievement, one school could find itself on a failed-AYP list simply because it is located in the state whose standard for proficient is higher than the other state’s….

Under NCLB all states must administer NAEP every other year in reading and mathematics in grades 4 and 8, starting in 2003. The idea is to use NAEP as a “check” on states’ assessment results under NCLB or as a benchmark for judging states’ definitions of proficient. If, for example, a state reports a very high percentage of proficient students on its state math test but its performance on math NAEP reveals a low percentage of proficient students, the inference would be that this state has set a relatively easy standard for math proficiency and is trying to “game” NCLB.

What’s Proficient?: The No Child Left Behind Act and the Many Meanings of Proficiency

In other words, NAEP was designed as a federal oversight of state assessments and not an evaluation tool to standardize “proficient” or to support education reform, instruction, or learning.

As a result, NAEP, as the SAT/ACT has done for years, feeds a constant education crisis cycle that also fuels concurrent cycles of education reform and education legislation that has become increasingly authoritarian (mandating specific practices and programs as well as banning practices and programs).

With the lessons from the SAT above, then, NAEP reform should include the following:

  • Standardizing “proficient” and shifting from grade-level to age-level metrics.
  • Ending state rankings and comparisons based on NAEP average scores.
  • Changing testing population of students by age level instead of grade level (addressing impact of grade retention, which is a form of state’s “gaming the system” that NAEP sought to correct). NAEP testing should include children in an annual band of birth months/years regardless of grade level.
  • Providing better explanations and guidance for reporting and understanding NAEP scores in the context of longitudinal data.
  • Developing a collaborative relationship between federal and state education departments and among state education departments.

While I remain a strong skeptic of the value of standardized testing, and I recognize that we over-test students in the US, I urge NAEP reform and that we have a NAEP reckoning for the sake of students, teachers, and public education.

Recommended

Literacy and NAEP Proficient, Tom Loveless

The NAEP proficiency myth, Tom Loveless