The US is in its fifth decade of high-stakes accountability education reform.

A cycle of education crisis has repeated itself within those decades, exposing a very clear message: We are never satisfied with the quality of our public schools regardless of the standards, tests, or policies in place.

The sixteen years of the George W. Bush and Barack Obama administrations were a peak era of education reform, culminating with a shift from holding students (grade-level testing and exit exams) and schools (school report cards) accountable to holding teachers accountable (value-added methods [VAM] of evaluation).

The Obama years increased education reform based on choice and so-called innovation (charter schools) and doubled-down on Michelle Rhee’s attack on “bad” teachers and Bill Gates’s jumbled reform-of-the-moment approaches (in part driven by stack ranking to eliminate the “bad” teachers and make room for paying great teachers extra to teach higher class sizes). [1]

Like Rhee and Gates, crony appointee Secretary of Education Arne “Game Changer” Duncan built a sort of celebrity status (including playing in the NBA All-Star celebrity games) on the backs of the myth of the bad teacher, charter schools, and arguing that education reform would transform society.

None the less, by the 2010s, the US was right back in the cycle of shouting education crisis, pointing fingers at bad teachers, and calling for science-based reform, specifically the “science of reading” movement.

Reading legislation reform began around 2013 and then the media stoked the reading crisis fire starting in 2018. However, this new education crisis is now paralleled by the recent culture war fought in schools with curriculum gag orders and book bans stretching from K-12 into higher education.

Education crisis, teacher bashing, public school criticism, and school-based culture wars have a very long and tired history, but this version is certainly one of the most intense, likely because of the power of social media.

The SOR movement, however, exposes once again that narratives and myths have far more influence in the US than data and evidence.

Let’s look at a lesson we have failed to learn for nearly a century.

Secretary Duncan was noted (often with more than a dose of satire) for using “game changer” repeatedly in his talks and comments, but Duncan also perpetuated a myth that the teacher is the most important element in a child’s learning.

As a teacher for almost 40 years, I have to confirm that this sounds compelling and I certainly believe that teachers are incredibly important.

Yet decades of research reveal a counter-intuitive fact that is also complicated:

But in the big picture, roughly 60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty). Observable and unobservable schooling factors explain roughly 20 percent, most of this (10-15 percent) being teacher effects. The rest of the variation (about 20 percent) is unexplained (error). In other words, though precise estimates vary, the preponderance of evidence shows that achievement differences between students are overwhelmingly attributable to factors outside of schools and classrooms (see Hanushek et al. 1998; Rockoff 2003; Goldhaber et al. 1999; Rowan et al. 2002; Nye et al. 2004).

Teachers Matter, But So Do Words

Measurable student achievement is by far more a reflection of out-of-school factors (OOS) such as poverty, parental education, etc., than of teacher quality, school quality, or even authentic achievement by students. Historically, for example, SAT data confirm this evidence:

Test-score disparities have grown significantly in the past 25 years.  Together, family income, education, and race now account for over 40% of the variance in SAT/ACT scores among UC applicants, up from 25% in 1994.  (By comparison, family background accounted for less than 10% of the variance in high school grades during this entire time) The growing effect of family background on SAT/ACT scores makes it difficult to rationalize treating scores purely as a measure of individual merit or ability, without regard to differences in socioeconomic circumstance.

Family Background Accounts for 40% of SAT/ACT Scores Among UC Applicants

Let’s come back to this, but I want to frame this body of scientific research (what SOR advocates demand) with the SOR movement claims [2] that teachers do not teach the SOR (because teacher educators failed to teach that) and student reading achievement is directly linked to poor teacher knowledge and instruction (specifically the reliance on reading programs grounded in balanced literacy).

This media and politically driven SOR narrative is often grounded in a misrepresentation of test-based data, NAEP:

First, the SOR claims do not match grade 4 data on NAEP in terms of claiming we have a reading crisis (NAEP scores immediately preceding the 2013 shift in reading legislation were improving), that SOR reading policies and practices are essential (NAEP data have been flat since 2013 with a Covid drop in recent scores), and that 65% of students aren’t proficient at reading.

On that last point, the misinformation and misunderstanding of NAEP are important to emphasize:

1.  Proficient on NAEP does not mean grade level performance.  It’s significantly above that.
2.  Using NAEP’s proficient level as a basis for education policy is a bad idea.

The NAEP proficiency myth

Now if we connect the SOR narrative with NAEP data and the research noted above about what standardized test scores are causally linked to, we are faced with very jumbled and false story.

Teacher prep, instructional practices, and reading programs would all fit into that very small impact of teachers (10-15%), and there simply is no scientific research that shows a causal relationship between balanced literacy and low student reading proficiency. Added to the problem is that balanced literacy and the “simple view” of reading (SVR) have been central to how reading is taught for the exact same era (yet SOR only blames balanced literacy and aggressively embraces SVR as “settled science,” which it isn’t).

One of the worst aspects of the SOR movement has been policy shifts in states that allocate massive amount of public funds to retraining teachers, usually linked to one professional development model, LETRS (which isn’t a scientifically proven model [3]).

Once again, we are mired in a myth of the bad teacher movement that perpetuates the compelling counter myth that the teacher is the most important element in a child’s education.

However, the VAM era flamed out, leaving in its ashes a lesson that we are determined to ignore:

VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.

ASA Statement on Using Value-Added Models for Educational Assessment (2014)

Let me emphasize: “the majority of opportunities for quality improvement are found in the system-level conditions,” and not through blaming and retraining teachers.

The counterintuitive part in all this is that teachers are incredibly important at the practical level, but isolating teaching impact at the single-teacher or single-moment level through standardized testing proves nearly impossible.

The VAM movement failed to transform teacher quality and student achievement because, as the evidence form that era proves, in-school only education reform is failing to address the much larger forces at the systemic level that impact measurable student achievement.

Spurred by the misguided rhetoric and policies under Obama, I began advocating for social context reform as an alternative to accountability reform.

The failure of accountability, the evidence proves, is that in-school only reform never achieves the promises of the reformers or the reforms.

Social context reform calls for proportionally appropriate and equity-based reforms that partner systemic reform (healthcare, well paying work, access to quality and abundant food, housing, etc.) with a new approach to in-school reform that is driven by equity metrics (teacher assignment, elimination of tracking, eliminating punitive policies such as grade retention, fully funded meals for all students, class size reduction, etc.).

The SOR movement is repeating the same narrative and myth-based approach to blaming teachers and schools, demanding more (and earlier) from students, and once again neglecting to learn the lessons right in front of us because the data do not conform to our beliefs.

I have repeated this from Martin Luther King Jr. so often I worry that there is no space for most of the US to listen, but simply put: “We are likely to find that the problems of housing and education, instead of preceding the elimination of poverty, will themselves be affected if poverty is first abolished.”

While it is false or at least hyperbolic messaging to state that 65% of US students are not proficient readers, if we are genuinely concerned about the reading achievement of our students, we must first recognize that reading test scores are by far a greater reflection of societal failures—not school failures, not teacher failures, not teacher education failures.

And while we certainly need some significant reform in all those areas, we will never see the sort of outcomes we claim to want if we continue to ignore the central lesson of the VAM movement; again: “the majority of opportunities for quality improvement are found in the system-level conditions.”

The SOR movement is yet another harmful example of the failures of in-school only education reform that blames teachers and makes unrealistic and hurtful demands of children and students.

The science from the VAM era contradicts, again, the narratives and myths we seem fatally attracted to; if we care about students and reading, we’ll set aside false stories, learn our evidence-based lessons, and do something different.

[1] TAKING TEACHER EVALUATION TO SCALE: THE EFFECT OF STATE REFORMS ON ACHIEVEMENT AND ATTAINMENT

Joshua Bleiberg
Eric Brunner
Erica Harbatkin
Matthew A. Kraft
Matthew G. Springer
Working Paper 30995
http://www.nber.org/papers/w30995

ABSTRACT

Federal incentives and requirements under the Obama administration spurred states to adopt major reforms to their teacher evaluation systems. We examine the effects of these reforms on student achievement and attainment at a national scale by exploiting the staggered timing of implementation across states. We find precisely estimated null effects, on average, that rule out impacts as small as 0.015 standard deviation for achievement and 1 percentage point for high school graduation and college enrollment. We also find little evidence that the effect of teacher evaluation reforms varied by system design rigor, specific design features or student and district characteristics. We highlight five factors that may have undercut the efficacy of teacher evaluation reforms at scale: political opposition, the decentralized structure of U.S. public education, capacity constraints, limited generalizability, and the lack of increased teacher compensation to offset the non-pecuniary costs of lower job satisfaction and security.

[2] I recommend the following research-based analysis of the SOR movement claims:

The Science of Reading and the Media: Is Reporting Biased?, Maren Aukerman

The Science of Reading and the Media: Does the Media Draw on High-Quality Reading Research?, Maren Aukerman

The Science of Reading and the Media: How Do Current Reporting Patterns Cause Damage?, Maren Aukerman

[3] See:

Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

Research Roundup: LETRS (PDF in link above also)

Recommended

Part of the problem in debates about schools and education is the relentless use of “teacher quality” as a proxy for understanding “teaching quality”. This focuses on the person rather than the practice.

This discourse sees teachers blamed for student performance on NAPLAN and PISA tests, rather than taking into account the systems and conditions in which they work.

While teaching quality might be the greatest in school factor affecting student outcomes, it’s hardly the greatest factor overall. As Education Minister Jason Clare said last month:

“I don’t want us to be a country where your chances in life depend on who your parents are or where you live or the colour of your skin.”

We know disadvantage plays a significant role in educational outcomes. University education departments are an easy target for both governments and media.

Blaming them means governments do not have to try and rectify the larger societal and systemic problems at play.

Our study found new teachers perform just as well in the classroom as their more experienced colleagues