Category Archives: Teacher Evaluation

Big Lies of Education: Series

Here I will collect a series dedicated to the Big Lies of Education. The initial list of topics include :

  • A Nation at Risk and education “crisis”
  • Poverty is an excuse in educational achievement
  • 2/3 students not proficient/grade level readers; NAEP
  • Elementary teachers don’t know how to teach reading
  • NRP = settled science
  • Teacher education is not preparing teachers based on science/research
  • Education “miracles”
  • Reading program X has failed
  • Whole language/balanced literacy has failed
  • Systematic phonics necessary for all students learning to read
  • Nonsense word assessments measure reading achievement
  • Reading in US is being taught by guessing and 3 cueing
  • Balanced literacy = guessing and 3 cueing
  • K-3 students can’t comprehend
  • 40% of students are dyslexic/ universal screening for dyslexia needed
  • Grade retention
  • Grit/ growth mindset
  • Parental choice
  • Education is the great equalizer
  • Teacher quality is most important factor in student achievement (VAM)

Series:

Big Lies of Education: A Nation at Risk and Education “Crisis”

Big Lies of Education: Reading Proficiency and NAEP

Big Lies of Education: National Reading Panel (NRP)

Big Lies of Education: Poverty Is an Excuse

Big Lies of Education: International Test Rankings and Economic Competitiveness

Education Lies that Won’t Die: Teacher Value

The list is mind-numbingly long—education lies that won’t die.

I have detailed often the standard manufactured crisis/miracle rhetoric surrounding discussions of education in media, among the public, and by politicians.

And currently, the manufactured reading crisis is grounded in the Big Lie about NAEP and reading proficiency.

Many education lies that won’t die are ideological beliefs masquerading as evidence-based claims; two of the most persistent of those involve assertions about teacher value and merit pay schemes.

Of course, teacher value is incredibly important to student learning; however, this argument is misleading at best, at worst a lie: “Research has shown that the number one factor influencing individual student achievement is the quality of the teacher in the classroom.”

It is an incredibly compelling misrepresentation of the data available, primarily because it appears to support the value of teachers.

Yet, the evidence over many years shows that measurable student achievement is mostly driven by out-of-school factors (OOS) with in-school factors and teacher quality as a subset of that, both significantly overshadowed by those OOS factors:

Teachers Matter, But So Do Words
ASA Statement on Using Value-Added Models for Educational Assessment (2014)
Family Background Accounts for 40% of SAT/ACT Scores Among UC Applicants

The paradox is that teacher value is very difficult to measure through student achievement tests. Further, claims about teacher value that misrepresent that value force teachers and students into nearly impossible conditions to be successful.

Locally, a district is revisiting value-added efforts to attract and retain teachers in high-poverty, majority-minority school, paired with merit pay schemes (that have been tried multiple times in South Carolina unsuccessfully).

Value-added methods of teacher evaluation and merit pay are ideological commitments, and both are strongly refuted by a large body of evidence.

Simultaneously with these failed schemes, education is increasingly hostile to teachers—parent organizations framing teachers as groomers and indoctrinators along with states and districts trending toward curriculum bans and scripted curriculum that de-professionalize those teachers:

“Phonics Monkeys” and “Real Life Reading”: Heteroglossic Views of a State Reading Initiative

If we genuinely value teachers—and having been in this profession 40 years, I suspect that on balance we do not—we would address teaching and learning conditions (class size, teacher autonomy, etc.) within a larger effort to address social inequity in children’s communities and homes.

Ideology is not evidence, and education, teachers, and students deserve much better than political leaders and administrators using our schools as experiments of that ideology—especially when we have ample evidence that ideology is flawed.


Recommended

VAM: Why Is This Zombie Policy Still Around? Peter Greene


UPDATE

Maroun, Jamil, and Christopher H. Tienken. 2024. “The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States” Education Sciences 14, no. 2: 129. https://doi.org/10.3390/educsci14020129

Abstract

The purpose of this study was to determine the predictiveness of community and family demographic variables related to the development of student academic background knowledge on the percentage of students who pass a state-mandated, commercially prepared, standardized Algebra 1 test in the state of New Jersey, USA. This explanatory, cross-sectional study utilized quantitative methods through hierarchical regression analysis. The results suggest that family demographic variables found in the United States Census data related to the development of student academic background knowledge predicted 75 percent of schools in which students achieved a passing score on a state standardized high school assessment of Algebra 1. We can conclude that construct-irrelevant variance, influenced in part by student background knowledge, can be used to predict standardized test results. The results call into question the use of standardized tests as tools for policy makers and educational leaders to accurately judge student learning or school quality.

The Myth of the Bad Teacher: 2023

If you are paying attention to traditional or social media, you are aware of the following stories being told about US public school teachers in 2023:

  • Elementary teachers are failing to teach reading effectively to US students.
  • That failure is “because many deans and faculty in colleges of education either don’t know the science or dismiss it,” according to Hanford.
  • Elementary, literature/ELA teachers, and history teachers are brainwashing students with CRT.
  • Elementary and literature/ELA teachers are grooming children to be gay or transgender by forcing them to read diverse books and stories.

Except for teachers themselves and some education scholars, these new bad teacher myths are both extremely compelling and almost entirely false. Writing in 2010 during a peak bad teacher movement in the US, Adam Bessie explains about the bad teacher stories represented by Michelle Rhee and perpetuated by the Obama administration and Bill Gates:

The myth is now the truth.

The Bad Teacher myth, [Bill] Ayers admits, is appealing, which is why it’s spread so far and become so commonly accepted. Who can, after all, disagree that we “need to get the lazy, incompetent teachers out of the classroom?” Even Ayers agrees that he, like all of us, “nods stupidly” along with this notion. As a professor at a community college and former high school teacher, I nod stupidly as well: I don’t want my students held back, alienated, or abused by these Bad Teachers.

This myth is also seductive in its simplicity. It’s much easier to have a concrete villain to blame for problems school systems face. The fix seems easy, as well: all we need to do is fire the Bad Teachers, as controversial Washington, DC, school chancellor superstar Michelle Rhee has, and hire good ones, and students will learn. In this light, Gates’ effort to “fix” the bug-riddled public school operating system by focusing on teacher development makes perfect sense. The logic feels hard to argue with: who would argue against making teachers better? And if, as a teacher, you do dare to, you must be “anti-student,” a Bad Teacher who is resistant to “reforms,” who is resistant to improvements and, thus, must be out for himself, rather than the students.

The Myth of the Bad Teacher

Bessie concludes, “The only problem with the Bad Teacher myth, as anyone involved with education is intimately aware of, is that problems in education are anything but simple,” and ultimately, in 2023, these myths are not supported by the evidence.

For example, as the authors of a report out of UCLA assert about anti-CRT attacks on teachers:

We put “CRT” in quotation marks throughout this report because so often the conflict campaign’s definition of “CRT” (like its description of actual K–12 practice) is a caricatured distortion by loud opponents as self-appointed “experts.” The conflict campaign thrives on caricature — on often distorting altogether both scholarship and K–12 educators’ efforts at accurate and inclusive education, deeming it (and particularly K–12 efforts to discuss the full scope of racism in our nation) wholly inappropriate for school.

The Conflict Campaign

The bad teacher myth in 2023 “thrives on caricature” and anecdotes that, as noted above, as very compelling but ultimately not only lack credible evidence [1] and logic, but also cause far more harm than good in terms of reforming education, serving student needs, or recruiting and retaining high quality teachers.

The bad teacher myth in 2023 is targeting educators who are 70-90% women, and those teachers under the most intense attacks tend to be elementary teachers who are even more disproportionately women and the lowest paid educators [2]:

Further, there is little evidence that students today are uniquely underperforming in reading achievement, yet the bad reading teacher myth is perpetuated by misrepresenting reading achievement through misleading messages around NAEP reading data (see Hanford’s chart that ironically suggests gradual improvement, not a crisis).

Two problems with the bad reading teacher myth is that NAEP reading proficiency is not grade level reading, as Loveless examines:

NAEP does not report the percentage of students performing at grade level.  NAEP reports the percentage of students reaching a “proficient” level of performance.  Here’s the problem. That’s not grade level. 

In this post, I hope to convince readers of two things:

1.  Proficient on NAEP does not mean grade level performance.  It’s significantly above that.
2.  Using NAEP’s proficient level as a basis for education policy is a bad idea.

The NAEP proficiency myth

And the low levels of reading proficiency are historical, not a recent set of data that constitutes a reading crisis:

If we want to rely on NAEP reading scores, however flawed that metric, the historical patterns suggest a relatively flat state of reading achievement with some trends of improvement in the 1970s (which was followed by the manufactured myth of schools failing with A Nation at Risk [1983]) and steadily from about 1990 until 2012 (an era demonized as a failure due to reliance on balanced literacy).

Notably, the “science of reading” movement tends to be connected to legislation starting around 2013 and Hanford’s journalism beginning in 2018, and that NAEP data has remained relatively flat except for the Covid drop.

Again, as Bessie acknowledged over a decade ago, the real problems with education, teaching, and learning are very complex and far larger than pointing fingers at teachers as “villains.”

For most of the history of US education, student reading achievement has been described as “failing,” and vulnerable student populations (minoritized races, impoverished students, students with special needs such as dyslexia, and MLLs) have always been underserved.

The dirty little secret about teacher quality related to student reading proficiency is that those vulnerable students are disproportionately sitting in class with early-career and uncertified teachers who are struggling with high student/teacher ratios.

Are students being underserved? Yes, but this is a historical fact of US public education not a current crisis.

Are low student achievement and reading proficiency the result of bad teachers? No, but these outcomes are definitely correlated with bad teaching/learning conditions and bad living conditions for far too many students.

In 2023, just as in 2010, the myth of the bad teacher is a lie, a political and marketing lie that will never serve the needs of students, teachers, or society.

Way back in 1984 when I entered the classroom, I was excited to begin my career but quickly discovered that despite my respect and even love for my English professors and teacher educators in my undergraduate degree, I simply was not prepared well enough to do my job, notably as a teacher of writing.

I set out to learn by teaching, and do better. During the late 1980s, I was fortunate to learn further through the Spartanburg Writing Project (Nation Writing Project), where I discovered that much of my on-the-job training was misguided (thanks, Brenda Davenport).

Anyone who teaches knows that becoming an effective teacher is a journey and that those first 3, 5, or even 10 years are challenging and include a great deal of growth that cannot be accomplished in teacher certification programs.

None the less, everything surrounding teaching, and especially the teaching of reading, can and should be better.

That was true in 1940 and every decade since then.

Teacher and school bashing, shouting “crisis”—these have been our responses over and over; these are not how we create a powerful teacher workforce, and these will never serve the needs of our students who need great teachers and public education the most.

The myth of the bad teacher is a Great American Tradition that need to end.


[1] Valcarcel, C., Holmes, J., Berliner, D. C., & Koerner, M. (2021). The value of student feedback in open forums: A natural analysis of descriptions of poorly rated teachers. Education Policy Analysis Archives29(January – July), 79. https://doi.org/10.14507/epaa.29.6289

[2] See Our study found new teachers perform just as well in the classroom as their more experienced colleagues

Commentary: There’s no merit in merit pay plans for rewarding, retaining SC teachers (The Post and Courier)

Commentary: There’s no merit in merit pay plans for rewarding, retaining SC teachers

While the Editorial Staff of the Post and Courier rightfully raises cautions about newly elected Superintendent of Education Ellen Weaver, who has a history of extremely conservative positions on public education, that caution should also extend to Weaver’s plans for merit pay to address teacher needs in South Carolina.

I am in my fifth decade as an educator in SC, beginning as a high school English teacher in Upstate SC in 1984. Over that career I have witnessed one frustrating pattern: A constant state of education reform that recycles the exact same crisis rhetoric followed by the same education reforms.

Over and over again.

In fact, in very recent history, SC and the nation have experienced a high intensity focus on teacher quality, teacher evaluation, and teacher merit pay models under the Obama administration.

And just like in the so-called real world of business, merit pay models for teachers have failed.

Under Obama and then-Secretary of Education Arne Duncan, federal and state policy helped implement teacher evaluation and pay schemes modeled on Bill Gates’s use of stack ranking. Notably this value-added model of teacher evaluation and pay was famously heralded by the media when Michelle Rhee was chancellor of DC schools.

However, over time, Rhee’s tenure was unmasked as mostly a fraud but also as extremely harmful for teachers and students.

While merit pay remains a popular approach to recruiting and maintaining workers across many fields, research has consistently shown that merit pay does not work, and often has negative consequences, especially in education.

Research by Dan Pink and Alfie Kohn, for example, highlight the disconnect between what merit pay promises and how that plays out in the real world.

Gary Clabaugh challenged Obama’s merit pay polices, offering evidence that remains valid today and suggests not only caution about Weaver’s merit pay plan but also solid reasons not to try the same thing again.

Merit pay fails education, teachers, and students in the following ways:

  • Merit pay assumes workers need motivation to work harder; teachers are often overworked and their ability to be effective is not a result of how hard they are working, but the conditions under which they work.
  • Merit pay is often linked to standardized testing in education. As a result, merit pay incentivizes teaching to the tests and corrupts evaluation systems intended to measure learning.
  • Merit pay creates a culture of competition, instead of cooperation. Research also shows that competition is more harmful than cooperation, especially in the field of education where all educators should be invested in the success of all students.
  • Measurable student achievement, mostly through standardized testing, is more heavily linked to out-of-school factors (60-80%) than to in-school factors or teacher quality (10-15%). Therefore, merit pay overemphasizes direct and measurable teacher impact and often holds teachers and students accountable for factors beyond their control.

Policy makers in SC are faced with two facts: (1) Teacher pay is important to address and long overdue in the state; however, (2) merit pay is an ineffective and even harmful approach to addressing pay and teacher shortages.

Since SC has tried to use pay incentive to address teacher shortages in struggling districts already, we will better serve the needs of our students if we commit to new and different reforms.

The greatest need in SC is that elected officials directly address poverty across the state—access to healthcare, stable jobs with strong pay, and access to affordable housing.

But we can also do better with in-school reform.

If we want to bolster the teaching profession among our high-poverty districts, we must address teaching and learning conditions, which include the following:

  • Parent, community, and administrative support for teachers.
  • School facilities in good repair.
  • Fully funding teaching and learning technologies and materials.
  • Guaranteeing students who are struggling have access to experienced and certified teachers.
  • Recognizing that student success is linked to teacher quality, but that teacher quality is only one element in a complex network of forces that help children learn.

SC remains faced with a very old problem—high-poverty students struggle to achieve well enough or fast enough compared to their more affluent peers.

Those children deserve new solutions, and merit pay is a tired gimmick that has never worked and will fail children and teachers once again.


See Also

Edujournalism and Eduresearch Too Often Lack Merit

Does Instruction Matter?

For me, the pandemic era (and semi-post-pandemic era) of teaching has included some of the longest periods in my 39-year career as an educator when I have not been teaching.

The first half of my career as a high school English teacher for 18 years included also teaching adjunct at local colleges during the academic year along with always teaching summer courses (even while in my doctoral program).

Currently in my twenty-first year as a college professor, in addition to my required teaching load, I have always taught overloads during the main academic year, our optional MayX session, and (again) summer courses.

Teaching has been a major part of who I am as a professional and person since my first day at Woodruff High (South Carolina) in August of 1984.

However, during pandemic teaching, I have experienced several different disruptions to that teaching routine—shifting to remote, courses being canceled or not making (especially in MayX and summer), and then coincidentally, my first ever sabbatical during this fall of 2022 (in year 21 at my university).

One aspect of sabbatical often includes the opportunity to reset yourself as a scholar and of course as a teacher. As I was preparing my Moodle courses for Spring 2023, I certainly felt an unusually heightened awareness around rethinking my courses—an introductory education course, a first-year writing seminar, and our department upper-level writing and research course.

Here is an important caveat: I always rethink my courses both during the course and before starting new courses. Yes, the extended time and space afforded by sabbatical makes that reflection deeper, I think, but rethinking what and how I teach is simply an integral part of what it means for me to be a teacher.

For two decades now, I have simultaneously been both a teacher and teacher educator; in that latter role, I have been dedicated to practicing what I preach to teacher candidates.

I am adamant that teacher practice must always reflect the philosophies and theories that the teacher espouses, but I am often dismayed that instructional practices in education courses contradict the lessons being taught on best practice in instruction.

Not the first day, but a moment from my teaching career at WHS.

In both my K-12 and higher education positions, for example, I have practiced de-grading and de-testing the classroom because I teach pre-service teachers about the inherent counter-educational problems with traditional grades and tests.

Now, here is the paradox: As both a teacher and teacher educator my answer to “Does instruction matter?” is complicated because I genuinely believe (1) teacher instructional practices are not reflected in measures of student achievement as strongly (or singularly) as people believe and therefore, (2) yes and no.

The two dominant education reform movements over the past five decades I have experienced are the accountability movement (standards and high-stakes testing) and the current “science of reading” movement.

The essential fatal flaw of both movements has been a hyper-focus on in-school education reform only, primarily addressing what is being taught (curriculum and standards) and how (instruction).

I was nudged once again to the question about instruction because of this Tweet:

I am deeply skeptical of “The research is clear: PBL works” because it is a clear example of hyper-focusing on instructional practices and, more importantly, it is easily misinterpreted by lay people (media, parents, and politicians) to mean that PBL is universally effective (which is not true of any instructional practice).

Project-based learning (PBL) is a perfect example of the problem with hyper-focusing on instruction; see for example Lou LaBrant confronting that in 1931:

The cause for my wrath is not new or single. It is of slow growth and has many characteristics. It is known to many as a variation of the project method; to me, as the soap performance. With the project, neatly defined by theorizing educators as “a purposeful activity carried to a successful conclusion,” I know better than to be at war. With what passes for purposeful activity and is unfortunately carried to a conclusion because it will kill time, I have much to complain. To be, for a moment, coherent: I am disturbed by the practice, much more common than our publications would indicate, of using the carving of little toy boats and castles, the dressing of quaint dolls, the pasting of advertising pictures, and the manipulation of clay and soap as the teaching of English literature. (p. 245)

LaBrant, L. (1931, March). Masquerading. The English Journal, 20(3), 244-246. http://www.jstor.org/stable/803664

LaBrant and I both are deeply influenced by John Dewey’s progressive philosophy of teaching (noted as the source for PBL), but we are also both concerned with how the complexities of progressivism are often reduced to simplistic templates and framed as silver-bullet solutions to enormous and complex problems.

As LaBrant notes, the problem with PBL is not the concept of teaching through projects (which I do endorse as one major instructional approach), but failing to align the project in authentic ways with instructional goals. You see, reading a text or writing an essay is itself a project that can be authentic and then can be very effective for instruction.

My classrooms are driven, for example, by two instructional approaches—class discussions and workshop formats.

However, I practice dozens of instructional approaches, many planned but also many spontaneously implemented when the class session warrants (see Dewey’s often ignored concept of “warranted assertion”).

This is why Deweyan progressivism is considered “scientific”—not because we must use settled science to mandate scripted instructional practices but because teaching is an ongoing experiment in terms of monitoring the evidence (student artifacts of learning) and implementing instruction that is warranted to address that situation and those students.

So this leads to a very odd conclusion about whether or not instruction matters.

There are unlikely any instructional practices that are universally “good” or universally “bad” (note that I as a critical educator have explained the value of direct instruction even as I ground my teaching in workshop formats).

The accountability era wandered through several different cycles of blame and proposed solutions, eventually putting all its marbles in teacher quality and practice (the value-added methods era under Obama). This eventually crashed and burned because as I have noted here, measurable impact of teaching practice in student achievement data is very small—only about 10-15% with out-of-school factors contributing about 60-80+%.

The “science of reading” movement is making the exact same mistake—damning “balanced literacy” (BL) as an instructional failure by misrepresenting BL and demonizing “three cueing” (see the second consequence HERE, bias error 3 HERE, and error 2 HERE).

Here is a point of logic and history to understand why blaming poor reading achievement on BL and three cueing: Over the past 80 years, reading achievement has never been sufficient despite dozens of different dominant instructional practices (and we must acknowledge also that at no period in history or today is instructional practice monolithic or that teachers in their classrooms are practicing what is officially designated as their practice).

In short, no instructional practice is the cause of low student achievement and no instructional practice is a silver-bullet solution.

Therefore, does instruction matter? No, if that means hyper-focusing on singular instructional templates for blame or solutions.

But of course, yes, if we mean what Dewey and LaBrant argued—which is an ongoing and complicated matrix of practices that have cumulative impact over long periods of time and in chaotic and unpredictable ways.

From PBL to three cueing—no instructional practice is inherently right or wrong; the key is whether or not teachers base instructional practices on demonstrated student need and whether or not teachers have the background, resources, teaching and learning conditions, and autonomy to make the right instructional decisions.

Finally, hyper-focusing on instruction also contributes to the corrosive impact of marketing in education, an unproductive cycle of fadism and boondoggles.

In the end, we are trapped in a reform paradigm that is never going to work because hyper-focusing on instruction while ignoring larger and more impactful elements in the teaching/learning dynamic (out-of-school factors, teaching and learning conditions, etc.) creates a situation in which all instruction will appear to be failing.

Reforming, banning, and mandating instruction, then, is fool’s gold unless we first address societal/community and school inequities.

Lehre Ist Tot

This past week an early career teacher, highly regarded in the classroom and very accomplished in the field of education, received a parental request that a student not be required to read The Great Gatsby. That parent, however, had signed a consent agreement with all texts, including that novel, identified as required reading at the beginning of the course.

The parent then reached out to the administration, who confirmed that the teacher had to assign a different work. This, of course, undermines the teacher and the process established, but it also creates more work for teachers already under incredible strain.

While parental oversight of assigned reading has been common in education for decades, this situation comes as states are increasingly passing parental trigger legislation, which moves the parental power from each parent’s own children to parents being able to ban works for all teachers or students to explore in classes.

That same teacher, frustrated and disillusioned, later that day read aloud their resignation letter to me in the context of telling me that much that they had taught in the first three years of teaching could no longer be taught in the last couple years—and increasingly will be directly banned in the coming year (as my home state is poised to pass its own educational gag order this spring).

The teacher cried while reading the letter aloud, and added that the resignation was depressing; this, you see, was a career they had been working toward since high school—and within 6 years, teaching is dead.

The current anti-teacher climate in the U.S. is incredibly harsh and driven by orchestrated false narratives:

Right-wing media are creating parental trigger structures even without the concurrent legislation:

While teacher and school bashing (notably as “liberal indoctrination”) has a long history in the U.S., reaching back to Catholic schools fighting for market space as public schooling increased in the 19th century, the current anti-teacher climate has its roots not in Republican politics but in the Obama administration’s education agenda.

Obama’s appointment of Arne Duncan as Secretary of Education heralded an era of education reform that actually doubled-down on the worst aspects under George W. Bush, and that doubling down feed into a growing media attack on “bad teachers.”

Instead of rejecting the standard approaches to education reform begun under Reagan and federalized under W. Bush, the Obama administration turned their blame to teachers and teacher quality. During the Obama years, the great experiment in value-added methods (VAM) devastated the teaching profession.

The perennial paradox of education has always been that teacher quality matters but it remains a very small part of measurable student achievement (only about 10-15%). Therefore, the Holy Grail of the VAM experiment—identifying “good” and “bad” teachers through standardized test scores of students—was always doomed to fail.

But it did accomplish planting the seeds of today’s multi-pronged attack on teachers—the “science of reading” movement blaming teachers and teacher educators for student reading achievement and the anti-CRT/educational gag order movements being linked to parent trigger laws.

Throughout the education reform era over the past 40 years, many of us in education have argued that education reform initiatives are less about improving education and more about killing public education and the teaching profession—charter schools and voucher schemes, Teach For America, VAM and merit pay, demonizing and dismantling unions and tenure, etc., to name a few.

From Fox News lies to parental trigger laws and education gag orders, the evidence is very clear now that this current wave of teacher bashing is definitely about killing the profession, and not about student discomfort.

Let me return to the opening teacher story.

When the parent was asked for reasons why they wanted their child not to read The Great Gatsby—so the teacher had context for choosing an alternate text—the parent responded that they did not want the child (a high school student) to read about inappropriate relationships and sexual content. So here is a point of fact about the insincerity of these challenges; that student had already read and studied The Crucible, without any complaint, a play grounded in adultery.

I am certain some parents challenging what their children are being taught are sincere, but I am also certain the larger political motivation among conservatives is to completely dismantle public education.

Just as I have explained that there simply is no CRT propaganda agenda in K-12 schools, there is no liberal indoctrination/grooming occurring in K-12 (or K-16) education either.

The Ingraham rants are simply political lies.

And these lies are not improving education.

They have one goal and it seems to have been effective: Teaching is dead.

Student Evaluations of Teaching: Research and Commentary [Updated]

One of the prevalent contradictions in higher education is the high-stakes use of student evaluations of teaching (SET) despite the overwhelming evidence that SETs are flawed measures of teacher/teaching quality and are often harmful for faculty already marginalized in society and academia.

Here is a collection of research and commentary highlighting those flaws and calling for ending this traditional practice:

[Updates]

Educational Accountability and the Science of Scapegoating the Powerless

Several years ago when I submitted an Op-Ed to the largest newspaper in my home state of South Carolina, the editor rejected the historical timeline I was using for state standards and testing, specifically arguing that accountability had begun in the late 1990s and not in the early 1980s as I noted.

Here’s the interesting part.

I began teaching in South Carolina in the fall of 1984, the first year of major education reform under then-governor Richard Riley. That reform included a significant teacher pay raise, extended days of working for teachers, and the standards-testing regime that would become normal for all public education across the U.S.

In fact, SC’s accountability legislation dates back to the late 1970s (I sent her links to all this).

As a beginning teacher, the only public schooling I ever knew was teaching to standards and high-stakes tests by identifying standards on my lesson plans and implementing benchmark assessments throughout the academic year to document I was teaching what was mandated as a bulwark against low student tests scores. State testing, including punitive exit exams, pervaded everything about being an English teacher.

Yet, an editor, herself a career journalist, was quick to assume my expertise as a classroom practitioner and then college professor of education was mistaken.

This is a snapshot of how mainstream media interact with education as a topic and educators as professionals.

I am reminded of that experience over and over in fact as I read media coverage of education. Take for example this from Education Week, Want Teachers to Motivate Their Students? Teach Them How, which has the thesis:

Most teachers intrinsically understand the need to motivate their students, experts say, but teaching on intuition alone can lead to missteps in student engagement.

A study released in May by the Mindset Scholars Network, a collaborative of researchers who study student motivation, found most teacher education programs nationwide do not include explicit training for teachers on the science of how to motivate students.

Want Teachers to Motivate Their Students? Teach Them How

Two key elements of this article stand out: The new scapegoat in proclaiming education a failure is teacher education and the go-to failure is always about a lack of “science” in teacher education.

This article on motivation is following a media template well worn recently about students in the U.S. can’t read because teachers are not taught the “science of reading,” you guessed it, in their teacher education programs.

As I detailed in a Twitter thread, scapegoating teacher education has many flaws, and my experience and expertise as a teacher educator for almost two decades, following almost two decades as a classroom teacher, inform my understanding of how finding scapegoats for educational failure during the accountability era is fool’s gold.

How has the accountability era gone in terms of where the accountability and locus of power lie, then?

In the 1980s and 1990s, the accountability mechanisms focused on holding students accountable (think exit exams) and schools accountable (student test scores often translated into school rankings or grades, designating schools as “failing,” for example).

Keep in mind that students had no power in that process, and that schools were merely agents of the standards being implemented, again outside the power dynamics of those mandates being determined.

With No Child Left Behind spawned by the false claims of the Texas Miracle, the accountability era was greatly accelerated, including a creeping sense that the process wasn’t improving education but it was punishing students (lower graduation rates due to exit exams) and demonizing schools (most high-poverty and high-racial minority schools were labeled as “failing”).

By the administration of Barak Obama, with education policy under another false narrative (the Chicago Miracle) and false ambassador with no background in education other than appointments (Arne Duncan), the scapegoating took a turn—the problem, went the new message, was “bad” teachers and the solution was not holding students or schools accountable for test scores but those teachers (the era of value-added methods [VAM]).

As some have noted and documented, teacher bashing increased and then prompted a backlash (see magazine covers from Time for a great series of artifacts on this); it seems that VAM proved to be a false metric for accountability and that maybe teachers were not the problem after all.

With the scapegoat role now vacant, the media have discovered a new candidate, teacher education.

Let’s here recognize that once again the power context is way off in who is determining the accountability and who is being held accountable. For the most part, teachers and teacher educators are relatively powerless agents who are mandated to implement standards and assessments that they do not create and often do not endorse as valid.

Now consider another really important reason accountability in education is deeply flawed: The constant misguided scapegoating of powerless agents in formal teaching and learning is a distraction from the actual causal sources for educational challenges.

Fun fact: Decades of research from educators and education scholars have detailed that out-of-school factors overwhelmingly determine measurable student outcomes, some estimates as high as 80+% and most scholars agreeing on 60%. Teacher quality’s impact on measurable student achievement has been identified repeatedly as only about 10-15%.

Yet, the entire accountability era since the early 1980s has focused on in-school reforms only (scapegoating along the way), while tossing up hands and embracing harsh ideologies such as “no excuses” practices that argue teachers fail students with the “soft bigotry of low expectations” and students fail because they lack “grit” or a growth mindset.

Many of us have doggedly argued for social context reform, addressing socio-economic reform first and then reforming education along equity (not accountability) lines next, or concurrently. Many of us have also demonstrated that “grit” and growth mindset have racist and classist groundings that are harmful.

For those positions, we have been demonized and marginalized for decades.

So imagine my surprise when, first, the tide shifted on teacher bashing (I have 34 posts on my blog discrediting VAM and dozens on misunderstanding teacher quality) and then these articles: Better Schools Won’t Fix America (The Atlantic), The Harsh Discipline of No-Excuses Charter Schools: Is It Worth the Promise? (Education Week), and Unchartered territory: 2020 Democrats back away from charter schools (MSN).

My blog posts, however, on social context reform and poverty (157), “no excuses” reform (70), and the mirage of charter schools (80) have either mostly been ignored or are harshly (even angrily) rejected. Like my interaction with the editor discussed in the opening, my experience and expertise as an educator and education scholar have held almost no weight with those in power pr the media.

The media and journalists as generalists seem deeply resistant to learning a lesson they create over and over.

Take for a current example Karin Wulf’s examination of Naomi Wolff and Cokie Roberts; Wulf herself is a historian:

It’s been a tough few weeks for amateur history. First, journalist Naomi Wolf discovered on live radio that she had misinterpreted key historical terms in her new book, “Outrage,” leading her to draw the wrong conclusions. A week later, journalist Cokie Roberts, too, got a quick smackdown when she claimed on NPR that she couldn’t find any incidence of abortion advertised in 19th century newspapers, a claim quickly disproved by historians.

Wolf and Roberts fell victim to a myth widely shared with the American public: that anyone can do history. Whether it’s diving into genealogy or digging thorough the vast troves of digital archives now online, the public has an easy way into the world of the past. And why would they imagine it takes any special training? After all, the best-selling history books are almost always written by non-historians, from conservative commentators like Bill O’Reilly to journalists like Wolf and Roberts.

What Naomi Wolf and Cokie Roberts teach us about the need for historians

Wulf’s confronting “that anyone can do history” immediately prompted in me my experience when I first moved from teaching high school English (and adjuncting at several colleges, including being a lead instructor in a university-based summer institute of the National Writing Project) to higher education. My university was debating a curriculum change that included dropping traditional composition courses (popularly known as English 101 and English 102) for first-year seminars.

One of those first-year seminars was to be writing-intensive, and the argument being posed was that any professor could teach writing.

This change passed, and the English department and professors were relieved of sole responsibility for teaching writing.

Over the next eight years or so, the university learned a really disturbing lesson (one I could have shared in the beginning): “Any professor can teach writing” is false.

As Wulf argues about history, with writing and education, experience and expertise matter.

So here I sit again, writing over and over that the media are getting reading wrong, that scapegoating teacher education is missing the real problem.

How many years will it take until I see articles “discovering” these facts as if no one with experience and expertise ever raised the issue?

“Despite the Data”: Higher Education Fails Equity, Inclusion with SETs

Entering higher education in my early 40s after 18 successful years as a high school English teacher, I remain 17 years later baffled and even disappointed at the mess of contradictions that characterizes an institution populated by the most educated people possible.

Immediately I had to hold my tongue against the pervasive culture of college professors bemoaning constantly how busy they are. When my high school teaching career ended, I was wearing a wrist brace because I was hand marking about 4000 essays and 6000 journals per year while teaching five course and about 100 students (many colleagues taught 20+ more students per year).

I also coached many of those years, with work days from about 7:30 AM until 10 or 11 PM in the evening.

By contrast, I teach two first-year writing courses each fall (as part of my full load, a minimum of five course per academic year), a total of 24 students, and my teaching schedule tends to be three days a week, often a Monday evening class included.

The Ivory Tower effect is a bit more accurate than I would prefer.

More disturbing, however, is the power of tradition among academics, a dynamic that works against practices and policies being based on evidence (and thus in a state of flux when that evidence changes).

While the U.S. has a long history of characterizing and even demonizing higher education as some sort of liberal cult, the truth is that the very worst qualities of higher education are from its conservative urges as institutions.

Of course, you can find a disproportionate number of professors who have left-leaning social and philosophical ideologies, but the most powerful department/colleges in higher education are often the most conservative—political science, economics—or the most apt to take non-political poses—the hard sciences.

This disconnect between how higher education is perceived and how higher education exists stems from, in part, I think, higher education presenting itself rhetorically as progressive—mission statements, social justice initiatives, etc.

However, with a little unpacking, we can expose that practices and policies often contradict and even work against that rhetoric and those initiatives.

One example that I have addressed again and again is the use of student evaluations of teaching (SET) to drive significantly the promotion, tenure, and reward process.

Consider a few points raised in Colleges Are Getting Smarter About Student Evaluations. Here’s How by Kristen Doerer:

“Having a female instructor is correlated with higher student achievement,” Wu said, but female instructors received systematically lower course evaluations. In looking at prerequisite courses, the two researchers found a negative correlation between students’ evaluations and learning. “If you took the prerequisite class from a professor with high student teaching evaluations,” Harbaugh said, “you were likely, everything else equal, to do worse in the second class.”

The team found numerous studies with similar findings. “It replicates what many, many other people found,” said Harbaugh. “But to see it at my own university, I sort of felt like I had to do something about it.”…

Studies since the 1980s have found gender bias in student evaluations and, since the early 2000s, have found racial bias as well. A 2016 study of data from the United States and France found that students’ teaching evaluations “measure students’ gender biases better than they measure the instructor’s teaching effectiveness,” and that more-effective instructors got lower ratings than others did….

Despite the data, at many colleges, particularly research-based institutions, student evaluations are still the main measure, if not the only one, of teaching effectiveness in promotion-and-tenure decisions.

Common among universities and colleges across the U.S., diversity and inclusion are pervasive problems. Poor students and students of color are underrepresented in many colleges, especially the so-called elite institutions; women and people of color are equally underrepresented on faculties.

Nothing rings more true or frustrating than Doerer’s use of “despite the data.”

I have rejected SETs directly in my bi-annual self-evaluation for merit raises. I have consistently advocated the administration and our faculty status committee to end or greatly reduce the influence of SETs.

In all of the situations, I have repeatedly shared the research, the data:

And without fail, those with power, who tend to be white men, offer a tepid acknowledgement of the research followed by a quick “But we have to do something.” Doerer includes a response (from a white man) that sounds all too familiar:

Ken Ryalls, president of the IDEA Center, a nonprofit higher-education consulting organization, recognizes the bias but thinks doing away with evaluations isn’t the answer. He opposes efforts to eliminate the voice of students. “It seems ludicrous,” he said, “to have the hubris to think that students sitting in the classroom have nothing to tell us.”

“The argument that you should get rid of student evaluations because there is bias inherently is a bit silly,” he said. “Because basically every human endeavor has bias.”

The “yes, but” dynamic works to maintain the inequitable status quo. And as Ryalls’s comment shows, the “yes, but” response is often a distraction.

No one is arguing to remove the voice of students, but as Doerer’s reporting confronts and as the research base shows, student evaluations of teaching are fraught with student biases that corrupt the teacher evaluation process, effectively discouraging women, people of color, and international faculty from remaining in a hostile environment with very real negative career consequences.

For example, calls to end SETs a primary or major instruments for promotion, tenure, and merit pay are often part of a larger examination of how to make student feedback more effective for teaching and learning.

Doerer notes:

That’s in large part why Oregon decided to try a midterm student-experience survey that only the applicable faculty member can view. An instructor can make changes in the middle of a semester, when students can still benefit, encouraging them to give constructive feedback.

For many years, I have asked students for feedback at midterm, and explained that I would like the opportunity to address their concerns, and also to identify what is working well, because receiving complaints after a course really benefits no one.

Further, when student feedback is for the professor only, it becomes a conversation about improving teaching and learning, and as a professor myself, I am best equipped to interpret student comments. I consistently receive feedback intended as negative by students, but will never change them because they misunderstand my role and their roles in the classroom.

Yes, student feedback is valuable, but it likely cannot be simply or easily reduced to numbers, formulas, or even verbatim interpretations of their direct words.

It has taken nearly four decades of high-stakes accountability in K-12 education for people to begin to acknowledge that high-stakes accountability causes far more harm than good.

In higher education, if equity and inclusion are real goals, we can and must seek ways that students have safe and open spaces for providing their professors feedback, we can and must better support faculty in how to interpret that feedback in ways that improve their teaching and student learning, but to reach those goals, we must end the practice of using SETs in significant ways to evaluate faculty.

Higher education must end the tradition of “despite the data,” recognize that rhetoric means less than nothing if contradicted by practices, policies, and a culture of “yes, but.”

Teacher v. Professor: On Why Anyone Would Be an Educator

While cycling with a new acquaintance, I navigated through the usual questions about how long I have been a professor, and then, after I mentioned that I was a high school English teacher for 18 years before moving to higher education, the follow up about which was easier, or which I preferred.

On this ride and during the conversation, I realized I am quickly approaching the tipping point in my career since I am starting my 17th year as a professor, just one year away from having been a professor for as long as I was a high school teacher—the identity I remain strongly associated with about myself professionally.

Being a writer has held both aspects of my being an educator together, but K-12 teaching and being a professor in higher education (especially my role as a teacher educator) are far more distinct than alike.

However, and most disturbing, K-12 teachers and college professors are increasingly sharing disillusionment with being educators as K-12 teachers are fleeing public education and fewer are majoring/certifying to teach while quit lit has become a phenomenon throughout higher education.

My journey as an educator offers some unique insight since I have nearly two decades in each contexts, K-12 and higher education. As well, my work in higher education remains directly connected to K-12 classroom teaching (I am a teacher educator and spend time observing in public schools as well as having professional and personal relationships with public school teachers).

For a majority of my time as a high school teacher, I was department chair, and several of those years included being a coach. My days were often very long, starting at 7:30 AM and ending on match days as late as 11 PM.

As a high school English teacher, I taught around 100 students or more at a time and had five classes a day, usually 2-3 preps. Since I focused on teaching my students to write, I responded to about 4000 essays and 6000 journals a year.

In 1995, I entered a doctoral program, an EdD in curriculum and instruction; I continued to work full-time and even maintained my adjunct work at local colleges. The doctorate experience was sobering since every other candidate I encountered was seeking a doctorate to leave the K-12 classroom—except me.

I completed my degree and was still resolute I would teach high school until I retired, and then maybe seek something at the university level. My pay bump for the advanced degree was, in fact, quite good.

The summer of 2002 was not something I planned, but when a position opened at a nearby university (a position held by my former high school teacher and mentor), I applied with no real expectations about making the transition so soon.

After a flurry of on-campus interviewing, and then a disheartening negotiation about salary (the opening offer was a $17,500 pay cut), I agreed to leave my high school job for higher education. The final pay was still $6000 below my public school salary, but the university promised I would have overloads and summer work to make up the difference.

So I sit here this summer about to start my 17th year as a college professor, a full professor with tenure. I have learned a great deal.

First, the prestige and respect shift from K-12 teaching to higher education was stunning, especially since I taught high school for 4 years while I had a doctorate; the degree was not the key factor in how people viewed and treated me.

Professors receive immediate respect and assumptions about our expertise that K-12 teachers never experience. My ability to publish, for example, in local, state, and national newspaper magically appeared once I could list my university instead if my high school when querying.

Next, and related, that respect divide cannot be disassociated from the impact of gender: More than 3 of 4 K-12 teachers are women, but the largest group of professors is men, and that imbalance is even greater at the higher ranks, where men are the majority. The university where I teach, for example, is well over 60% male faculty.

Possibly the greatest differences, however, between K-12 teaching and being a professor are expectations for labor and what counts as your professional obligations.

By the time I left K-12 teaching, I was wearing a wrist brace; my right hand was nearly immobile from marking essays, and to be honest, teaching English as I knew I should [1] was nearly unmanageable against the rest of my responsibilities and having a family or any sort of recreational life.

Burn out is a common term associated with K-12 teaching, but since teaching doesn’t appear to be manual labor (such as construction) or isn’t associated with production (most of us balk at seeing our students as widgets), those who have not taught fail to recognize the physical and psychological wear that comes with teaching.

I joke, though it isn’t funny, that being stared at by 100 or so students per day is stunningly exhausting. But most K-12 teachers have no real time to eat alone (or with only other adults), to go to the restroom, or to do with their work day anything other than grade, respond to student work, plan, or address the never-ending minutia of bureaucracy that is teaching (standards, meetings, paperwork, etc.).

Teaching—even just lasting past the first 3-5 years—two or three decades is a herculean task in surviving a career; too many teachers out of self-preservation learn to work in auto-mode, mailing in a profession because it has simply erased your humanity.

Along with professional respect, I gained a great deal of professional autonomy (which K-12 teachers have almost none) and, most of all, time. A heavy semester for me is teaching several courses three days a week, usually M, W, F and from about early morning to mid-afternoon.

Except for meetings (and higher education has an ugly committee and departmental meetings problem), I have multiple days a week to devote to my professional commitments other than teaching, for me, being a writer.

And as a professor, I have never fretted about going to the bathroom, and making sure I eat, calmly, is nearly never a struggle.

I also teach with almost no direct evaluative surveillance or oversight (which can be a bad thing, of course); this I note because it reduces the unnecessary stress of teaching in a high-stakes accountability environment that allows you no professional autonomy (what it means to be a K-12 teacher).

I must stress that a great deal of pettiness and an inordinate amount of unhealthy practices still plague higher education—the tenure and promotion process along with the faculty evaluation process are steeped in sexism and inequity, for example.

And the cancer that is high-stakes accountability and reducing education to work-preparation is creeping, no galloping, toward and eventually over higher education.

When I first took my university position, I was surprised at how out of touch professors were with K-12 teaching and the negative impact of the standards and high-stakes testing movement. I, in fact, warned my colleagues that the accountability movement would some day come to colleges and universities so it was in their own self-interest to begin fighting the movement in K-12 schools.

But they didn’t listen.

Higher education isn’t called the Ivory Tower for nothing.

So this brings me to why anyone would be an educator—especially in 2018 when the consequences weighed against the rewards for being a K-12 teacher or a college professor are tipping mightily in the wrong direction.

To teach, at any level, for many of us is something like a calling. Just as one day in my first year of college I recognized I am a writer (I did not choose that), I know myself to be a teacher.

Despite my introversion, and my discomfort with people, crowds, I am never more relaxed than in a classroom with students. We are there with common purpose and we mostly are seeking ways to be a community.

These are things I believe in, things I trust about the possibility of humans being better than we have been so far.

To be an educator, then, is not the problem in that the profession itself, whether K-12 or in higher education, is compelling and deeply fulfilling.

The problem is that to be a teacher in the U.S. is colored by the cultural negative attitude toward labor, being a worker, and the power of collective workers against the wishes of corporatism.

Teaching at all levels has continually been corrupted by the urge to reduce public institutions to private entities driven by corporate paradigms.

K-12 teachers have always worked in environments that isolate us, overwork us so that we cannot resist, and have gradually become less and less unionized. Much of higher education (because of the tenure and promotion process as well as departmental politics) has also allowed competition to trump collaboration.

It is not so much why anyone would be an educator, but why those of us who teach at any level have allowed our profession to be dismantled, devalued, and dehumanizing.

And finally, teachers and professors are regularly policed for being political, admonished for being activists. And to that we must ask, in whose interest is this political call for teachers and professors to not be political?

Isolated, silenced, and depoliticized, we educators are failing a profession that deserves better.

In solidarity, raising our voices, and actively exercising our politics, we educators can resurrect one of the most valuable acts of labor humans can embrace.

The latter is why anyone would be an educator.


[1] The martyr/missionary dilemma.