Tag Archives: high-stakes accountability

Beyond Doing the Wrong Thing the Right Way

My nephew is in elementary school, and my parents drive him to school each morning and arrive at his school an hour or two before school lets out each afternoon. This is a rural community in the South where many family members do the same—surrounding the school well before dismissal and often socializing.

Recently, my mother told me about parents of a child at that school who are refusing to allow their son to be placed in a test-prep class (and removed from his normal class) because of his low score on a MAP test. The parents are adamant that his test grades in class are high 90s, and they see no reason for his being out of that class to prepare for a test. [1]

Over the past year, I have also been a part of or observed two situations with education policy: one involving a compromise about reading legislation linked to retaining 3rd graders and another about language in the state’s science standards.

In both cases, good pedagogy and foundational aspects of the fields have been sacrificed for political expediency.

The situation at my nephew’s school and both of these education policy developments represent for me the central problem with the Common Core and high-stakes testing arguments: We are content to find the right ways to do the wrong things.

For example, a new report Data-Driven Improvement and Accountability by Andy Hargreaves, Henry Braun, and Kathleen Gebhardt for NEPC is excellent work that confronts how accountability has failed as well as how data should be used more effectively.

However, despite the high quality of this report, it doesn’t allow us to move a few steps further back and consider not using the accountability paradigm at all.

While there are certainly some outrageous claims made against the Common Core (the Tea Party railings against Obama and big government that often play loose with facts) and some passionate arguments against CC that are credible but tarnished by that passion (concerns about Gates money and its influence as well as the role of David Coleman), the dominant narratives about CC and the high-stakes testing connected with the new standards are about how critics are focusing on bad implementation, and not flaws in the standards or the tests. From that, the arguments are how to implement CC and the tests right.

And here is where we are failing.

Setting aside the impassioned arguments against CC and more high-stakes testing, a good deal of evidence shows that most of our educational problems have nothing to do with either the presence or quality of standards or tests (see Mathis, 2012, for example).

As well, we have considerable reason to be concerned about accountability based on high-stakes tests—Campbell’s Law and Gerald Bracey’s caution about what is tested is what is taught.

Simply put, there is no right way to implement standards and high-stakes tests in an accountability framework because neither the goals/purposes nor problems of U.S. public education call for that paradigm; schools are not failing due to a lack or poor quality of accountability.

And that leads to the next typical response: All critics do is criticize. Where is your alternative?

Let’s consider that, then.

Is there any value in a cohesive body of knowledge associated with the major disciplines (what we typically call standards)? Yes.

So what is wrong with Common Core? CC is a bureaucratic, top-down mandate. In all fields, there exists a cohesive body of agreed upon knowledge, a set of contemporary debates, and a set of enduring debates. Public school standards fail because they are primarily bureaucratic and essentially partisan political documents.

Building on that essential problem, then, a cohesive body of knowledge identified for a field of study that is a resource for an autonomous teacher—this should be the starting point of education reform.

However, even if we address re-tooling how we view standards, even if we drop high-stakes testing (and we should), and even if we afford teachers the professional autonomy they deserve, schools will still ultimately fail unless we address equity and opportunity both in the lives and in the education of all children.

We now face a tremendous wake-up call since—despite the increasingly influential and pervasive accountability movement in our schools—the majority of students in U.S. public schools in the South and urban schools live in poverty.

That fact itself calls into question our social policy and the likelihood that schools alone can overcome social dynamics.

There are no right ways to do the wrong things. CC, new high-stakes testing, and more accountability are simply the wrong things.

[1] Evidence from the SAT seems to support these parents’ wishes since GPA remains a better predictor of college success than SAT scores. Despite claims to the contrary, teachers’ subjective grading is quite powerful, and more powerful than a so-called objective measure.

Medicating ADHD in the Brave New World of High-Stakes Accountability

Miranda: O wonder!
How many goodly creatures are there here!
How beauteous mankind is! O brave new world,
That has such people in’t.

—William Shakespeare, The Tempest, Act V, Scene I, ll. 203–206

—–

Utopias seem much more attainable than one may have previously thought. And we are now faced with a much more frightening thought: how do we prevent their permanent fulfillment?…Utopias are attainable. The way of life points towards them. But perhaps a new century will begin, a century in which intellectuals and the educated class will find means of preventing utopias, and we will return to a non-utopian society, which may be less perfect, but will offer more freedom.

—Nicolas Berdiaeff

My love of science fiction (SF) has its roots firmly in Marvel comic books from the 1970s and the SF novels of Arthur C. Clarke and Niven/Pournelle. When I became acquainted with what teachers called “good” and “real” literature, I was immediately drawn to George Orwell and Aldous Huxley as anointed SF writers.

As an adult, I am the sort of SF reader who treasures Kurt Vonnegut and Margaret Atwood, and if I am pushed, I have to admit I value Orwell more as an essayist than novelist and always enjoyed Huxley’s Brave New World over 1984—believing both works are brilliant, but preferring BNW as a reader.

The opening passages are two foundational quotes behind the message of BNW, the Shakespeare the source of the title and the Berdiaeff a nod to Huxley’s parody of utopian fiction.

In his Foreword to the Perennial Classic edition of BNW, Huxley explains:

But Brave New World is a book about the future and, whatever its artistic or philosophical qualities, a book about the future can interest us only if its prophesies look as though they might conceivably come true….The theme of Brave New World is not the advancement of science as such; it is the advancement of science as it affects human individuals. (p. ix)

Beyond purpose, Huxley continues, speculating about “A really efficient totalitarian state”

would be one in which the all-powerful executive of political bosses and their army of managers control a population of slaves who do not have to be coerced, because they love their servitude. To make them love it is the task assigned, in present-day totalitarian states, to ministries of propaganda, newspaper editors and schoolteachers….The great triumphs of of propaganda have been accomplished, not by doing something, but by refraining from doing. Great is truth, but still greater, from a practical point of view, is silence about the truth. (p. xii)

SF, at its best, however, is not predictive, but cautionary; as Neil Gaiman has reminded us, “The world doesn’t have to be like this. Things can be different.” Thus, Huxley warns:

Round pegs in square holes tend to have dangerous thoughts about the social system and to infect others with their discontents….In conjunction with the freedom to daydream under the influence of dope and movies and the radio, it will help to reconcile his subjects to the servitude which is their fate….Indeed, unless we choose to decentralize and to use applied science [1], not as the end to which human beings are to be made the means, but as the means to producing a race of free individuals, we have only two alternatives to choose from: either a number of national, militarized totalitarianisms…or else one supra-national totalitarian. (pp. xiii, xiv)

And then, Huxley conclude: “You pays your money and you takes you choice” (p. xiv).

The speculative and cautionary possibilities found in SF rarely come to fruition in the real world in the dramatic ways of novels or films (or in the somewhat looney ways political factions rant and rave in public discourse). So it seems likely that we are apt never to listen or to act in ways that we should and could.

Huxley, I think, was in many ways speaking to this—“The Not-So-Hidden Cause Behind the A.D.H.D. Epidemic”:

Between the fall of 2011 and the spring of 2012, people across the United States suddenly found themselves unable to get their hands on A.D.H.D. medication. Low-dose generics were particularly in short supply. There were several factors contributing to the shortage, but the main cause was that supply was suddenly being outpaced by demand.

The number of diagnoses of Attention Deficit Hyperactivity Disorder has ballooned over the past few decades. Before the early 1990s, fewer than 5 percent of school-age kids were thought to have A.D.H.D. Earlier this year, data from the Centers for Disease Control and Prevention showed that 11 percent of children ages 4 to 17 had at some point received the diagnosis — and that doesn’t even include first-time diagnoses in adults. (Full disclosure: I’m one of them.)

That amounts to millions of extra people receiving regular doses of stimulant drugs to keep neurological symptoms in check. For a lot of us, the diagnosis and subsequent treatments — both behavioral and pharmaceutical — have proved helpful. But still: Where did we all come from? Were that many Americans always pathologically hyperactive and unable to focus, and only now are getting the treatment they need?

Probably not. Of the 6.4 million kids who have been given diagnoses of A.D.H.D., a large percentage are unlikely to have any kind of physiological difference that would make them more distractible than the average non-A.D.H.D. kid. It’s also doubtful that biological or environmental changes are making physiological differences more prevalent. Instead, the rapid increase in people with A.D.H.D. probably has more to do with sociological factors — changes in the way we school our children, in the way we interact with doctors and in what we expect from our kids.

As disturbing as this is, the final paragraph of this article may be the most significant:

Today many sociologists and neuroscientists believe that regardless of A.D.H.D.’s biological basis, the explosion in rates of diagnosis is caused by sociological factors — especially ones related to education and the changing expectations we have for kids. During the same 30 years when A.D.H.D. diagnoses increased, American childhood drastically changed. Even at the grade-school level, kids now have more homework, less recess and a lot less unstructured free time to relax and play [emphasis added]. It’s easy to look at that situation and speculate how “A.D.H.D.” might have become a convenient societal catchall for what happens when kids are expected to be miniature adults. High-stakes standardized testing, increased competition for slots in top colleges [emphasis added], a less-and-less accommodating economy for those who don’t get into colleges but can no longer depend on the existence of blue-collar jobs — all of these are expressed through policy changes and cultural expectations, but they may also manifest themselves in more troubling ways — in the rising number of kids whose behavior has become pathologized.

The rise of ADHD diagnoses and medications has run concurrent to the accountability era in education, sharing the same thirty-year history. O brave new world of high-stakes accountability and the ADHD medication needed to make the students love their servitude to the tests…

[1] Paul Boyle, “A U.K. View on the U.S. Attack on Social Sciences,” Science, 341 (August 16, 2013), p. 719.

Limes or Leeches: A Thought Experiment about High-Stakes Accountability

History is a powerful teacher—if we are willing to learn.

Many educators and scholars have triggered the truism “Insanity is doing the same thing over and over while expecting different results,” but that argument seems always to fall on deaf ears among our self-chosen education reformers, the media, and the public.

So let’s venture into history and explore a thought experiment: Would you prefer limes or leeches?

Study the history of identifying and treating scurvy, including how British sailors became known as “limeys.”

Now study the history of blood letting, and the use of leeches.

Solutions, history shows, must be built on a clear identification of the problems and then a careful analysis of what those solutions must be.

Limes and citrus fruits proved credible solutions for preventing scurvy—while “[i]n the overwhelming majority of cases, the historical use of bloodletting was harmful to patients.”

High-stakes accountability built on standards and high-stakes testing is the type of insanity found in bloodletting because the overwhelming majority of educational problems have nothing to do with accountability, standards, or testing—not the lack of accountability, standards, or testing, not the quality of accountability, standards, or testing.

Nothing.

In the education of your child, would you prefer limes or leeches? [And now let’s apply that answer to “other people’s children” because “they’re all our children.”]

Let’s stop the bloodletting.

“We Are Entering the Age of Infinite Examination”

In 2011, Jim Taylor entered the poverty and education debate, asking U.S. Secretary of Education Arne Duncan and billionaire/education entrepreneur Bill Gates a direct question*:

I really don’t understand you two, the U.S. Secretary of Education and the world’s second richest man and noted philanthropist. How can you possibly say that public education can be reformed without eliminating poverty?

Taylor’s discussion comes to an important element in the debate when he addresses Gates: “Because without understanding the causes of problems, we can’t find solutions,” explains Taylor, adding. “You’re obviously trying to solve public education’s version of the classic ‘chicken or egg’ conundrum.”

Here, recognizing the education/poverty debate as a chick-or-egg problem is the crux of how this debate is missing the most important questions about poverty—and as a result, insuring that Duncan, Gates, Michelle Rhee, Paul Vallas, and other corporate reformers are winning the argument by perpetuating the argument.

The essential questions about poverty and education should not focus on whether we should address poverty to improve education (where I stand, based on the evidence and the words of Martin Luther King, Jr.) or whether we should reform education as the sole mechanism to alleviate poverty (the tenant of the “no excuses” ideology found at Knowledge Is Power Program [KIPP] charters); the essential question about poverty is: Who creates and allows poverty to exist in the wealthiest and most powerful country in recorded history?

The Conservative Nature of Power

As a basic point of logic, any organized entity—a society, a business, a school—has characteristics that are either created or tolerated by those in power controlling that organization. All entities are by their nature conservative—functioning to maintain the entity itself. In other words, institutions and their norms resist change, particularly radical change that threatens the hierarchy of power.

In the U.S., then, poverty exists in the wider society and performs a corrosive influence in the education system (among all of our social institutions, our Commons) because the ruling elite—political and corporate leaders—need poverty to maintain their elite status at the top of the hierarchy of power.

While the perpetual narratives promoted by the political and corporate elite through the media elite have allowed this point of logic to be masked and ignored in American society, we must face the reality that people with power drive the realities of those without power. Yes, the cultural narratives driven by the elite suggest that people trapped in poverty are somehow in control of that poverty—either creating it themselves due to their own sloth, that they somehow deserve their station in life, or failing to rise above that poverty (and this suggestion allows the source of poverty to be ignored) from their own failure to lift themselves up by their own bootstraps.

But that narrative has no basis in evidence—since those without power have control of that which creates the conditions benefiting the elite. The powerful allow those without power to have some token or artificial autonomy—as parents with children—in order to create the illusion of autonomy to keep revolt at bay; this is why the political and corporate elite use the word “choice” and perpetuate the myth that all classes in America have the same access to choice.

Poverty as Necessary for Current Hierarchies of Power

How does poverty benefit the powerful in the U.S.?

  • U.S. cultural narratives depend on the Utopian elements of democracy, meritocracy, and individual freedom. Those ideals form the basis for most of the cultural narratives expressed by the political and corporate elite in the U.S. Poverty works as the Other in those narratives—that which we must all reject, that which we must strive to avoid. If the Utopian goals, including eliminating poverty, is ever achieved, however, the tension between the working-/middle- class and those in poverty would be eliminated as well, exposing the artificial perch upon which the ruling elite sit. The necessity of poverty works both to keep us from attaining the Utopian goals and to make the Utopian goals attractive.
  • Poverty contributes to the crisis motif that keeps the majority of any society distracted from the minority elite benefiting disproportionately from the labor of the majority. Crises large and small—from Nazis, Communists, and Terrorists to the War on Drugs to teen pregnancy to the achievement gap and the drop-out crisis—create the perception that the average person cannot possibly keep these crises under control (crises that would plunge otherwise decent people into the abyss of poverty) and, thus, needs the leadership and protection of the elite. The majority of average people can only be carried to the promised land of Utopian peace and equality by the sheer force of personality held by only a few; these ruling elite are the only defense against the perpetual crises threatening the ideals we hold sacred (see below for how we identify those elite).
  • Along with Utopian promises and the refrain of crisis, the ruling elite need the pervasive atmosphere of fear—whether real or fabricated—in order to occupy the time and energy of the majority. [1] Poverty becomes not just a condition to be feared, but also those people to be feared. The cultural narratives—in contrast to the evidence—about poverty and people living in poverty connect poverty and crime, poverty and drug abuse, poverty and domestic violence, poverty and unattractiveness, and most of all, poverty and the failure of the individual to grasp the golden gift of personal freedom afforded by the United States.

Just as we rarely consider the sources of poverty—who controls the conditions of our society—we rarely examine the conditions we are conditioned to associate with poverty and people living in poverty. Are the wealthy without crime? Without drug abuse? Without deceptions of all kinds? Of course not, but the consequences for these behaviors by someone living in privilege are dramatically different than the consequences for those trapped in poverty.

The ruling elite have created a culture where we see the consequences of poverty, but mask the realities of privilege.

Winners always believe the rules of the game to be fair, and winners need losers in order to maintain the status of “winner.” The U.S., then, is a democracy only as a masking narrative that maintains the necessary tension among classes—the majority working-/middle-class ever fearful of slipping into poverty, and so consumed by that fear that they are too busy and fearful to consider who controls their lives: “those who are stuck at a machine and supervised for the rest of their lives.” [2]

In the narrow debate about poverty and education, we are being manipulated once again by the ruling elite, within which Duncan and Gates function, to focus on the chicken-and-egg problem of poverty/education so that we fail to examine the ruling elite creating and tolerating poverty for their own benefit. By creating the debate they want, they are winning once again.

And that success derives in large part from their successful propaganda campaign about the value of testing.

The Meritocracy Myth, Science, and the Rise of New Gods

Now that I have argued for shifting the discourse about poverty and education away from the chick-and-egg problem to the role of sustaining and tolerating poverty for the benefit of the ruing elite, let’s look at the central role testing plays in maintaining the status quo of power in the U.S. And let’s build that consideration on a couple pillars of evidence.

First, despite decades committed to the science of objective, valid, and reliable standardized testing, outcomes from standardized tests remain most strongly correlated with the socio-economic status of the students. As well, standardized tests also remain biased instruments.

Next, more recently during the thirty-year accountability era, the overwhelming evidence shows that standards, testing, and accountability do not produce the outcomes that political proponents have claimed.

Thus, just as the poverty/education question should address who creates and allows poverty and why, the current and historical testing obsession should be challenged in terms of who is benefiting from our faith in testing and why.

The history of power, who sits at the top and how power is achieved, is one of creating leverage for the few at the expense of the many. To achieve that, often those at the top have resorted to explicit and wide-scale violence as well as fostering the perception that those at the top have been chosen, often by the gods or God, to lead—power is taken and/or deserved.

“God chose me” and “God told me” remain powerful in many cultures, but in a secular culture with an ambiguous attitude toward violence (keep the streets of certain neighborhoods here crime-free, but war in other countries is freedom fighting) such as the U.S., the ruling elite needed a secular god—thus, the rise of science, objectivity, and testing:

[A] correlative history of the modern soul and of a new power to judge; a genealogy of the present scientifico-legal complex from which the power to punish derives its bases, justifications, and rules; from which it extends it effects and by which it masks its exorbitant singularity. [3]

As I noted above, testing remains a reflection of the inequity gap in society and the high-stakes testing movement has not reformed education or society, so the rising call for even more testing of students, testing based on nationalized standards and used to control teachers, must have a purpose other than the Utopian claims by the political and corporate elite who are most invested in the rising testing-culture in the U.S.

That purpose, as with the necessity of poverty, is to maintain the status quo of a hierarchy of power and to give that hierarchy the appearance of objectivity, of science.

Standards, testing, and accountability are the new gods of the political and corporate elite.

Schools in the U.S. are designed primarily to coerce children to be compliant, to be docile; much of what we say and consider about education is related to discipline—classroom management is often central to teacher preparation and much of what happens during any school day:

The exercise of discipline presupposes a mechanism that coerces by means of observation; an apparatus in which the techniques that make it possible to see induce effects of power in which, conversely, the means of coercion make those on whom they are applied clearly visible. [4]

In education reform, the surveillance of students, and now the surveillance of teachers, is not covert, but in plain view in the form of tests (and even Gates calling for cameras in all classrooms) allowing that surveillance to be disembodied from those students and teachers—and thus appearing to be impersonal—and examined as if objective and a reflection of merit.

Testing as surveillance in order to create compliance is central to maintaining hierarchies of power both within schools (where a premium is placed on docility of students and teachers) and society, where well-trained and compliant voters and workers sustain the positions of those in power:

[T]he art of punishing, in the regime of disciplinary power, is aimed neither at expiation, nor precisely at repression….It differentiates individuals from one another, in terms of the following overall rule: that the rule be made to function as a minimal threshold, as an average to be respected, or as an optimum toward which one must move. It measures in quantitative terms and hierarchizes in terms of value the abilities, the level, the “nature” of individuals….The perpetual penalty that traverses all points and supervises every instant in the disciplinary institution compares, differentiates, hierachizes, homogenizes, excludes. In short, it normalizes. [5]

The political and corporate elite in the U.S. have risen to their status of privilege within the “scientifico-legal complex” that both created that elite and is then perpetuated by that elite. As I noted above, the winners always believe the rules of the game to be fair and will work to maintain the rules that have produced their privilege.

The Expanded Test Culture—“The Age of Infinite Examination”

Foucault has recognized the central place for testing within the power dynamic that produces a hierarchy of authority:

The examination combines the techniques of an observing hierarchy and those of normalizing judgment. It is a normalizing gaze, a surveillance that makes possible to qualify, to classify, and to punish. [6]

Thus, as the rise of corporate paradigms to replace democratic paradigms has occurred in the U.S. over the last century, we can observe a rise in the prominence of testing along with how those tests are used. From the early decades of the twentieth century, testing in the U.S. has gradually increased and expanded in its role for labeling, sorting, and controlling students. In the twenty-first century, testing is now being wedged into a parallel use to control teachers.

Those in power persist in both cases—testing to control students and testing to control teachers—to claim that tests are a mechanism for achieving Utopian goals of democracy, meritocracy, and individual freedom, but in both cases, those claims are masks for implementing tests as the agent of powerful gods (science, objectivity, accountability) to justify the current hierarchy of power—not to change society or education: “[T]he age of the ‘examining’ school marked the beginnings of a pedagogy that functions as science.” [7]

Foucault, in fact, identifies three ways that testing works to reinforce power dynamics, as opposed to providing data for education reform driven by a pursuit of social justice.

First, testing of individual students and using test data to identify individual teacher quality create a focus on the individual that reinforces discipline:

In discipline, it is the subjects who have to be seen. Their visibility assures the hold of the power that is exercised over them. It is the fact of their being constantly seen…that maintains the disciplined individual in his subjection. And the examination is the technique by which power…holds them in a mechanism of objectification. [8]

This use of testing resonated in President Obama’s first term as Secretary Duncan simultaneously criticized the misuse of testing in No Child Left Behind and called for an expansion of testing (more years of a student’s education, more areas of content, and more directly tied to individual teachers), resulting in: “We are entering the age of infinite examination and of compulsory objectification.” [9]

As Giles Deleuze confirms in “Postscript on the Societies of Control”:

We are in a generalized crisis in relation to all the environments of enclosure—prison, hospital, factory, school, family….The administrations in charge never cease announcing supposedly necessary reforms: to reform schools, to reform industries, hospitals, the armed forces, prisons….In disciplinary societies one was always starting again (from school to the barracks, from the barracks to the factory), while in the societies of control one is never finished with anything—the corporation, the educational system, the armed services being metastable states coexisting in one and the same modulation, like a universal system of deformation. (pp. 3-4, 5)

Next, testing has provided a central goal of sustaining the hierarchy of power—“the calculation of gaps between individuals, their distribution in a given ‘population.’” [10] Testing, in effect, does not provide data for addressing the equity/achievement gap, testing has created those gaps, labeled those gaps, and marginalized those below the codified level of standard.

What tends to be ignored in the testing debate is that some people with authority determine what is taught, how that content is taught, what is tested, and how that testing is conducted. In short, all testing is biased and ultimately arbitrary in the context of who has authority.

And finally, once the gaps are created and labeled through the stratifying of students and teachers:

[I]t is the individual as he[/she] may be described, judged, measured, compared with others, in his[/her] very individuality; and it is also the individual who has to be trained or corrected, classified, normalized, excluded, etc. [11]

Poverty and Testing—Tools of the Privileged

Within the perpetual education and education reform debates, the topics of poverty and testing are central themes (poverty is no excuse, and better tests are always being promised), but we too often are missing the key elements that should be addressed in the dynamic that exists between poverty and testing.

Yes, standardized tests remain primarily reflections of social inequity that those tests make possible, labeled as “achievement gaps.”

But the central evidence we should acknowledge is that the increased focus on testing coming from the political and corporate elite is proof that those in privilege are dedicated to maintaining poverty as central to their hierarchy of authority.

Standards, testing, accountability, science, and objectivity are the new gods that the ruling class uses to keep the working-/middle-class in a state of “perpetual anxiety,” fearing the crisis of the moment and the specter of slipping into poverty—realities that insure the momentum of the status quo.

* Reposted and revised/updated from earlier publication at Truthout.

References

[1] Foucault, M. (1984). The Foucault reader. Ed. P. Rabinow. New York: Pantheon Books. See Foucault’s discussion of “perpetual anxiety” (p. 144) in “The Birth of the Asylum” from Madness and Civilization.

[2] Ibid., p. 177.

[3] Ibid., p. 170.

[4] Ibid., p. 189.

[5] Ibid., p. 195.

[6] Ibid., p. 197.

[7] Ibid., p. 198.

[8] Ibid., p. 199.

[9] Ibid., p. 200.

[10] Ibid., p. 202.

[11] Ibid., p. 203.

NFL again a Harbinger for Failed Education Reform?

During the impending NFL strike in 2011—the act of a union—I drew a comparison between how the public in the U.S. responds to unionization in different contexts:

“I am speaking about the possible NFL strike that hangs over this coming Super Bowl weekend: a struggle between billionaires and millionaires, which, indirectly, shines an important light on the rise of teacher and teacher union-bashing in the US. Adam Bessie, in Truthout, identifies how the myth of the bad teacher has evolved.”

Once again, the NFL is facing a situation that I believe and even hope is another harbinger of how education reform can be halted: A suit filed by the family of Junior Seau:

“The family said the league not only ‘propagated the false myth that collisions of all kinds, including brutal and ferocious collisions, many of which lead to short-term and long-term neurological damage to players, are an acceptable, desired and natural consequence of the game,’ but also that ‘the N.F.L. failed to disseminate to then-current and former N.F.L. players health information it possessed’ about the risks associated with brain trauma.”

This law suit has prompted a considerable amount of debate concerning whether or not the NFL as we currently know it could be dramatically reconfigured under the pressure of more law suits. In other words, the inherent but often ignored or concealed dangers of football are now being exposed by legal action, in much the same way as the tobacco industry was unmasked and thus the entire culture of smoking has radically changed in the last couple decades.

With the release of the Education Policy Analysis Archives (EPAA) Special Issue on “Value-Added Model (VAM) Research for Educational Policy,” a similar question should now be raised about the future of implementing high-stakes accountability policies that focus on teacher evaluation and retention through VAM-style metrics.

“High-Stakes Implementation of VAM,…Premature”

Two articles in the special issue from EPAA examines the validity and reliability of VAM-based teacher evaluation in high-stakes settings and then places these policies in the context of legal ramifications faced by districts and states for those policies.

“The Legal Consequences of Mandating High Stakes Decisions Based on Low Quality Information: Teacher Evaluation in the Race-to-the-Top Era” (Baker, Oluwole, & Green, 2013) identifies the current trend: “Spurred by the Race-to-the-Top program championed by the Obama administration and a changing political climate in favor of holding teachers accountable for the performance of their students, many states revamped their tenure laws and passed additional legislation designed to tie student performance to teacher evaluations” (p. 3). Because of the political and public momentum behind reforming teacher evaluation, Baker, Oluwole, and Green seek “to bring some urgency to the need to re-examine the current legislative models that put teachers at great risk of unfair evaluation, removal of tenure, and ultimately wrongful dismissal” (p. 5).

While Baker, Oluwole, and Green offer a detailed and evidence-based examination of the VAM-based and student growth model approaches to high-stakes teacher accountability, they ultimately place the weaknesses of reform policies in the context of potential challenges from teachers who believe they have been wrongfully evaluated or dismissed:

“In this section, we address the various legal challenges that might be brought by teachers dismissed under the rigid statutory structures outlined previously in this article. We also address how arguments on behalf of teachers might be framed differently in a context where value-added measures are used versus one where student growth percentiles are used. Where value-added measures are used, we suspect that teachers will have to show that while those measures were intended to attribute student achievement to their effectiveness, the measures failed to do so in a number of ways. That is, where value-added measures are used to assign effectiveness ratings, we suspect that the validity and reliability, as well as understandability of those measures would need to be deliberated at trial. However, where student growth percentiles are used, we would argue that the measures on their face are simply not designed for attributing responsibility to the teacher, and thus making such a leap would necessarily constitute a wrongful judgment. That is, one would not necessarily even have to vet the SGP measures for reliability or validity via any statistical analysis, because on their face they are invalid for this purpose.”

The analysis ultimately discredits both the use of narrow metrics to determine teacher quality and the high-stakes policies being implemented using those metrics, concluding with the ironic consequences of these policies: “Overly prescriptive, rigid teacher evaluation mandates, in our view, are likely to open the floodgates to new litigation over teacher due process rights. This is likely despite the fact that much of the policy impetus behind these new evaluation systems is the reduction of legal hassles involved in terminating ineffective teachers” (pp. 18-19).

In “Legal Issues in the Use of Student Test Scores and Value-added Models (VAM) to Determine Educational Quality” (Pullin, 2013), the rapid increase of VAM-based accountability is further examined in the context of “a wide array of potential legal issues [that] could arise from the implementation of these programs” (p. 2).

Pullin notes the motivation for reforming teacher evaluation:

“VAM initiatives are consistent with a highly publicized press from the business community and many politicians to make government services more like private business, data-driven to measure productivity and accountability (Kupermintz, 2003). VAM approaches are in part a response to concerns that the current system of selecting and compensating teachers based their education and credentials is insufficient for insuring teacher quality (Corcoran, 2011; Gordon, Kane & Staiger, 2006; Hanushek & Rivkin, 2012; Harris, 2011). There have been increasing expressions of concern that teacher evaluation practices are not robust and do not improve practice (Kennedy, 2010). In the contemporary public policy context, much of the support for the use of student test scores for educator evaluation comes from a concern that the current system for evaluation is ineffective and that the current legal protections for teachers are too cumbersome for schools seeking to terminate teachers (Harris, 2009, 2011).”

While a business model for addressing quality control of a work force may seem efficient, Pullin highlights that legal ramifications are likely with these new models.

Pullin’s analysis offers a detailed and useful examination of previous court cases involving the use of test scores to evaluate educators, including recent cases involving VAM, concluding that the picture is not clear on how the courts may rule in the future, but that a pattern exists of “heavy judicial deference to state and local education policymakers and the allure of using test scores to make decisions about education quality” (p. 5).

Further, Pullin notes “there are differences of perspective among social scientists about VAM and the defensibility of using it to make high-stakes decisions about educators,” further complicating the concerns of legal action (p. 9).

While raising many other complications, Pullin also notes that students and parents may enter legal battles using VAM metrics “to substantiate their own legal claims that schools are not meeting their obligations to provide education” (p. 14).

Pullin concludes with a sobering look at teacher quality reform built on VAM and implemented in high-stakes environments:

“In the broad contemporary public policy context for education reform, the desire for accountability and transparency in government, coupled with heavily financed criticisms of public school teachers and their unions, may mean that VAM initiatives will prevail. The concerns of education researchers about VAM, coupled with legal obligations for the validity and reliability of education and evaluation programs should require judges and education policymakers to take a closer look for future decision-making. At the same time, the social science research community should be generating substantial new and persuasive evidence about VAM and the validity and reliability of all of its potential uses. For public policymakers, there are strong reasons to suggest that high-stakes implementation of VAM is, at best, premature and, as a result, the potential for successful legal challenge to its use is high. The use of VAM as a policy tool for meaningful education improvement has considerable limitations, whether or not some judges might consider it legally defensible.” (p. 17)

Like the NFL, federal and state governments may soon be compelled to reform the reform movement under the threat of legal action from a variety of stakeholders since the science of teacher evaluation remains far behind the curve of implementation, particularly when teacher evaluation is high-stakes and based on VAM and other metrics linked to student test scores.

The special issue from EPAA is yet another call for political leadership to pause if not end wide-scale teacher evaluation and retention models that pose legal, statistical, and funding challenges that those leaders appear unwilling to acknowledge or address.