Tag Archives: student test scores

Education, education reform, Teaching, Testing, VAM

When the Shoe Is on the Other Foot: Lessons for Teachers in Misguided Accountability

2 Comments

If we imagined a pictorial representation of the evolution of education accountability, similar to the standard image we associate with human evolution—

—then we’d have to confront that the accountability era begun in the early 1980s focused first on students, requiring them to pass exit exams (regardless of their having taken and passed all of the required courses for graduation) in order to receive their diplomas.

Next, schools were the target of accountability with the advent and distribution of school report cards.

By the end of the first and beginning of the second decades of the twenty-first century, teachers have found their place at the accountability table, with some suggesting that teachers are now being fed their just desserts. Merit pay linked to student test scores and the more recent flurry of implementing value-added methods (VAM) of teacher evaluation and retention in many ways bring teachers into decades-long predicaments faced by students and schools: the misguided and unfair weight of standardized testing used in dysfunctional and invalid ways.

When I posted about how absurd teacher accountability has become, I expected most on my Twitter feed to recognize the situation in New York as unfair and a harsh warning of the mounting weight of failed accountability:

A Bronx performing arts school’s dance instructor will be judged on students’ English exam scores. Physical education teachers at a transfer school in Brooklyn are going to teach Olympic history lessons to prepare students for the history tests that will help determine their ratings. And teachers in Queens are putting the fate of their evaluations into a final exam that they don’t teach, but yields high pass rates.

The scenarios are not unusual — across [New York City] this year, thousands of teachers will be rated in large part based on test scores of subjects and students that they do not teach.

Rather, the scenarios are examples of how schools have tried to comply with a new teacher evaluation system that must factor student performance into final ratings. They also represent how the original purpose of the evaluations, to differentiate teachers’ effectiveness, has been squeezed by restrictive state laws, limited resources, and a tight timeline for implementation.

“It’s insane to me that 40 percent of my evaluation is going to be based on someone else’s work,” said Jason Zanitsch, a high school drama teacher who will share the same “student growth” score with colleagues in his school this year.

However, the first response I received raised a much different point:

@plthomasEdD if teachers don’t like this then way assign all the group work which is just as bad for the kids? hmmm….

My first response was to note that holding teachers accountable for the work of other teachers and the test scores of students they do not even teach is not truly analogous to having students do group work, and then be graded for that group work.

As @Tim_10_ber and I exchanged tweets, I came to recognize that I was arguing from my idealized position on how best to implement group work (group work must require collaboration—or it is simply students sitting close to each other doing individual work—and any grades assigned to group work must be articulated to reflect participation) and @Tim_10_ber was confronting a position with which I agree—that group work is often implemented and graded carelessly and thus unfairly to students.

It is from that recognition, then, that I want to make an argument about the only potential positive outcome related to the unjustifiable use of merit pay and VAM in teacher evaluation, pay, and retention: teachers need to learn how to teach better now that the shoe is on the other foot. Some ironic lessons teachers should learn from invalid teacher accountability include the following:

  • Testing and grades often do far more educational harm than good; the time has come to consider de-testing and de-grading our teaching. Teacher feedback, student self-assessment, student-created rubrics, and re-imagined assessment situations (such as group assessments) and formats are all better alternatives to tests and grades, if our goal is equitable and effective learning opportunities for students.
  • The central flaw with teacher accountability being linked to student test scores and the standards movement is that teachers have experienced declining autonomy in both their content and pedagogy as well as the high-stakes tests themselves. Accountability without autonomy is tyranny. This lesson translates into how often student learning is reduced to mere compliance. Students being held accountable also must have their autonomy honored; thus, students deserve far more choice in their learning than they have been traditionally allowed.
  • As noted by @Tim_10_ber, teachers must be far more vigilant about designing, assigning, and assessing group work, with a keen eye on autonomy, engagement, and causation/correlation (what are fair associations between each student and the outcomes of the group).

The accountability era has nearly destroyed public education. Little about accountability based on standards and high-stakes testing can be embraced or endorsed.

But oppressive and even capricious mandates tend to be leveled at the least among us first; once those policies trickle up to those in power—in other words, when the shoe is on the other foot—living with inequity, unfair accountability, and unworkable conditions can open our eyes to our own flaws as teachers.

As we continue to fight for our professional autonomy and dignity, taking moral stands of non-cooperation, let’s be sure to bring that fight to our classrooms and honor the autonomy and dignity of all our students as a model for those in power who have yet to see the flaws of their ways through the distorting lens of privilege they wear.

In the words of Henry David Thoreau in “Civil Disobedience”:

If I devote myself to other pursuits and contemplations, I must first see, at least, that I do not pursue them sitting upon another man’s shoulders. I must get off him first, that he may pursue his contemplations too….

If the injustice is part of the necessary friction of the machine of government, let it go, let it go; perchance it will wear smooth — certainly the machine will wear out. If the injustice has a spring, or a pulley, or a rope, or a crank, exclusively for itself, then perhaps you may consider whether the remedy will not be worse than the evil; but if it is of such a nature that it requires you to be the agent of injustice to another, then, I say, break the law. Let your life be a counter friction to stop the machine. What I have to do is to see, at any rate, that I do not lend myself to the wrong which I condemn [emphasis added].

Teacher Evaluation, VAM

VAM: A Primer

4 Comments

Education reform has existed at some policy and public levels since at least the 1890s in the U.S. The current reform movement grounded in state-based accountability began in the early 1980s with a Nation at Risk, and then was nationalized in 2001 with No Child Left Behind.

In the first decades of the recent accountability era, standards and high-stakes testing were implemented and periodically revised at the state level with the primary focus being on student and school accountability. The current cycle, however, has seen an increase in policies and practices aimed at teacher accountability and increasing teacher quality—despite a solid research base showing that teacher quality constitutes only 10-15% of measurable student outcomes (test data).

The focus on increasing teacher quality and accountability has included both experiments and policies with value added methods (VAM) of determining teacher quality in order to label, rank, sort, and retain or dismiss teachers. VAM claims to isolate teacher quality through pre- and post-testing methods that seek to identify teacher quality and isolate that from the other factors reflected in test scores.

Before policy-makers and stakeholders in education commit to reforming teacher evaluation and retention, foundational questions must be addressed, and then the current facts about VAM must be acknowledged.

First, the foundational questions:

(1) What evidence exists identifying teacher quality as a primary or significant problem facing a school or district? Where, then, does teacher quality rank as a priority for a school, district, or state in terms of cost effectiveness in committing funding to the reform?

(2) Are all elements of implementing VAM at any percentage to the revised teacher evaluation process valid for determining teacher quality? In other words, what measures are taken to account for using student test scores (designed to reflect student learning, and not designed to reflect teacher quality) as data points for teacher quality?

(3) Have the teaching and learning environments for students’ home and schools been addressed to insure equitable teaching and learning environments in which determining teacher quality becomes valid?

Next, what is the current knowledge base about VAM*?:

(1) Including VAM at any percentage in reformed teacher evaluation models is currently in the experimental phase. Data and the validity of including VAM are being tested, but almost no researchers currently claim VAM (at any percentage but certainly at high percentages such as 40-50%) to be ready for widespread implementation.

(2) VAM models for labeling teacher quality are highly unstable. Teacher rankings tend to shift with new populations of students.

(3) Researchers agree that VAM is unlikely ever to be completely stable; thus, it is possible that VAM will never be a practical or fair element in teacher evaluation, particularly at the individual teacher level or in any single year.

(4) The statistical and practical requirements to isolate teacher quality in student test scores pose tremendous costs in time and funding that also may prove to be not cost effective in the context of most school systems’ priorities. In order to implement a fair and equitable teacher evaluation system that includes VAM at any percentage, states must create and implement pre- and post-tests to all students in all teachers’ courses, creating a new and costly commitment to education funding.

(5) Decades of research on high-stakes testing have shown many negative unintended consequences to accountability measures focusing on students and schools; including VAM in high-stakes accountability policies focusing on teachers is likely to have similar unintended negative consequences such as discouraging high-quality teachers from working with high-needs populations of students.

Thus, policy-makes and stakeholders are strongly cautioned to consider education reform priorities, the experimental nature of VAM, and the current knowledge base on VAM before committing tax dollars to either field testing or implementing new teacher evaluation policies built in any way on VAM.

* See a recent review of teacher evaluation reform for links and citations to numerous research studies on VAM.