From Failing to Killing Writing: Computer-Based Grading

[Header Photo by Clément Hélardot on Unsplash]

In Ernest Hemingway’s The Sun Also Rises, Bill and Mike discuss Mike’s bankruptcy:

“How did you go bankrupt?” Bill asked.

“Two ways,” Mike said. “Gradually and then suddenly.”

Someday soon, two teachers of writing will be sitting and discussing the death of teaching writing, and the conversation will sound much the same.

Teaching writing came into its own in the 1970s and 1980s with great promise that the discipline of teaching composition would find its way into K-12 classrooms; this potential rested in the arms of the National Writing Project and its state affiliates across the U.S., often connected with universities.

However, we have sat silently and watched the accountability era dismantle that hope, and as a result, we have failed the teaching of writing [1].

Standards and high-stakes testing have slowly bled that promise dry, and then the addition of the writing section of the SAT kicked writing instruction while it was down. But the final nail in the coffin?

Calls for computer-based grading of writing:

Here is where leadership is needed from teachers and administrators.  Before some company comes up with a way to grade essays and boards of education become enamored with the idea, and legislators find new ways to require their use…let’s lead.  The technology is here….

We must lead the conversation by knowing and understanding how the technology can improve the educational process, which is based on the most important relationship between teacher and student. In educating our communities, it is essential to begin with the intention of improving teacher and student contact time, not replacing it.  We need to design the solution, not be given it.  First steps are opening our minds to the possibilities.

If you take the time, this is the same self-defeating fatalism that accompanies advocacy for Common Core: Let’s shoot ourselves in the foot before someone else does it!

The piece quoted above asks Will We Ever Allow Computers To Grade Students’ Writing?—to which I say, probably because we tend to do whatever is least credible in our education policy.

A better question is Should We Ever Allow Computers To Grade Students’ Writing?—to which the answer is an unequivocal No! 

And thus I offer a reader of resources for speaking that truth to such calls:

Apologies to Sandra Cisneros, Maja Wilson

NCTE Position Statement on Machine Scoring

Thomas, P.L. (2005, May). Grading student writing: High-stakes testing, computers, and the human touch. English Journal, 94 (5), 28-30:

As a writer, I cannot imagine composing without my trusted iMac and iBook. And as a writing teacher, I watched the value computers and word processors had for my students—particularly as the technology contributed to students’ ability to write more and to revise more efficiently. While computers and computer programs do offer a huge benefit for the teaching of writing, they must remain merely a tool; we cannot allow anyone to suggest that computers can substitute for humans in the ultimate evaluation of a composition.

Our students’ writing has “something the tests and machines will never be able to measure,” and it is now the duty of all writing teachers to make known the art of human assessment of writing. (pp. 29-30)

[1] Please see the following:

Why Are We (Still) Failing Writing Instruction?

More on Failing Writing, and Students

New Criticism, Close Reading, and Failing Critical Literacy Again

RECOMMENDED: Writing Instruction that Works, Applebee and Langer

For Additional Reading

Computer Writer Vs. Computer Grader

Critique of Mark D. Shermis & Ben Hammer, “Contrasting State-of-the-Art Automated Scoring of Essays: Analysis,” Les C. Perelman

Writing Instructor, Skeptical of Automated Grading, Pits Machine vs. Machine

Computerized Grading: Purloining the Analysis, the Most Fundamental Exposition of Humanity

Flunk the robo-graders

Toni Morrison, the White Gaze, Race, and Writing

If Charlie Rose or Bill Moyers sat down with Tolstoy, do you imagine they’d ask him if he could write a book and not deal with race?

I invite you to view several clips of Toni Morrison, and others, exploring the white gaze, race, and writing. Very illuminating (and reminds me of those people who shout “why is it always about race” when it is—as in the Richard Sherman case).

Criticizing KIPP Critics

An early and consistent proponent of Knowledge Is Power Program (KIPP) charter schools, Jay Mathews has joined a rising group of KIPP advocates directly criticizing KIPP critics, offering two arguments: KIPP charter schools are not abusive or excessively authoritarian, and KIPP critics are prone to misleading hyperbole because they fail to visit the KIPP schools they criticize.

I have heard these concerns before, but in a previous example at Education Next, I had to raise my own caution [1] that if KIPP advocates will knowingly misrepresent critics to make their case, we may be well advised to be skeptical of the claims that there is now a softer side to KIPP.

Mathews, however, makes a balanced call for confronting whether or not KIPP schools practice what critics challenge and has posed a valid concern about KIPP critics not visiting actual KIPP schools. I want to address these concerns here, and also offer my own recommendations to KIPP advocates in the spirit offered at the end of Matthews’s piece: “KIPP welcomes visitors. Interested critics should stop by and see how it works.”

My concerns with and criticism of KIPP charter schools began in 2009, based on an Education Next article praising the “no excuses” ideologies of the schools (Whitman, 2008). That piece does exactly what I have done in my work criticizing KIPP: it associates KIPP practices with “no excuses” and “grit” narratives and practices, which are reinforced in KIPP’s own words, among its Five Pillars:

KIPP schools relentlessly focus on high student performance on standardized tests and other objective measures. Just as there are no shortcuts, there are no excuses. Students are expected to achieve a level of academic performance that will enable them to succeed at the nation’s best high schools and colleges.

My initial criticism of KIPP charter schools as the recognizable brand among many “no excuses” charter schools spreading across the U.S. (and influencing practices in public and private schools that directly note they are modeling their practices on KIPP) was also reinforced by Whitman’s article that included, for example, American Indian Public Charter School (AIPCS), Oakland, CA, along with a KIPP school in a broader endorsing of “no excuses” charter schools.

Landsberg (2009) captured for me why the “no excuses” model was deeply troubling:

Students, almost all poor, wear uniforms and are subject to disciplinary procedures redolent of military school. One local school district official was horrified to learn that a girl was forced to clean the boys’ restroom as punishment.

So my first request of Mathews and other KIPP advocates is that they offer clarification on the initial associations between KIPP and other “no excuses” charter schools: If that association is flawed, are advocates in part to blame for that association? And if KIPP, either in those early associations or more recently, has distanced themselves from those associations, will KIPP and its advocates please detail that for critics and the public?

Here, then, is my central problem with KIPP advocacy and their criticism of KIPP critics. My criticism of KIPP is one example of my larger rejecting of “no excuses” and “grit” narratives and practices (please see “No Excuses” and the Culture of Shame: Why Metrics Don’t Matter). And I am not in any way concerned about KIPP outcomes because I refuse to fall into the trap of allowing the ends to justify the means. The claims of “miracles” associated with outcomes are discredited, and ultimately, I am not interested in test scores, graduation rates, or college admittance rates decontextualized from how children are being treated in the schools each day.

I appreciate and accept Mathews’s direct evidence from KIPP schools in DC, and accept his evidence of changes and hope that the success is valid and potentially illustrative of education reform more broadly.

But in his brief critique of KIPP critics, Mathews does not offer any evidence about my primary concern about how students are treated [2]—a concern that I do not limit to KIPP but to any and all schools (I reject zero-tolerance policies, for example, and their disproportionate and negative influence on urban schools and students).

A second request is that KIPP leaders and advocates help those of us criticizing KIPP for direct and indirect connections to “no excuses” and “grit” narratives and practices with what I believe are legitimate concerns based on, I admit, secondary evidence [3].

Here, then, are my remaining concerns:

  • When Gary Rubinstein accepted the KIPP invitation to visit actual schools, his responses to that visit and my interpretation of his visits do not change my position. Thus, visiting KIPP isn’t necessarily going to end criticism, but if Rubinstein’s impressions are misleading, I’d like some clarifications.
  • The public narrative around KIPP is based on embracing an authoritarian and highly structured model for high-poverty and minority students. Public schools have failed high-poverty and minority students in terms of disproportionate discipline and academic policies, including expulsion, suspension, failure, and retention; it appears by the evidence that KIPP and other “no excuses” charter schools mirror those failures instead of alleviating them. The school-to-prison pipeline and the school-as-prison dynamic are key elements of the larger mass incarceration era; KIPP’s association with strict discipline, high attrition, and selectivity are problematic for those of us who wish to break those cycles.
  • Public and charter schools are experiencing an increase in segregation of students by race and class; KIPP appears to be a part of that troubling pattern, again not a solution.
  • A powerful reinforcement of both my criticism of charter schools implementing “no excuses” and “grit” narratives and policy as well as my direct criticism of KIPP schools is Sarah Carr’s Hope Against Hope. Like Whitman’s Education Next piece, Carr associates KIPP charter schools in New Orleans with other non-KIPP charter schools. As well, she details how KIPP and the other “no excuses” charters do in fact practice the sort of discipline policies about which I am critical: (1) a strict discipline code that includes SPARK (Carr, p. 11) and SLANT (making and maintaining eye contact, shaking hands, and other highly regimented behavior demands on students), (2) the Bench as as shaming discipline technique (Carr, p. 23), (3) a demanding culture that stresses “no excuses” for teachers and students (Carr, pp. 42-43), focusing almost exclusively on minority students from poverty (and not being implemented in white or affluent schools), and (4) depending so heavily on structure and external rewards that students falter once they enter college and have those elements removed (Carr, p. 188).
  • KIPP, specifically in its relationship with Teach for America (see Waiting for “Superman” and Carr), contributes directly and indirectly to several harmful and inaccurate claims about teaching and education: teaching quality is primarily a function of being demanding and not of experience or expertise (although this appears true only when dealing with high-poverty minority students since white and affluent students tend to have experienced and certified teachers), and public schools are failing because of corrupt unions (although non-union states n the U.S., mostly in the South, are routinely ranked at the bottom of educational quality).

If and when these concerns above are not applicable to KIPP specifically, I extend an invitation to KIPP leaders and advocates to set the record straight, and I offer my support when they do. I will apologize and join in with setting that record straight.

That I haven’t visited KIPP schools doesn’t discount my ability to raise concerns about large and evidence-based examples of how almost all types of schools are mis-serving children. In all of the issues above, I also criticize public education (re-segregating of schools, inequitable discipline policies, inequitable teacher assignment, tracking—just to name a few concerns).

But since much of my direct criticism of KIPP is based on secondary evidence, I must ask: Is Carr misguided in her depiction of KIPP in New Orleans, or is that situation unlike many or even some KIPP schools, such as the ones visited by Mathews in DC?

I think we all need to know and then remain vigilant about painting any set of schools with too wide a brush.

I also want to stress that Mathews’s concern about hyperbole is important because that hyperbole is unlikely to benefit either advocates or critics. KIPP leaders and advocates depend on the “miracle” school myth as much as critics depend on provocative prison [4] or concentration camp comparisons. I suspect both sides would be wise to set such hyperbole aside.

For my role in criticizing KIPP, I want to stress that I am not as concerned about criticizing any specific type of school as much as I am about exposing where we fail and calling for ways to reform all education so that we are more likely to address inequity in the lives and schools of children.

In that quest, I have no right or desire to misrepresent KIPP, but as I have detailed above, I have ample evidence that KIPP schools are one part of a set of large and pervasive problems that are not unique to KIPP.

I plan to take up Mathews on his invitation to visit some KIPP charter schools when I am in DC this fall, and I hope KIPP leaders and advocates will accept my invitations here to address my remaining concerns and not misrepresent my claims (as Education Next allowed) or simply discount my concerns since I have yet to walk the halls of a KIPP school.

References

Landsberg, M. (2009, May 31). Spitting in the eye of mainstream education. Los Angeles Times online. Retrieved 29 June 2009 from http://articles.latimes.com/2009/may/31/local/me-charter31

Whitman, D. (2008, Fall). An appeal to authority. Education Next, 8(4). Retrieved 29 June 2009 from http://educationnext.org/an-appeal-to-authority/

[1] Again, for the record, the Education Next article misrepresented me three times in the opening, and when I contacted a co-author, Bob Maranto, he admitted in an email exchange the article was unfair, promising to correct the piece and never following through on that promise. The ugliest and demonstrably untrue aspect of that piece was the charge “critics fear that disadvantaged parents do not know enough to choose wisely, or else do not have their children’s best interest at heart,” which my body of work directly disputes and about which I directly alerted Maranto that I in no way hold that position.

[2] Mathews notes that his Work Hard. Be Nice. does address discipline and his rebuttal to KIPP critics.

[3] I have direct experience with local public and charter schools that practice “no excuses” ideology and narratives, but openly acknowledge, that since no KIPP schools are in my area, I haven’t visited the KIPP schools in Arkansas or DC (or elsewhere). I don’t believe, still, that disqualifies me from rejecting “no excuses” and “grit” narratives and practices; as well, my criticism of KIPP within that larger concern is only unfair if KIPP isn’t embracing and practicing both. As I discuss above, the evidence I have suggests KIPP is a part of that larger dynamic.

[4] And let’s not ignore that Michel Foucault and Giles Deleuze have offered substantial examinations of why all schools and prisons are comparable.

New Criticism, Close Reading, and Failing Critical Literacy Again

When the Common Core debates drift toward advocacy or critiques of the standards themselves, I have refused, mostly, to engage with that conversation because I believe debating the quality of CC concedes too much. I remain opposed to CC regardless of the quality of the standards because of the following reasons: (1) CC cannot and will not be decoupled from the caustic influence of high-stakes testing, (2) all bureaucratic and mandated standards de-professionalize teaching, (3) accountability/standards/testing as a reform paradigm has failed and nothing about the CC iteration offers a different approach, except that this is called “national,” and (4) there is absolutely nothing in the CC agenda that addresses social or educational inequities such as disproportionate discipline policies, course access, and teacher assignment.

So with due trepidation, I now wade into the few but needed challenges being offered about how CC encourages “close reading” of texts.

First, let me highlight that my primary field of teaching writing offers a powerful and disturbing parallel model of how the accountability/standards/testing movement supplanted and destroyed evidence-based pedagogy.

I have detailed that the rise of best practice in the teaching of writing in the 1970s and 1980s was squelched by the accountability era begun in the 1980s; see Why Are We (Still) Failing Writing Instruction?

As well, Applebee and Langer offer a chilling refrain of best practice in writing wilting under the weight of standards and testing in their Writing Instruction That Works: Proven Methods for Middle and High School Classrooms.

Reading instruction and reading experiences for children, we must acknowledge, will suffer the same negative consequences under CC and the related high-stakes tests because there are no provisions for implementing CC that change how standards and tests are implemented (often each round of standards and tests are simply infused into the current practices) and, in reality, CC approaches to reading are new names for traditional (and flawed) reading practices.

Next, I strongly recommend the following pieces that essentially confront the central problem with CC’s focus on close reading (and as I’ll expand on below, how close reading continues the traditional view of text-based analysis grounded in New Criticism—and thus excluding critical literacy and the powerful contributions of marginalized writers and critics [1]):

Reading Without Understanding — Common Core Versus Abraham Lincoln, Alan Singer

Martin Luther King Jr. and the Common Core: A critical reading of “close reading,” Daniel E. Ferguson

I want here, then, to add just a few more thoughts on why committing to CC and close reading fails against the gains we have made in understanding the complexity of responding to texts in the context of the words on the page, the intent and biography of the writer, the biography of the reader, and the multiple historical contexts that intersect when anyone reads any text.

Let me start with an example.

I began my poetry unit always with “The Red Wheelbarrow” by William Carlos Williams:

so much depends
upon

a red wheel
barrow

glazed with rain
water

beside the white
chickens.

My instructional goals with starting here are many, but in part, this poem was ideal to make a key point about how we respond to text. I would read the poem aloud and then ask students to close their eyes and envision a wheelbarrow. Then I would ask several to describe what they saw.

The exercise highlighted that many students pictured wheelbarrows in various positions. I always shared with students that I always see any wheelbarrow turned up on its front edge, leaning against a tree because my father was adamant that a wheelbarrow must not sit with the body of the wheelbarrow turned so that it can gather water, which leads to rust forming.

This activity allowed us to discuss what readers can say about the text of a piece, distinguish that from their personal responses (the text says nothing of how the wheelbarrow is sitting, but dictates that it is red, for example), and tease out how writer intent, text, and reader affect create the possibility of dozens of credible, although different, interpretations.

From there we began to confront what counts as “right,” as well as who decides what is “right” as an interpretation.

I made certain my students understood how to conduct a New Criticism analysis and stressed that school, teachers, and many testing situations (notably Advanced Placement) honor only such approaches to text.

Next, however, we challenged that dynamic and began exploring how each student’s empowerment and autonomy rested on having a broad set of lens through which to engage with text, through which to unmask power dynamics embedded in authoritative interpretations of text. [2]

This, of course, is the province of critical literacy.

Ironically, if we use a critical reading of CC and calls for close reading, we discover that “close reading” (and the move by David Coleman from writing CC to leading College Board, where AP and SAT tests are spawned) is simply a repackaging of text-only approaches to text embraced by New Criticism (see the history of New Criticism in the ELA classroom in “A Richer, Not a Narrower, Aesthetic”: The Rise of New Criticism in English Journal (English Journal, 101(3), 52-57).

Like the mechanistic and reductive ways in which New Criticism has been implemented in formal schooling in order to control and measure objectively how students respond to text, CC and the focus on close reading are poised to serve efficiency models of high-stakes testing while also failing students who need and deserve the complex and challenging tools afforded with critical literacy.

CC and close reading—if we wade into debates about the quality of the standards—are nothing new, in fact. Advocates of CC are ironically proving why instead of close reading we need critical reading.

Context matters.

[1] See, for example, Literature: The Reader’s Role, Louise M. Rosenblatt (May, 1960), The English Journal, 49(5), 304-310, 315-316.

[2] See how I use a children’s book, Click, Clack, Moo, to introduce students to Marxist and Feminist critical lenses for texts as a contract to text-based analyses: “Click, Clack, Moo”: Why the One Percent Always Wins.

A Critical Truce in the War between Traditionalists and Progressives

Harry Webb has launched A War of Words: “The war is between traditionalists and progressives and it is an old war.”

Yes, this is an old war, and what is most frustrating about this battle for me is that, once again, critical perspectives are left out entirely. So let me offer here a brief critical truce to this war between traditionalists and progressives.

First, Webb’s post highlights some of the essential problems with the war itself.

Since the mid-1900s, progressive educators and progressive pedagogy have been demonized (and usually misrepresented) as key sources of educational failures, but traditional practices have historically dominated and currently dominate what happens in real classrooms daily.

We have ample anecdotal (I have been in education for 31 years) and research-based evidence that even though, as Webb notes, colleges of education and education professors disproportionately claim to be progressive, that once teachers enter the classroom, they tend to shut the door and practice relatively traditional pedagogy—often teaching as they have been taught or defaulting to traditional practices since they are more efficient and more easily managed in the challenging environments of mixed-ability and overcrowded classrooms.

I invite everyone to read Alfie Kohn’s examination of this in Progressive Education: Why It’s Hard to Beat, But Also Hard to Find. Kohn offers not only a solid discussion of how rare progressive practices are, but also details how progressive practices are misrepresented along with what he considers to be genuine progressive pedagogy.

Another problem I have with this war, however, is that I am not a progressive and am not offering here an apology for progressivism.

I am noting that when I wear my history of education hat (I am the Council Historian for NCTE and wrote a biography for my doctoral work), I recognize a demonizing and marginalizing of progressives that is misleading. As a critical educator, I must add, I believe that progressives have failed and do fail in many ways similar to the failures I associate with traditional practices.

I will confess that it is likely we have failed progressivism, but that point is pretty academic.

Along with Kohn’s discussion of progressivism, I also invite you to examine what I believe is an accurate model of what progressivism is by exploring the work of Lou LaBrant, the focus of my educational biography. Her work disproves the stereotypes of progressives as “touchy-feely” educators who have no grounding in empirical evidence. LaBrant practiced classroom-based research and considered herself a scientific teacher throughout her career from 1906 to 1971. She also fiercely defended the progressivism of John Dewey (something, again, that almost no one represents accurately and then almost no one practices—even those education professors who claim to be progressives).

Another problem with the war is that once traditionalists have mischaracterized progressives in order to attack those mischaracterizations and progressives have mischaracterized the traditionalists in order to attack those mischaracterizations, little value comes from the war, and as is typical of wars, we have only collateral damage.

So let me pause on one comment from Webb: “Yet, their argument is weak and not supported by evidence,” he claims about progressives.

I must call a foul here. Education has a century of research, a research base that has been ignored by policymakers and often discredited by those with narrow definitions of what counts a research (action research by teachers doesn’t count, they say, effectively silencing teachers and indirectly the voices of women in their own profession). Thus when Webb proclaims, “There is an imbalance of power here,” there is an unintended irony since that imbalance is exactly what I am highlighting.

Just as one example, Zemelman, Daniels, and Hyde have offered for many years an examination of just what the body of evidence shows regarding effective pedagogy. This work calls into question two claims by Webb: first, it shows there is a robust research base, and second, the practices that are likely most effective are fairly characterized as progressive (the sorts of practices that reflect an accurate use of the term).

However, what is most important to note about Zemelman, Daniels, and Hyde’s work is that what we know about best practice includes that no pedagogy is rejected and no pedagogy is demanded; in other words, best practice is implementing the instructional practices that best meet the needs of the students and match the learning goals.

For example, the evidence on teaching writing since at least the 1930s and 1940s has shown that isolated grammar instruction does not transfer to original student compositions; in the mid-1990s, George Hillocks showed that isolated grammar instruction actually inhibits writing quality. So the most effective way to teach students to write, including the most effective way for students to learn standard grammar, is through actual writing—something most people would call a progressive perspective.

However, that same research base shows that evidence-based (the evidence being found in actual writing samples from students) direct instruction (what many would call a traditional practice) is vital, and that some students (although a minority) can benefit from targeted isolated grammar instruction.

In other words, the research base emphasizes both the effectiveness of pedagogy most would call progressive, but it certainly doesn’t discount that ultimately what works best is what each student needs. As Webb noted, lecturing can be highly effective, and it can be abysmal—but that has more to do with its delivery and appropriateness than to some default judgment on the practice itself.

When traditionalists say that all students must learn standard English, they likely have a point, but their goal often falls apart when they insist on instructional practices that the evidence has shown are ineffective. “I shall prove my pedagogy is king!” is a shallow thing against seeking ways to teach each student effectively and with  compassion and patience.

When progressives say that student must be engaged in authentic activities, they also have a point (although as Webb notes, and I agree, the jargon of education offers no proof that what is claimed is what is taking place), but that goal often falls apart when they fail to recognize that having students participating in a workshop demands a teacher who also provides a great deal of structure and manages purposeful direct instruction as student work reveals the need.

In my experience, traditionalists and progressives tend to become trapped in their pedagogy and fail to see their students or the evidence of their own ineffectiveness.

If you demand all children read The Scarlet Letter, lecture on it brilliantly for two weeks, prepare a detailed study guide, and then have a class score wonderfully on the test at the end of the unit, what have you gained if most of those students never actually read the book and the entire experience taught them to hate reading?

If you invite your students to participate in writing workshop, offer no structure, fail to provide expert feedback, have no process for students to revise and improve their essays, and then bundle a portfolio of all that work with a nice decorated folder cover, what have you gained if that workshop involved more time meandering and decorating, resulting in students writing no better at the end than the beginning? (See LaBrant’s brilliant critique of failed efforts at the project method in ELA classes, a sharp unmasking of failed progressive claims.)

So, where’s the truce? Because a reasonable person could read this so far and say that I have embedded in the discussion a sneaky endorsement of progressivism (do I associate more with progressives than traditionalists? Sure. But I find they fail just as often as traditionalists, and thus, my disappointment with progressives is much more intense).

Here’s my truce.

I bet that someone as thoughtful and purposeful as Harry Webb appears in his blogs is a stellar and effective teacher, despite our differences about pedagogy.

I have seen brilliant traditionalists teachers and lousy self-proclaimed progressives. More than anything, I have seen too many teachers bound to their practices, ignoring their students and the evidence of their ineffectiveness.

Thus, my truce is that the key (the olive branch?) to this war is whether or not a teacher has a critical lens.

Let me end with a couple invitations:

I have posted before a chart that I use to introduce students to the traditionalist v. progressive divide juxtaposed with the often ignored critical alternative; please see it here.

Also consider a longer post in which I explore this dynamic in detail, Education Done To, For, or With Students?

Maybe, as Webb suggests, there is no hope for ending this war, but I would prefer a different approach, one that requires that we all step away from our commitments (as Webb critiques well, our words, labels, and jargon), take an honest assessment of the impact our commitments have on students (because the only real things that matters are if students learn and that we never sacrifice their dignity and humanity in the process), and then begin again, determined to do better the next time.

Peace?

educator, public scholar, poet&writer – academic freedom isn't free