Category Archives: Testing

Education, Teaching, Testing

More on Failing Writing, and Students

16 Comments

Throughout the 1980s and 1990s, I taught English in the rural South Carolina high school I attended as a student. Many of those years, I taught Advanced Placement courses as part of my load (I taught all levels of English and usually sophomores and seniors) and was department chair.

Over the years, I worked hard to create an English department that served our students well. We made bold moves to provide all students in each grade the same literature textbooks (not different texts for different levels, as was the tradition, thus labeling students publicly) and to stop issuing to students grammar texts and vocabulary books (teachers retained classroom sets to use as they chose).

And a significant part of our English classes was the teaching of writing—having students write often and to produce multiple-draft essays. I stressed the need to end isolated grammar instruction (worksheets and textbook exercises) and urged that grammar, mechanics, and usage be addressed directly in the writing process.

Even though the principal was supportive and a former English teacher, at one faculty meeting while the administrators were discussing recent standardized test scores for the school (yes, this test-mania was in full force during the 80s and 90s in SC), the principal prefaced his comments about the English test scores with, “Keep in mind that the English scores may not reflect what we are doing here since we don’t teach grammar.”*

In a nut shell, that sort of mischaracterization and misunderstanding about best practice is at the foundation of my previous post exploring Joan Brunetta’s writing about how standards- and test-based schooling had failed her.

A few comments on the post and a follow up discussion in the comments with Robert Pondiscio—as well as a subsequent post by Pondiscio at Bridging Differences—have prompted me to continue to address not only how we still fail the teaching of writing but also how that failure is a subset of the larger failure of students by traditional approaches to teaching that are teacher-centered and committed to core knowledge.

Revisiting “The Good Student Trap” in the Accountability Era

Adele Scheele has coined the term “the good student trap,” which perfectly captures how schools create a template for what counts as being a good student and then how that template for success fails students once they attend college and step into the real world beyond school. My one caveat to Scheele’s ideas is that especially during the accountability era—a ramping up of traditional practices and norms for education—this trap affects all students, not just the good ones.

And the trap goes something like this, according to Scheele:

Most of us learned as early as junior high that we would pass, even excel if we did the work assigned to us by our teachers. We learned to ask whether the test covered all of chapter five or only a part of it, whether the assigned paper should be ten pages long or thirty, whether “extra credit” was two book reports on two books by the same author or two books written in the same period. Remember?

We were learning the Formula.

• Find out what’s expected.
• Do it.
• Wait for a response.

And it worked. We always made the grade. Here’s what that process means: You took tests and wrote papers, got passing grades, and then were automatically promoted from one year to the next. That is not only in elementary, junior, and senior high school, but even in undergraduate and graduate school. You never had to compete for promotions, write résumés, or rehearse yourself or even know anyone for this promotion. It happened automatically. And we got used to it….

What we were really learning is System Dependency! If you did your work, you’d be taken care of. We experienced it over and over; it’s now written in our mind’s eye. But nothing like this happens outside of school. Still, we remain the same passive good students that we were at ten or fourteen or twenty or even at forty-four. The truth is, once learned, system dependency stays with most of us throughout our careers, hurting us badly. We keep reinforcing the same teacher-student dichotomy until it is ingrained. Then we transfer it to the employers and organizations for whom we’ll work.

This model of traditional schooling includes a teacher who makes almost all the decisions and students who are rewarded for being compliant—and that compliance is identified as “achievement.”

In English classes, a subset of this process is reflected in how we teach, and fail, writing. As I noted in my earlier post, Hillocks and others have noted that traditional commitments to the five-paragraph essay (and cousin template-models of essays) and a return to isolated grammar exercises have resulted from the rise of high-stakes testing of writing. As well, the accountability era has included the central place of rubrics driving what students write, how teachers respond to student writing, and how students revise their essays.

So what is wrong with five-paragraph essays, grammar exercises, and rubrics?

Let’s focus on rubrics to examine why all of these are ways in which we fail writing and students. Alfie Kohn explains:

Mindy Nathan, a Michigan teacher and former school board member told me that she began “resisting the rubric temptation” the day “one particularly uninterested student raised his hand and asked if I was going to give the class a rubric for this assignment.”  She realized that her students, presumably grown accustomed to rubrics in other classrooms, now seemed “unable to function [emphasis added] unless every required item is spelled out for them in a grid and assigned a point value.  Worse than that,” she added, “they do not have confidence in their thinking or writing skills and seem unwilling to really take risks.”

Rubric-based writing and assessment, then, reflect the exact problem I highlighted earlier, one noted by Applebee and Langer: teachers know more today than ever about how to teach writing, but commitments to accountability and testing prevent that awareness from being applied in class; as Kohn explains:

What all this means is that improving the design of rubrics, or inventing our own, won’t solve the problem because the problem is inherent to the very idea of rubrics and the goals they serve.   This is a theme sounded by Maja Wilson in her extraordinary new book, Rethinking Rubrics in Writing Assessment. In boiling “a messy process down to 4-6 rows of nice, neat, organized little boxes,” she argues, assessment is “stripped of the complexity that breathes life into good writing.”  High scores on a list of criteria for excellence in essay writing do not mean that the essay is any good because quality is more than the sum of its rubricized parts.  To think about quality, Wilson argues, “we need to look to the piece of writing itself to suggest its own evaluative criteria” – a truly radical and provocative suggestion.

Wilson also makes the devastating observation that a relatively recent “shift in writing pedagogy has not translated into a shift in writing assessment.”  Teachers are given much more sophisticated and progressive guidance nowadays about how to teach writing but are still told to pigeonhole the results, to quantify what can’t really be quantified.  Thus, the dilemma:  Either our instruction and our assessment remain “out of synch” or the instruction gets worse in order that students’ writing can be easily judged with the help of rubrics.

Once fulfilling the expectations of the rubric becomes the primary if not exclusive goal for the student, we have the SAT writing section and the unintended consequences, as Newkirk explains (English Journal, November 2005) about students writing to prompts and rubrics for high-stakes testing:

George Hillocks Jr. has shown that another persistent problem with these types of prompts concerns evidence—the writer must instantly develop instances or examples to be used for support. In a sample of the released papers from the Texas state assessment, some of this evidence looks, well, manufactured….When I first read this essay, I imagined some free spirit, some rebel, flaunting the ethics of composition and inventing evidence to the point of parody. But when I shared this letter with a teacher from Texas, she assured me that students were coached to invent evidence if they were stuck [emphasis added]. In my most cynical moment, I hadn’t expected that cause. And what is to stop these coached students from doing the same on the SAT writing prompt? Who would know?

As but one example above, “the good student trap” is replicated day after day in the ways in which students are prompted to write and then how teachers respond to and grade that writing. The failure lies in who makes almost all of the decisions, the teacher, and who is rewarded for being mostly compliant, students.

While core knowledge advocates and proponents of rubric-driven assessment tend to misrepresent critical and progressive educators who seek authentic learning experiences for students with charges of “not teaching X” or “So what shall we teach?” (with the implication that core knowledge educators want demanding content but critical and progressive educators don’t), the real question we must confront is not what content we teach and students learn, but who decides and why.

If we return to rubrics, well designed rubrics do everything for students (see Education Done To, For, or With Students? for a full discussion of this failure), everything writers need to do in both college and the real world beyond school.

Rubric-driven writing is asking less of students than authentic writing in a writing workshop.

Traditional core knowledge classrooms are also deciding for students what knowledge matters, and again, asking less of students than challenging students to identify what knowledge matters in order to critique that knowledge as valuable (or not) for each student as well as the larger society. The tension of this debate is about mere knowledge acquisition versus confronting the norms of knowledge in the pursuit of individual autonomy and social justice—making students aware of the power implications of knowledge so that they live their lives with purpose and dignity instead of having life happen to them.

My call is not for ignoring the teaching of grammar, but for confronting the norms of conventional language so that students gain power over language instead of language having power over them. Why do we feel compelled not to end a sentence with a preposition? Where did that claim come from and who benefits from such a convention?

Why does academic writing tend to erase the writer from the writing (“No ‘I’!”) and who benefits from that convention?

You see, critical approaches to teaching go beyond the mere acquisition of knowledge that some authority has deemed worthy (what Freire labels the “banking concept” of teaching). Yes, knowledge matters, but not in the fixed ways core knowledge advocates claim and pursue. Critical approaches to knowledge honor the dignity of human autonomy in children, something that many adults seem at least leery if not fearful of allowing in their classrooms.

Core knowledge, rubrics, templates, prescriptions, and prompts are all tools of control, ways to trap students in the pursuit of compliance. They aren’t challenging (or “rigorous” as advocates like to say), and they aren’t learning.

As Scheele explains:

System dependency is not the only damaging thing we learned in the context of school: We learned our place….

Yet most of us were falsely lulled into a false self labeled “good” by fulfilling the expected curriculum. The alternative was being “bad” by feeling alienated and losing interest or dropping out….

So what’s the problem? The problem is the danger. The danger lies in thinking about life as a test that we’ll pass or fail, one or the other, tested and branded by an Authority. So, we slide into feeling afraid we’ll fail even before we do-if we do. Mostly we don’t even fail; we’re just mortally afraid that we’re going to. We get used to labeling ourselves failures even when we’re not failing. If we don’t do as well as we wish, we don’t get a second chance to improve ourselves, or raise our grades. If we do perform well, we think that we got away with something this time. But wait until next time, we think; then they’ll find out what frauds we are. We let this fear ruin our lives. And it does. When we’re afraid, we lose our curiosity and originality, our spirit and our talent-our life.

Beyond Rigor, Templates, and Compliance

In my position at a small and selective liberal arts university, I now teach mostly good students in my writing-intensive first year seminars. Students are asked to read and discuss Style, a descriptive look at grammar, mechanics, and usage that raises students’ awareness and skepticism about conventional uses of language, but rejects seeing conventions as fixed rules. (We ask why teachers in high school tend to teach students that fragments are incorrect when many published works contain fragments, leading to a discussion of purposeful language use.)

Throughout the course, students are asked to plan and then write four original essays that must be drafted several times with peer and my feedback. The focus, topic, and type of essay must be chosen by the student. To help them in those choices, we discuss what they have been required to do in high school for essays, we explore what different fields expect in college writing, and we read and analyze real-world essays in order to establish the context for the choices, and consequences of those choices, that writers make—specifically when those writers are students.

I offer this here in case you think somehow I am advocating “fluffy thinking” or a “do-your-own-thing philosophy” of teaching, as some have charged. And I invite you to ask my students which they prefer, which is easier—the template, prompt-based writing of high school that created their good student trap or my class. [HINT: Students recognize that five-paragraph essays and rubrics are easier, and they often directly ask me to just tell them what to write and how. As Mindy Nathan noted above, good students are “unable to function [emphasis added] unless every required item is spelled out for them in a grid and assigned a point value.”]

My students reinforce for me every class session that we have failed the teaching of writing and those students by doing everything for them in school. They are nearly intellectually paralyzed with fear about the consequences of their own decisions.

When challenged and supported to be agents of their own learning, their own coming to understand the world, and their own decisions about what knowledge matters and why, however, they are more than capable of the tasks.

And with them in mind, I must ask, who benefits from compliant, fearful students as intellectual zombies, always doing as they are told?

—–

* Although he phrased his comment poorly, my principal was, in fact, making a valid point that a multiple-choice English (grammar) test was unlikely to fairly represent what our students had learned about composing original essays. He intended to make a swipe at the quality of the test, although he did so gracelessly.

Education, education reform, Testing

The New York Times in an Era of Kool-Aid Journalism

2 Comments

With Advertisements for the Common Core, the Editorial Board at The New York Times has offered its special brand of Kool-Aid journalism to the careless claim that 2013 NAEP data somehow prove education reform is a success:

The country is engaged in a fierce debate about two educational reforms that bear directly on the future of its schoolchildren: first, teacher evaluation systems that are taking hold just about everywhere, and, second, the Common Core learning standards that have been adopted by all but a few states and are supposed to move the schools toward a more challenging, writing-intensive curriculum.

Both reforms — or at least the principles behind them — got a welcome boost from reading and math scores released recently by the federal government. …

Two examples are the District of Columbia and Tennessee, among the first to install more ambitious standards and teacher evaluations. Tennessee jumped from 46th in the country in fourth-grade math two years ago to 37th, and from 41st in the nation to 34th in eighth-grade reading. The District of Columbia, though still performing below the national average, has also shown progress. The scores of its students improved significantly in both math and English.

Moreover, according to Education Secretary Arne Duncan, the eight states that managed to get the Common Core standards in place in time for the latest National Assessment of Educational Progress exams this year showed improvement from 2009 scores in either reading or math.

Kool-Aid journalism occurs when journalists relinquish their work as researchers and reporters to political appointees—in this case the Editorial Board of the NYT decides to turn Secretary Duncan’s baseless claims into statements of fact that support an editorial position. The Board concludes:

But the progress seen elsewhere — like Tennessee and the District of Columbia — shows that improvement is possible if the states strengthen their resolve and apply solutions that have been shown to work.

However, if the Editorial Board at the NYT had made even a basic effort at confirming Duncan’s claims, the Board could have discovered that NAEP data are complicated and cannot prove in any way that recent reforms are a success.

As I have detailed, and despite my not having any training as a journalist or as an investigative reporter, the Editorial Board could have benefitted from the following clarifications about NAEP that I found easily—all of which discredit Duncan’s claims and the Board’s position:

When I point out that raw changes in state proficiency rates or NAEP scores are not valid evidence that a policy or set of policies is “working,” I often get the following response: “Oh Matt, we can’t have a randomized trial or peer-reviewed article for everything. We have to make decisions and conclusions based on imperfect information sometimes.”

This statement is obviously true. In this case, however, it’s also a straw man. There’s a huge middle ground between the highest-quality research and the kind of speculation that often drives our education debate. I’m not saying we always need experiments or highly complex analyses to guide policy decisions (though, in general, these are always preferred and sometimes required). The point, rather, is that we shouldn’t draw conclusions based on evidence that doesn’t support those conclusions.

This shows that the places with the greatest gains were D.C., Tennessee, and Indiana, three places that have embraced the corporate reform strategy of testing, closing down schools, and opening charters.  If this was the only data we had access to, it would seem to prove that “the ends justify the means” when it comes to education reform….

There are many other things to analyze, and I’m looking forward to reading how others analyze the data.  For example, it is curious that Louisiana had ‘gains’ that were smaller than the national average despite that state having, certainly, the most aggressive reforms occurring.  For ‘reformers’ who are so obsessed with test scores and test score gains, this is certainly something that shouldn’t be ignored.  Also, Washington and Hawaii were pretty high up on the ‘growth’ numbers even though Washington does not have charter schools and Hawaii has been very slow to adopt Race To The Top reforms so their ‘gains’ can’t be attributed to those.

I’m still pretty confident that in the long run education reform based primarily on putting pressure on teachers and shutting down schools for failing to live up to the PR of charter schools will not be good for kids or for the country, in general.  I hope politicians won’t accept the first ‘gains’ chart without putting it into context with the rest of the data.

  • Latest NAEP Results, by G.F. Brandenburg exposes that DC gains pre-date the reforms championed by Duncan and the NYT:

First of all, the increases in some of the scores in DC (my home town) are a continuation of a trend that has been going on since about 2000. As a result of those increases, DC’s fourth grade math students, while still dead last in the nation, have nearly caught up with MISSISSIPPI, the lowest-scoring state in the US.

You will have to strain your imagination to see any huge differences between the trends pre-Rhee and post-Rhee. (She was installed after testing was over in 2007.)…

So, the Educational DEforms instituted by Rhee, Henderson, and their corporate masters have not produced the promised miracles.

Yesterday gave us the release of the 2013 NAEP results, which of course brings with it a bunch of ridiculous attempts to cast those results as supporting the reform-du-jour. Most specifically yesterday, the big media buzz was around the gains from 2011 to 2013 which were argued to show that Tennessee and Washington DC are huge outliers – modern miracles – and that because these two settings have placed significant emphasis on teacher evaluation policy – that current trends in teacher evaluation policy are working – that tougher evaluations are the answer to improving student outcomes – not money… not class size… none of that other stuff.

I won’t even get into all of the different things that might be picked up in a supposed swing of test scores at the state level over a 2 year period. Whether 2 year swings are substantive and important or not can certainly be debated (not really), but whether policy implementation can yield a shift in state average test scores in a two  year period is perhaps even more suspect….

Is Tennessee’s 2-year growth an anomaly? we’ll have to wait at least another two years to figure that out. Was it caused by teacher evaluation policies? That’s really unlikely, given that those states that are equally and even further above their expectations have approached teacher evaluation in very mixed ways and other states that had taken the reformy lead on teacher policies – Louisiana and Colorado – fall well below expectations.

As it stands, the position taken by the NYT Editorial Board lacks even the barest qualities of credibility, but it does expose the utter failure of Kool-Aid journalism.

reading, Testing

NAEP? Nope: Why (Almost) Everyone Will Misread (Again) Data on Gaps

12 Comments

Let the data orgy begin!

NAEP data have been released and I anticipate almost as much time and money will be wasted on the data as has been wasted on administering the tests, scoring the tests, and creating the handy web link to all that data—notably the predictable link to gaps. [For the record, most of these data charts can be prepared without any child ever taking tests; just use the socioeconomic data on each child and extrapolate.]

Take a moment and scroll through the gray space between myriad groups in both math and reading.

…

There, enjoy it?

While you’re at it, look at the historical gaps between males and females in the SAT.

Males on average outscore females in reading and math (though females outscore males in writing, the one section of the SAT that doesn’t count for anything anywhere, hmmmm).

The problem, of course, is that standardized test data are simply metrics for social conditions that we pretend are measures of learning and teaching.

It is a particularly nasty game, but it seems few are going to stop playing any time soon. “Achievement gap”* has now ascended to the point of being classified as a subset of Tourette syndrome among politicians and education reformers.

The problems with persisting to lament achievement gaps and then address those gaps with new standards and more testing are that the solutions both primarily measure those gaps and contribute to them:

  • Standardized testing remains biased by class, race, and gender.
  • Standardized test scores remain mostly a reflection of any child’s home (from about 60% to as much as 86%).
  • School and classes students take are more often than not a reflection of the community and homes children are born into; thus, school/learning quality is determined by a child’s socioeconomic status, but those schools do not change that status.
  • If affluent children and impoverished children are provided equal learning opportunities (which they are not), the gap cannot close (go back and look at the handy NAEP charts on gaps, by the way).

The short point is something different has to be done in both the lives and schools of children in poverty (as well as racial and language subgroups overrepresented in poverty) if those data-point gaps are ever going to be reduced.

David Berliner (2013) is illustrative of what those differences should entail, using PISA data often instrumental in ranking educational quality of countries:

Let me look at inequality and schooling internationally: Do countries with greater income inequality generally do worse on achievement tests than countries where income inequality and poverty is lower? The answer is yes (Condron, 2011). Larger income disparities within a nation are associated with lower scores on international tests of achievement. For example, on the 2006 mathematics tests of the Program on International Student Achievement, with a mean score near 500, Finland scored above all other nations (548), and substantially beat the United States of America (474). But Finland is a country with low inequality and a very low childhood poverty rate. But suppose that Finland had the same rate of childhood poverty as the United States of America, and the United States of America had the same rate of childhood poverty as Finland. What might the scores of these two nations be like then? If one statistically adjusted each nation’s scores using the poverty rate of the other, then Finland’s score is predicted to be 487, a long way from the top position it had attained. The score for the United States of America would have been 509, quite a bit better than it actually did. Clearly, inequality within a nation matters. If large numbers of youth in a nation are poor, then achievement test scores are likely to be lower. If there were a reduction in the poverty rate of a nations’ youth, achievement scores are likely to go up….

To those who say that poverty will always exist, it is important to remember that many Northern European countries such as Norway and Finland have virtually wiped out childhood poverty. (pp. 205, 208)

Thus, if we are bound and determined to persist in our fetish for test scores and remain committed to raising test scores (instead of actually alleviating inequity or providing all children with wonderful and rich school days that would end in learning and happiness), guess what?

We need to do something different than what we have been doing for thirty-plus years!

First, end the standards-testing rat race.

Second, end childhood poverty.

Reference

David C. Berliner (2013) Inequality, Poverty, and the Socialization of America’s Youth for the Responsibilities of Citizenship, Theory Into Practice, 52:3, 203-209, DOI: 10.1080/00405841.2013.804314

* Please see my series on “achievement gaps”:

Achievement Gap Misnomer for Equity Gap, pt. 1

Achievement Gap Misnomer for Equity Gap, pt. 2

CCSS, Charter schools, Education, education reform, grade retention, No Excuses Reform, Poverty, Segregation, Testing

The U.S. Formula for Children and the Choices We Refuse to Make

3 Comments

The formula for children in the U.S. can be summed up in one word, I think: “harsh.” And the response we should have to this formula is “inexcusable.”

Let’s consider the U.S. formula for children:

If children in the U.S. can survive the gauntlet that is the national formula for children, as young adults they can look forward to crushing debt to attend college so that they can enter a nearly non-existent workforce.

But there is a caveat to this formula: The U.S. formula for children above is for “other people’s children,” that new majority in U.S. public schools and those children living in homes of the working poor, the working class, and the dwindling middle class.

Children of the privileged are exempt.

And what are the choices we refuse to make?

The Joseph Rowntree Foundation (UK) has released “Does money affect children’s outcomes?”—based in part on “many studies…from the US.” The key points include:

  • This review identified 34 studies with strong evidence about whether money affects children’s outcomes. Children in lower-income families have worse cognitive, social-behavioural and health outcomes in part because they are poorer, not just because low income is correlated with other household and parental characteristics.
  • The evidence was strongest for cognitive development and school achievement, followed by social-behavioural development. Income also affects outcomes indirectly impacting on children, including maternal mental health, parenting and home environment.
  • The impact of increases in income on cognitive development appears roughly comparable with that of spending similar amounts on school or early education programmes. Increasing household income could substantially reduce differences in schooling outcomes, while also improving wider aspects of children’s well-being.
  • A given sum of money makes significantly more difference to children in low-income than better-off households (but still helps better-off children).
  • Money in early childhood makes most difference to cognitive outcomes, while in later childhood and adolescence it makes more difference to social and behavioural outcomes.
  • Longer-term poverty affects children’s outcomes more severely than short-term poverty. Although many studies were from the US, the mechanisms through which money appears to affect children’s outcomes, including parental stress, anxiety and material deprivation, are equally relevant in the UK.

The third bullet point should not be ignored: The key to eradicating poverty and the negative consequences of poverty for children is to address poverty directly in the lives of children—money—and to address inequity directly in the education of children.

There is no either/or, then, in the education reform debate. It is imperative that we do both.

Ultimately, the U.S. formula for children is based on flawed assumptions. Before we can change that formula, we must change our views of poverty as well as people and children trapped in poverty.

Scarcity and abundance are powerful forces; in the U.S., both are allowed to exist as an ugly game of chance.

The choice of abundance for all is there to be embraced, however, if compassion and community are genuinely a part of the American character.

CCSS, Testing

Beyond Doing the Wrong Thing the Right Way

2 Comments

My nephew is in elementary school, and my parents drive him to school each morning and arrive at his school an hour or two before school lets out each afternoon. This is a rural community in the South where many family members do the same—surrounding the school well before dismissal and often socializing.

Recently, my mother told me about parents of a child at that school who are refusing to allow their son to be placed in a test-prep class (and removed from his normal class) because of his low score on a MAP test. The parents are adamant that his test grades in class are high 90s, and they see no reason for his being out of that class to prepare for a test. [1]

Over the past year, I have also been a part of or observed two situations with education policy: one involving a compromise about reading legislation linked to retaining 3rd graders and another about language in the state’s science standards.

In both cases, good pedagogy and foundational aspects of the fields have been sacrificed for political expediency.

The situation at my nephew’s school and both of these education policy developments represent for me the central problem with the Common Core and high-stakes testing arguments: We are content to find the right ways to do the wrong things.

For example, a new report Data-Driven Improvement and Accountability by Andy Hargreaves, Henry Braun, and Kathleen Gebhardt for NEPC is excellent work that confronts how accountability has failed as well as how data should be used more effectively.

However, despite the high quality of this report, it doesn’t allow us to move a few steps further back and consider not using the accountability paradigm at all.

While there are certainly some outrageous claims made against the Common Core (the Tea Party railings against Obama and big government that often play loose with facts) and some passionate arguments against CC that are credible but tarnished by that passion (concerns about Gates money and its influence as well as the role of David Coleman), the dominant narratives about CC and the high-stakes testing connected with the new standards are about how critics are focusing on bad implementation, and not flaws in the standards or the tests. From that, the arguments are how to implement CC and the tests right.

And here is where we are failing.

Setting aside the impassioned arguments against CC and more high-stakes testing, a good deal of evidence shows that most of our educational problems have nothing to do with either the presence or quality of standards or tests (see Mathis, 2012, for example).

As well, we have considerable reason to be concerned about accountability based on high-stakes tests—Campbell’s Law and Gerald Bracey’s caution about what is tested is what is taught.

Simply put, there is no right way to implement standards and high-stakes tests in an accountability framework because neither the goals/purposes nor problems of U.S. public education call for that paradigm; schools are not failing due to a lack or poor quality of accountability.

And that leads to the next typical response: All critics do is criticize. Where is your alternative?

Let’s consider that, then.

Is there any value in a cohesive body of knowledge associated with the major disciplines (what we typically call standards)? Yes.

So what is wrong with Common Core? CC is a bureaucratic, top-down mandate. In all fields, there exists a cohesive body of agreed upon knowledge, a set of contemporary debates, and a set of enduring debates. Public school standards fail because they are primarily bureaucratic and essentially partisan political documents.

Building on that essential problem, then, a cohesive body of knowledge identified for a field of study that is a resource for an autonomous teacher—this should be the starting point of education reform.

However, even if we address re-tooling how we view standards, even if we drop high-stakes testing (and we should), and even if we afford teachers the professional autonomy they deserve, schools will still ultimately fail unless we address equity and opportunity both in the lives and in the education of all children.

We now face a tremendous wake-up call since—despite the increasingly influential and pervasive accountability movement in our schools—the majority of students in U.S. public schools in the South and urban schools live in poverty.

That fact itself calls into question our social policy and the likelihood that schools alone can overcome social dynamics.

There are no right ways to do the wrong things. CC, new high-stakes testing, and more accountability are simply the wrong things.

[1] Evidence from the SAT seems to support these parents’ wishes since GPA remains a better predictor of college success than SAT scores. Despite claims to the contrary, teachers’ subjective grading is quite powerful, and more powerful than a so-called objective measure.

education reform, Testing

Medicating ADHD in the Brave New World of High-Stakes Accountability

4 Comments

Miranda: O wonder!
How many goodly creatures are there here!
How beauteous mankind is! O brave new world,
That has such people in’t.

—William Shakespeare, The Tempest, Act V, Scene I, ll. 203–206

—–

Utopias seem much more attainable than one may have previously thought. And we are now faced with a much more frightening thought: how do we prevent their permanent fulfillment?…Utopias are attainable. The way of life points towards them. But perhaps a new century will begin, a century in which intellectuals and the educated class will find means of preventing utopias, and we will return to a non-utopian society, which may be less perfect, but will offer more freedom.

—Nicolas Berdiaeff

My love of science fiction (SF) has its roots firmly in Marvel comic books from the 1970s and the SF novels of Arthur C. Clarke and Niven/Pournelle. When I became acquainted with what teachers called “good” and “real” literature, I was immediately drawn to George Orwell and Aldous Huxley as anointed SF writers.

As an adult, I am the sort of SF reader who treasures Kurt Vonnegut and Margaret Atwood, and if I am pushed, I have to admit I value Orwell more as an essayist than novelist and always enjoyed Huxley’s Brave New World over 1984—believing both works are brilliant, but preferring BNW as a reader.

The opening passages are two foundational quotes behind the message of BNW, the Shakespeare the source of the title and the Berdiaeff a nod to Huxley’s parody of utopian fiction.

In his Foreword to the Perennial Classic edition of BNW, Huxley explains:

But Brave New World is a book about the future and, whatever its artistic or philosophical qualities, a book about the future can interest us only if its prophesies look as though they might conceivably come true….The theme of Brave New World is not the advancement of science as such; it is the advancement of science as it affects human individuals. (p. ix)

Beyond purpose, Huxley continues, speculating about “A really efficient totalitarian state”

would be one in which the all-powerful executive of political bosses and their army of managers control a population of slaves who do not have to be coerced, because they love their servitude. To make them love it is the task assigned, in present-day totalitarian states, to ministries of propaganda, newspaper editors and schoolteachers….The great triumphs of of propaganda have been accomplished, not by doing something, but by refraining from doing. Great is truth, but still greater, from a practical point of view, is silence about the truth. (p. xii)

SF, at its best, however, is not predictive, but cautionary; as Neil Gaiman has reminded us, “The world doesn’t have to be like this. Things can be different.” Thus, Huxley warns:

Round pegs in square holes tend to have dangerous thoughts about the social system and to infect others with their discontents….In conjunction with the freedom to daydream under the influence of dope and movies and the radio, it will help to reconcile his subjects to the servitude which is their fate….Indeed, unless we choose to decentralize and to use applied science [1], not as the end to which human beings are to be made the means, but as the means to producing a race of free individuals, we have only two alternatives to choose from: either a number of national, militarized totalitarianisms…or else one supra-national totalitarian. (pp. xiii, xiv)

And then, Huxley conclude: “You pays your money and you takes you choice” (p. xiv).

The speculative and cautionary possibilities found in SF rarely come to fruition in the real world in the dramatic ways of novels or films (or in the somewhat looney ways political factions rant and rave in public discourse). So it seems likely that we are apt never to listen or to act in ways that we should and could.

Huxley, I think, was in many ways speaking to this—“The Not-So-Hidden Cause Behind the A.D.H.D. Epidemic”:

Between the fall of 2011 and the spring of 2012, people across the United States suddenly found themselves unable to get their hands on A.D.H.D. medication. Low-dose generics were particularly in short supply. There were several factors contributing to the shortage, but the main cause was that supply was suddenly being outpaced by demand.

The number of diagnoses of Attention Deficit Hyperactivity Disorder has ballooned over the past few decades. Before the early 1990s, fewer than 5 percent of school-age kids were thought to have A.D.H.D. Earlier this year, data from the Centers for Disease Control and Prevention showed that 11 percent of children ages 4 to 17 had at some point received the diagnosis — and that doesn’t even include first-time diagnoses in adults. (Full disclosure: I’m one of them.)

That amounts to millions of extra people receiving regular doses of stimulant drugs to keep neurological symptoms in check. For a lot of us, the diagnosis and subsequent treatments — both behavioral and pharmaceutical — have proved helpful. But still: Where did we all come from? Were that many Americans always pathologically hyperactive and unable to focus, and only now are getting the treatment they need?

Probably not. Of the 6.4 million kids who have been given diagnoses of A.D.H.D., a large percentage are unlikely to have any kind of physiological difference that would make them more distractible than the average non-A.D.H.D. kid. It’s also doubtful that biological or environmental changes are making physiological differences more prevalent. Instead, the rapid increase in people with A.D.H.D. probably has more to do with sociological factors — changes in the way we school our children, in the way we interact with doctors and in what we expect from our kids.

As disturbing as this is, the final paragraph of this article may be the most significant:

Today many sociologists and neuroscientists believe that regardless of A.D.H.D.’s biological basis, the explosion in rates of diagnosis is caused by sociological factors — especially ones related to education and the changing expectations we have for kids. During the same 30 years when A.D.H.D. diagnoses increased, American childhood drastically changed. Even at the grade-school level, kids now have more homework, less recess and a lot less unstructured free time to relax and play [emphasis added]. It’s easy to look at that situation and speculate how “A.D.H.D.” might have become a convenient societal catchall for what happens when kids are expected to be miniature adults. High-stakes standardized testing, increased competition for slots in top colleges [emphasis added], a less-and-less accommodating economy for those who don’t get into colleges but can no longer depend on the existence of blue-collar jobs — all of these are expressed through policy changes and cultural expectations, but they may also manifest themselves in more troubling ways — in the rising number of kids whose behavior has become pathologized.

The rise of ADHD diagnoses and medications has run concurrent to the accountability era in education, sharing the same thirty-year history. O brave new world of high-stakes accountability and the ADHD medication needed to make the students love their servitude to the tests…

[1] Paul Boyle, “A U.K. View on the U.S. Attack on Social Sciences,” Science, 341 (August 16, 2013), p. 719.

Education, education reform, Teaching, Testing, VAM

When the Shoe Is on the Other Foot: Lessons for Teachers in Misguided Accountability

2 Comments

If we imagined a pictorial representation of the evolution of education accountability, similar to the standard image we associate with human evolution—

—then we’d have to confront that the accountability era begun in the early 1980s focused first on students, requiring them to pass exit exams (regardless of their having taken and passed all of the required courses for graduation) in order to receive their diplomas.

Next, schools were the target of accountability with the advent and distribution of school report cards.

By the end of the first and beginning of the second decades of the twenty-first century, teachers have found their place at the accountability table, with some suggesting that teachers are now being fed their just desserts. Merit pay linked to student test scores and the more recent flurry of implementing value-added methods (VAM) of teacher evaluation and retention in many ways bring teachers into decades-long predicaments faced by students and schools: the misguided and unfair weight of standardized testing used in dysfunctional and invalid ways.

When I posted about how absurd teacher accountability has become, I expected most on my Twitter feed to recognize the situation in New York as unfair and a harsh warning of the mounting weight of failed accountability:

A Bronx performing arts school’s dance instructor will be judged on students’ English exam scores. Physical education teachers at a transfer school in Brooklyn are going to teach Olympic history lessons to prepare students for the history tests that will help determine their ratings. And teachers in Queens are putting the fate of their evaluations into a final exam that they don’t teach, but yields high pass rates.

The scenarios are not unusual — across [New York City] this year, thousands of teachers will be rated in large part based on test scores of subjects and students that they do not teach.

Rather, the scenarios are examples of how schools have tried to comply with a new teacher evaluation system that must factor student performance into final ratings. They also represent how the original purpose of the evaluations, to differentiate teachers’ effectiveness, has been squeezed by restrictive state laws, limited resources, and a tight timeline for implementation.

“It’s insane to me that 40 percent of my evaluation is going to be based on someone else’s work,” said Jason Zanitsch, a high school drama teacher who will share the same “student growth” score with colleagues in his school this year.

However, the first response I received raised a much different point:

@plthomasEdD if teachers don’t like this then way assign all the group work which is just as bad for the kids? hmmm….

My first response was to note that holding teachers accountable for the work of other teachers and the test scores of students they do not even teach is not truly analogous to having students do group work, and then be graded for that group work.

As @Tim_10_ber and I exchanged tweets, I came to recognize that I was arguing from my idealized position on how best to implement group work (group work must require collaboration—or it is simply students sitting close to each other doing individual work—and any grades assigned to group work must be articulated to reflect participation) and @Tim_10_ber was confronting a position with which I agree—that group work is often implemented and graded carelessly and thus unfairly to students.

It is from that recognition, then, that I want to make an argument about the only potential positive outcome related to the unjustifiable use of merit pay and VAM in teacher evaluation, pay, and retention: teachers need to learn how to teach better now that the shoe is on the other foot. Some ironic lessons teachers should learn from invalid teacher accountability include the following:

  • Testing and grades often do far more educational harm than good; the time has come to consider de-testing and de-grading our teaching. Teacher feedback, student self-assessment, student-created rubrics, and re-imagined assessment situations (such as group assessments) and formats are all better alternatives to tests and grades, if our goal is equitable and effective learning opportunities for students.
  • The central flaw with teacher accountability being linked to student test scores and the standards movement is that teachers have experienced declining autonomy in both their content and pedagogy as well as the high-stakes tests themselves. Accountability without autonomy is tyranny. This lesson translates into how often student learning is reduced to mere compliance. Students being held accountable also must have their autonomy honored; thus, students deserve far more choice in their learning than they have been traditionally allowed.
  • As noted by @Tim_10_ber, teachers must be far more vigilant about designing, assigning, and assessing group work, with a keen eye on autonomy, engagement, and causation/correlation (what are fair associations between each student and the outcomes of the group).

The accountability era has nearly destroyed public education. Little about accountability based on standards and high-stakes testing can be embraced or endorsed.

But oppressive and even capricious mandates tend to be leveled at the least among us first; once those policies trickle up to those in power—in other words, when the shoe is on the other foot—living with inequity, unfair accountability, and unworkable conditions can open our eyes to our own flaws as teachers.

As we continue to fight for our professional autonomy and dignity, taking moral stands of non-cooperation, let’s be sure to bring that fight to our classrooms and honor the autonomy and dignity of all our students as a model for those in power who have yet to see the flaws of their ways through the distorting lens of privilege they wear.

In the words of Henry David Thoreau in “Civil Disobedience”:

If I devote myself to other pursuits and contemplations, I must first see, at least, that I do not pursue them sitting upon another man’s shoulders. I must get off him first, that he may pursue his contemplations too….

If the injustice is part of the necessary friction of the machine of government, let it go, let it go; perchance it will wear smooth — certainly the machine will wear out. If the injustice has a spring, or a pulley, or a rope, or a crank, exclusively for itself, then perhaps you may consider whether the remedy will not be worse than the evil; but if it is of such a nature that it requires you to be the agent of injustice to another, then, I say, break the law. Let your life be a counter friction to stop the machine. What I have to do is to see, at any rate, that I do not lend myself to the wrong which I condemn [emphasis added].

Education, education reform, Poverty, Standards, Testing

An Alternative to Accountability-Based Education Reform

14 Comments

During three decades of accountability based on standards and high-stakes testing at the state level and another decade-plus of federal oversight of that accountability, the overwhelming evidence has exposed accountability as a failed network of policies in education reform.

Education reform in the U.S. now faces a potential watershed moment in which setting aside accountability and embracing a school reform agenda that acknowledges social and educational inequity offer a promise of success that accountability, standards, and high-stakes testing have failed to achieve.

First, education does not exist in a vacuum. Teaching and learning are impacted by out-of-school factors and impact the world beyond the walls of schools; thus, the primary foundation upon which education reform must be built is acknowledging that the U.S. currently has one of the highest childhood poverty rates among nations against which U.S. schools are commonly compared:

Relative child poverty rates

Next, another powerful example of inequity in the U.S. is that upward mobility has stagnated—notably in the top and bottom fifths—and, as Matt Bruenig has explained “you are better off being born rich regardless of whether you go to college than being born poor and getting a college degree”:

The third and final context for understanding an alternative to accountability-based education reform is the rise in the working poor in the U.S. and the increase in part-time work that leaves many working-poor families with adults holding multiple jobs but not having access to health care or retirement benefits.

Education reform must be built on policies that directly address the rising social inequity in the U.S. The essential shift away from accountability, then, must begin with social reform that addresses inequity. Social reform is necessarily the responsibility of state and federal legislation; thus, some of the policy targets addressing social inequity that are likely to impact positively a new vision of school-based reform include the following:

  • Food security: Children in poverty face food insecurity, but also suffer from access to low-quality foods (for example, fast food). Nutrition during pregnancy for women in poverty, early childhood nutrition, and nutrition during school ages are all essential elements for providing children the equity of opportunities that schools could provide.
  • Health care: Children and families in poverty tend to avoid needed preventative health care, and then are forced to seek out the least economically efficient avenues for receiving basic and urgent care, emergency rooms. If public education is to transform society and the lives of children, all children must be guaranteed the health (and nutrition) that children in affluence experience.
  • Stable work with rewarding salaries: Children and families in poverty often experience instability in the work of the parents and their homes since impoverished workers are competing with each other for entry-level and transient jobs. A stable workforce and increasing full-time jobs with benefits provide the basis upon which education can succeed where it has traditionally failed.

Certainly, many other social policies need to be addressed, but the foundational point here is that social inequity currently overwhelms public education in the U.S. A first step to education reform is social reform. As well, the public in the U.S. currently supports seeking greater equity: “The Pew Research Center has found that some 90 percent of Americans believe that the government should do everything it can to ensure equality of opportunity” (NYT February 16, 2013). What is lacking is the political will to make commitments to social equity of opportunity for all in the U.S.

Within the larger commitment to social reform, a new vision of education reform must include a broad commitment to providing an equity of opportunity for all children, and some of the policy changes must include the following:

  • End accountability based on standards and high-stakes testing: A growing body of research has shown that the accountability era has failed: “the absence or presence of rigorous or national standards says nothing about equity, educational quality, or the provision of adequate educational services, there is no reason to expect CCSS or any other standards initiative to be an effective educational reform by itself” (Mathis, 2012). A first and essential step to a new vision of education reform is to end the accountability era by shifting away from focusing on outcomes and toward attending to the conditions of teaching and learning—with an emphasis on equity of opportunity.
  • Implement a small and robust measurement system: As Stephen Krashen and others have argued, the existing National Assessment of Educational Progress (NAEP) assessment system in the U.S. provides a more than adequate foundation upon which the U.S. can develop a systematic and limited process for administering tests to random samples of students in all states and gathering descriptive data on the effectiveness of schools. This new system must be low-stakes and should dramatically reduce the funding committed to testing in the U.S.
  • Scale back and eventually end tracking: The most accurate criticism of U.S. education is that it has historically perpetuated and currently perpetuates social inequity. Tracking remains grounded in data that reflect out-of-school influences and tends to funnel impoverished students into narrow academic settings and affluent children into rich educational experiences.
  • Focus on equitable teacher assignments: The focus on teacher quality within the accountability movement has tended to mislead the public about the importance of teacher quality connected to measurable outcomes while ignoring that impoverished, minority, and special needs students along with English language learners disproportionately are assigned to inexperienced and un-/under-certified teachers. Education reform committed to equity must monitor teacher assignments so that no students experience inequitable access to high-quality, experienced teachers.
  • Decrease bureaucracy of teacher licensing and increase academic quality of education degrees: Another legitimate criticism of traditional education is that teacher licensing has many flaws built into the bureaucracy of attaining a teaching certificate. Certification and accreditation mandates and systems tend to fail educators, and thus students. However, as in other fields, the quality of education degree programs still offer a tremendous promise for preparing teachers well for the teaching profession.
  • Honor school and teacher autonomy: Individual schools and classrooms vary dramatically across the U.S. School autonomy and teacher professionalism are the greatest sources of understanding what populations of students need. The current move toward national standards and tests is inherently a flawed concept since student needs in Orangeburg, SC, are dramatically different than student needs in Seattle, WA.
  • Replace accountability with transparency: High-stakes accountability has not only failed to produce outcomes promised by its advocates, but also has created negative unintended consequences (cheating scandals, for example). A more promising approach to insuring that a public institution provides that public with needed services is to require schools to be transparent: identifying educational needs and providing evidence for practices being implemented to meet those needs.
  • Address wide range of issues impacting equity—funding, class size, technology, facilities: Moving away from accountability and toward equity is a shift in the goals and then standards against which education policy is evaluated. Issues of funding, class size, technology, and facilities must be addressed to assure all children experience an equity of opportunities in every school.
  • Abandon ranking: Education in the U.S. has suffered the negative consequences of ranking for over a century. Ranking nearly always distorts data and typically fails goals of equity. Instead of ranking, education should honor how conditions of learning match clearly identified learning goals.
  • Rethink testing and grades: Tests and grades have been the foundation upon which education in the U.S. rests, but both tend to distort education seeking equity, autonomy, and democracy. Rich feedback that challenges learners and contributes to learning, however, is the lifeblood of learning.
  • Practice patience: Crisis and urgency have characterized the accountability era, and both states have contributed to the failure of accountability. Teaching and learning are complex and unpredictable, requiring political and public patience for reaching the goals that everyone seeks.

The points identified above are not intended to be exhaustive, but the evidence is clear that education reform has been on the wrong path for three decades. Accountability has failed, but that experiment has exposed a wealth of data that should inform a new vision of the need to address social and educational inequity through policies that fulfill the promises driving our democracy and our commitment to universal public education.

For Further Reading

Closing the Opportunity Gap: What America Must Do to Give Every Child an Even Chance, Carter and Welner, eds.

Reign of Error: The Hoax of the Privatization Movement and the Danger to America’s Public Schools, Ravitch (September 17, 2013)

Social Context Reform: A Pedagogy of Equity and Opportunity, Thomas, Porfilio, Gorlewski, and Carr, eds. (under contract, Routledge)

CCSS, Charter schools, Education, education reform, Educational Research, Incentives, KIPP, Media, Merit Pay, miracle schools, NCTQ, Poverty, SAT, School choice, Segregation, Standards, Statistics, Teach for America, Teacher Evaluation, Testing, VAM

What We Know Now (and How It Doesn’t Matter)

52 Comments

Randy Olson’s Flock of Dodos (2006) explores the evolution and Intelligent Design (ID) debate that represents the newest attack on teaching evolution in U.S. public schools. The documentary is engaging, enlightening, and nearly too fair considering Olson admits upfront that he stands with scientists who support evolution as credible science and reject ID as something outside the realm of science.

Olson’s film, however, offers a powerful message that rises above the evolution debate. Particularly in the scenes depicting scientists discussing (during a poker game) why evolution remains a target of political and public interests, the documentary shows that evidence-based expertise often fails against clear and compelling messages (such as “teach the controversy”)—even when those clear and compelling messages are inaccurate.

In other words, ID advocacy has often won in the courts of political and public opinion despite having no credibility within the discipline it claims to inform—evolutionary biology.

With that sobering reality in mind, please identify what XYZ represents in the following statement about “What We Know Now”:

Is there a bottom line to all of this? If there is one, it would appear to be this: Despite media coverage, which has been exceedingly selective and misrepresentative, and despite the anecdotal meanderings of politicians, community members, educators, board members, parents, and students, XYZ have not been effective in achieving the outcomes they were assumed to aid….

This analysis is addressing school uniform policies, conducted by sociologist David L. Brunsma who examined evidence on school uniform effectiveness (did school uniform policies achieve stated goals of those policies) “from a variety of data gathered during eight years of rigorous research into this issue.”

This comprehensive analysis of research from Brunsma replicates the message in Flock of Dodos—political, public, and media messaging continues to trump evidence in the education reform debate. Making that reality more troubling is that a central element of No Child Left Behind was a call to usher in an era of scientifically based education research. As Sasha Zucker notes in a 2004 policy report for Pearson, “A significant aspect of the No Child Left Behind Act of 2001 (NCLB) is the use of the phrase ‘scientifically based research’ well over 100 times throughout the text of the law.”

Brunsma’s conclusion about school uniform policies, I regret to note, is not an outlier in education reform but a typical representation of education reform policy. Let’s consider what we know now about the major education reform agendas currently impacting out schools:

Well into the second decade of the twenty-first century, then, education reform continues a failed tradition of honoring messaging over evidence. Neither the claims made about educational failures, nor the solutions for education reform policy today are supported by large bodies of compelling research.

As the fate of NCLB continues to be debated, the evidence shows not only that NCLB has failed its stated goals, but also that politicians, the media, and the public have failed to embrace the one element of the legislation that held the most promise—scientifically based research—suggesting that dodos may in fact not be extinct.

* Santelices, M. V., & Wilson, M. (2010, Spring). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106-133.; Spelke, E. S. (2005, December). Sex differences in intrinsic aptitude for mathematics and science? American Psychologist, 60(9), 950-958; See page 4 for 2012 SAT data: http://media.collegeboard.com/digitalServices/pdf/research/TotalGroup-2012.pdf

Education, education reform, Florida Miracle, Testing

The Unintended Lessons from Florida: Class Grades, pt. 2

7 Comments

After recognizing the excellent analysis by Michael Vasquez and David Smiley of Florida’s school grades being strongly correlated with out-of-school factors associated with the students*, I have received several important related points from Vasquez about the unintended lessons coming from one of the most often cited reform states in the U.S.

First, Vasquez pointed me to Matthew DiCarlo’s What Florida’s School Grades Measure, And What They Don’t, in which DiCarlo explains:

A while back, I argued that Florida’s school grading system, due mostly to its choice of measures, does a poor job of gauging school performance per se. The short version is that the ratings are, to a degree unsurpassed by most other states’ systems, driven by absolute performance measures (how highly students score), rather than growth (whether students make progress). Since more advantaged students tend to score more highly on tests when they enter the school system, schools are largely being judged not on the quality of instruction they provide, but rather on the characteristics of the students they serve.

New results were released a couple of weeks ago. This was highly anticipated, as the state had made controversial changes to the system, most notably the inclusion of non-native English speakers and special education students, which officials claimed they did to increase standards and expectations. In a limited sense, that’s true – grades were, on average, lower this year. The problem is that the system uses the same measures as before (including a growth component that is largely redundant with proficiency). All that has changed is the students that are included in them. Thus, to whatever degree the system now reflects higher expectations, it is still for outcomes that schools mostly cannot control.

The really important aspect of DiCarlo’s analysis is that Florida’s accountability system has likely caused harmed instead of attaining the lofty goals often associated with accountability policies, as DiCarlo concludes:

In other words, there are many Florida schools with lower-performing students that are actually very effective in accelerating student performance (at least insofar as tests can measure it). This particular ratings system, however, is so heavily driven by absolute performance – how highly students score, rather than how much progress they have made – that it really cannot detect much of this variation.

Closing or reconstituting these schools is misguided policy; their replacements are unlikely to do better and are very likely to do worse. Yet this is what will happen if such decisions are made based on the state’s ratings.

Florida will have to do a lot more than make tweaks to truly improve the high-stakes utility of this system. In the meantime, one can only hope that state and district officials exercise discretion in how it is applied.

Both Vasquez and Smiley’s 2013 analysis and DiCarlo’s 2012 analysis, however, are even more troubling in light of a St. Petersburg Times (Florida) March 21, 1999, article, “A lesson in grading schools,” by Kent Fischer and Geoff Dougherty:

Bush believes the grades, A through F, will make it easier for parents to compare schools and assess how they are doing. A noble goal.

But a St. Petersburg Times analysis indicates his school-grading system may be fundamentally flawed.

It takes no account of the impact poverty has on student achievement, though many studies have proven that children from wealthy families generally outscore children whose parents are poor. So Bush’s grades are more apt to reflect the relative wealth of a school’s student body rather than the competency of its teachers, the newspaper’s analysis shows.

Fourteen years ago, then, Fischer and Dougherty accurately identified the flawed Florida school grading plan, but also acknowledged the key ignored hurdle facing education, poverty:

If there’s one thing that has been firmly established by research, it is the impact social factors have on student achievement. This does not mean poor kids can’t achieve. Many do. But poor children often lead transient lives, may suffer from malnutrition and endure higher rates of abuse and neglect than other children. They also tend not to be exposed to books, music and other cultural influences that help ready young minds for school.

Research – and the Times’ analysis – shows that when large numbers of students are considered, poverty reliably predicts test scores. The Times’ analysis found that depending on which test is given, from 69 to 79 percent of the difference in test scores among schools is explained by poverty.

That seems to ignore several significant demographic factors, like single-parent homes and student mobility. But many of those factors have an extremely high correlation to poverty and thus are effectively included in the analysis.

The bottom line, according to many experts: Any grading system that fails to take poverty into account is flawed.

Ultimately, Fischer and Dougherty offered Florida parents a message still relevant today: “There’s more to a school than good test scores. When trying to gauge school quality, educators suggest parents do some investigating.”

* See below a graphic (click to open and then click again to enlarge) related to Vasquez and Smiley’s article they were unable to include in the online article:

gradeschart copy