Category Archives: Statistics

Education, education reform, Educational Research, NAEP, reading, SAT, Science of Reading, Statistics, Testing

SAT Lessons Never Learned: NAEP Edition

Yesterday, I spent an hour on the phone with the producer of a national news series.

I realized afterward that much of the conversation reminded me of dozens of similar conversations with journalists throughout my 40-year career as an educator because I had to carefully and repeatedly clarify what standardized tests do and mean.

Annually for more than the first half of my career, I had to watch as the US slipped into Education Crisis mode when SAT scores were released.

Throughout the past five decades, I have been strongly anti-testing and anti-grades, but most of my public and scholarly work challenging testing addressed the many problems with the SAT—and notably how the media, public, and politicians misunderstand and misuse SAT data.

See these for example:

Over many years of critically analyzing SAT data as well as the media/public/political responses to the college entrance exam, many key lessons emerged that include the following:

  • Lesson: Populations being tested impact data drawn from tests. The SAT originally served the needs of elite students, often those seeking Ivey League educations. However, over the twentieth century, increasingly many students began taking the SAT for a variety of reasons (scholarships and athletics, for example). The shift in population of students being tested from an elite subset (the upper end of the normal curve) to a more statistically “normal” population necessarily drove the average down (a statistical fact that has nothing to do with school or student quality). While statistically valid, dropping SAT scores because of population shifts created media problems (see below); therefore, the College Board recentered the scoring of the SAT.
  • Lesson: Ranking by test data must account for population differences among students tested. Reporting in the media of average SAT scores for the nation and by states created a misleading narrative about school quality. Part of that messaging was grounded in the SAT reporting average SAT scores by ranking states, and then, media reporting SAT average scores as a valid assessment of state educational quality. The College Board eventually issued a caution: “Educators, the media and others should…not rank or rate teachers, educational institutions, districts or states solely on the basis of aggregate scores derived from tests that are intended primarily as a measure of individual students.” However, the media continued to rank states using SAT average scores. SAT data has always been strongly correlated with parental income, parental level of education, and characteristics of students such as gender and race. But a significant driver of average SAT scores also included rates of participation among states. See for example a comparison I did among SC, NC, and MS (the latter having a higher poverty rate and higher average SAT because of a much lower participation rate, including mostly elite students):
  • Lesson: Conclusions drawn from test data must acknowledge purpose of test being used (see Gerald Bracey). The SAT has one very narrow purpose—predicting first-year college grades; and the SAT has primarily one use—a data point for college admission based on its sole purpose. However, historically, media/public/political responses to the SAT have used the data to evaluate state educational quality and the longitudinal progress of US students in general. In short, SAT data has been routinely misused because most people misunderstand its purpose.

Recently, the significance of the SAT has declined, students taking the ACT at a higher rate and more colleges going test-optional, but the nation has shifted to panicking over NAEP data instead.

The rise in significance of NAEP includes the focus on “proficiency” included in NCLB mandates (which required all states to have 100% student proficiency by 2014).

The problem now is that media/public/political responses to NAEP mimic the exact mistakes during the hyper-focus on the SAT.

NAEP, like the SAT, then, needs a moment of reckoning also.

Instead of helping public and political messaging about education and education reform, NAEP has perpetuated the very worst stories about educational crisis. That is in part because there is no standard for “proficiency” and because NAEP was designed to provide a check against state assessments that could set cut scores and levels of achievement as they wanted:

Since states have different content standards and use different tests and different methods for setting cut scores, obviously the meaning of proficient varies among the states. Under NCLB, states are free to set their own standards for proficiency, which is one reason why AYP school failure rates vary so widely across the states. It’s a lot harder for students to achieve proficiency in a state that has set that standard at a high level than it is in a state that has set it lower. Indeed, even if students in two schools in two different states have exactly the same achievement, one school could find itself on a failed-AYP list simply because it is located in the state whose standard for proficient is higher than the other state’s….

Under NCLB all states must administer NAEP every other year in reading and mathematics in grades 4 and 8, starting in 2003. The idea is to use NAEP as a “check” on states’ assessment results under NCLB or as a benchmark for judging states’ definitions of proficient. If, for example, a state reports a very high percentage of proficient students on its state math test but its performance on math NAEP reveals a low percentage of proficient students, the inference would be that this state has set a relatively easy standard for math proficiency and is trying to “game” NCLB.

What’s Proficient?: The No Child Left Behind Act and the Many Meanings of Proficiency

In other words, NAEP was designed as a federal oversight of state assessments and not an evaluation tool to standardize “proficient” or to support education reform, instruction, or learning.

As a result, NAEP, as the SAT/ACT has done for years, feeds a constant education crisis cycle that also fuels concurrent cycles of education reform and education legislation that has become increasingly authoritarian (mandating specific practices and programs as well as banning practices and programs).

With the lessons from the SAT above, then, NAEP reform should include the following:

  • Standardizing “proficient” and shifting from grade-level to age-level metrics.
  • Ending state rankings and comparisons based on NAEP average scores.
  • Changing testing population of students by age level instead of grade level (addressing impact of grade retention, which is a form of state’s “gaming the system” that NAEP sought to correct). NAEP testing should include children in an annual band of birth months/years regardless of grade level.
  • Providing better explanations and guidance for reporting and understanding NAEP scores in the context of longitudinal data.
  • Developing a collaborative relationship between federal and state education departments and among state education departments.

While I remain a strong skeptic of the value of standardized testing, and I recognize that we over-test students in the US, I urge NAEP reform and that we have a NAEP reckoning for the sake of students, teachers, and public education.

Recommended

Literacy and NAEP Proficient, Tom Loveless

The NAEP proficiency myth, Tom Loveless

Education, education reform, Educational Research, grade retention, reading, Science of Reading, Statistics, Testing

Even More Problems with Grade-Level Proficiency

I have explained often about the essential flaw with grade-level proficiency, notably the third-grade reading myth.

Grade level in reading is a calculation that serves textbook companies and testing, but fulfills almost no genuine purpose in the real world; it is a technocratic cog in the efficiency machine.

Now that we are squarely in the newest reading war, the “science of reading,” two other aspects of grade-level proficiency have been central to that movement—the hyper-focus on third-grade reading proficiency that includes high-stakes elements such as grade retention and the misinformation rhetoric that claims 65% of students are not reading at grade-level (the NAEP proficiency myth).

These alone are enough to set aside or at least be skeptical about rhetoric, practice, and policy grounded in grade-level proficiency, but there is even more to consider.

A Twitter thread examines grade-level achievement aggregated by month of birth:

The thread builds off a blog post: Age-Related Expectations? by James Pembroke.

The most fascinating aspect of this analysis thread is the series of charts provided:

As the analysis shows, student achievement is strongly correlated with birth month, which calls into question how well standardized testing serves high-stakes practices and how often standardized testing reflects something other than actual learning.

Being older in your assigned grade level is not an aspect of merit, and being older in your assigned grade seems to have measured achievement benefits that aren’t essentially unfair to younger members of a grade.

Further, this sort of analysis helps contribute to concerns raised about grade retention, which necessarily removes students most likely to score low on testing and reintroduces those students as older than their peers in the assigned grade, which would seem to insure their test data corrupts both sets of measurements.

This data above are from the UK, but a similar analysis by month/year of birth applied to retained students and their younger peers would be a powerful contribution to understanding how grade retention likely inflates test data while continuing to be harmful to the students retained (and not actually raising achievement).

There appears to be even more problems with grade-level proficiency than noted previously, and now, even more reason not to continue to use the rhetoric or the metric.

Education, reading, Standards, Statistics

The Big Lie about the “Science of Reading”: NAEP 2019 Edition

After the release of the 2017 NAEP reading scores, states such as Mississippi launched a campaign to celebrate the success of their reading legislation. This effort coincided with a recent explosion in states adopting reading legislation driven by dyslexia advocates who promote systematic intensive phonics for all students.

The claims coming from Mississippi didn’t seem credible, so I began what turned into a very long (and maybe endless) examination of the growing power of dyslexia advocates to drive what are essentially very bad forms of reading legislation, notably third grade retention and systematic intensive phonics for all students.

In my initial analysis of 2017 NAEP reading scores for 4th and 8th grades, I addressed the use of “the science of reading” as veneer for ideological advocacy; I also focused on the misuse by dyslexia/phonics advocates and the media of the National Reading panel and flawed claims about and definitions of “balanced literacy” and “whole language,” including mostly ahistorical understandings of how reading has been taught and discussed in political and public forums.

With the release of 2019 NAEP data, as we should expect, the same folk are back at over-reacting and misunderstanding standardized reading test data (mostly mainstream media), and dyslexia/phonics advocates are cherry picking evidence to reinforce their ideological advocacy.

All in all, these responses to NAEP data are lazy, and incredibly harmful.

Broadly, responses by the media and advocates have been overly simplistic, and lacking even a modicum of effort to tease out in a scientific way (ironic, eh?) mere correlations from actual causal associations among student demographics, reading policy, reading programs, the fidelity of implementing policy/programs, NAEP testing quality (how valid a proxy is NAEP reading tests for critical reading ability?), etc.

In a Twitter thread, I attempt to make a case against rushing to judgment based on 2019 NAEP reading data:

A little NAEP thread:

In 2017 MS made overstated claims about their NAEP reading scores, hiding the fact that 4th grade bumps disappeared by 8th grade and that NAEP scores remain mostly correlated with poverty; see:

The Big Lie about the “Science of Reading” (Updated)

2019 NAEP reading scores are likely to be a reboot of that for MS since 4th grade reading is an outlier among states in terms of gains but MS remains about average in 4th grade.

But very low still in 8th.

Only fair things to say about new round of NAEP reading scores:

• The US has never had a period over the last 100 years when we said “reading scores are where they should be.”
• There is always a claim of “reading crisis.”
• This is irrespective of how reading is taught.
• NAEP scores, like all standardized test scores, are mostly (60% +) correlated to out-of-school factors.
• NAEP scores only marginally about student achievement/reading, teacher/teaching quality, reading program effectiveness.
• NAEP scores are very pale proxies of reading

Recent rounds of NAEP reading scores, however, are revealing how really bad reading policies (grade retention, intensive systematic phonics for all) can in the short term raise scores while likely deeply harming reading and readers. 4th-grade reading score bumps are mirages.

Equity gap between rich/poor reflected in NAEP reading scores amplifies the reality in the US that the rich get richer while the poor get poorer. Wealth = high achievement; poverty = low achievement. Student outcomes are a consequence of social negligence not student ability.

Placed in recent context of 2017 NAEP reading data and a wider recognition that student demographics (race, socioeconomic status) are historically and currently the greatest causal factors in student standardized test scores, the most fair argument to make in the wake of NAEP 2019 is that the matrix of reading policies and whether or not the policies are implemented at all or well (elements that we do not have data to support in any fashion) cannot be identified as success or failure. We may, however, be able to suggest that focusing on policy, standards, programs, and high-stakes testing simply does not change measurable reading outcomes in positive ways.

If you are fair and careful with the data I am including below, the correlations among all of the factors do not paint any clear picture at all about the effectiveness of programs or policy (again, even if we assume those programs and policies are being implemented at all or well).

Anyone using this data to claim “grade retention works” or “systematic intensive phonics works” is simply being deeply dishonest because no one has done any of the necessary work to tease out those claims in a scientific way (random sampling, controlling for non-instructional factors, investigating fidelity to policies and programs, etc.).

In other words, those advocating for the “science of reading” are making no effort to be scientific themselves in the pursuit of proving if their claims are valid, or not.

None the less, here are the updated data in a manageable chart:

NAEP R 2019 1

NAEP R 2019 2b

NAEP R 2019 3NAEP R 2019 4NAEP R 2019 5NAEP R 2019 6NAEP R 2019 7NAEP R 2019 8

If we genuinely believe a few points here or there, comparing entirely different populations of students under ever-shifting conditions both in their lives and in their education, are in fact not just statistically significant but significant, then we have a wealth of evidence above to suggest that all the standards, testing, and policies are actually degrading student reading achievement.

Finally, I want to stress, the greatest problem exposed by how the media and dyslexia/phonics advocates are responding to NAEP 2019 is that reading is too often a political and ideological football, and students in real classrooms and real lives are being reduced to petty games.

Again, at no point over the past 100 years have the crisis and failure arguments about reading achievement been any different than at this exact moment—regardless of how students have been taught to read (including peak years of intensive phonics and jumbled claims of implementing whole language).

How states mandate and implement reading instruction as well as relentlessly test it in the worst possible formats is a tale of too many cooks in the kitchen, with most of them having no credibility.

See Also

Third-Grade Reading Legislation

retention legislation UPDATED

Third Grade Reading Policies (2012)

From Bruce Baker

 

Education, education reform, Gun Control, Liberal Media, Media, politics, Progressivism, School Security, social justice, socialism, Statistics, Trumplandia

Dare the School Build a New Social Order?: A Reckoning 86 Years Later

1 Comment

The candidacy seemed at the time nothing more than sideshow, perverse reality TV, and then Donald Trump secured the Republican nomination for president, prompting many pundits to note that as a death knoll for the Republican Party.

Yet, Trump was elected president.

During the primaries and throughout his run against Hillary Clinton, Trump proved to be relentlessly dishonest, a liar. However, mainstream media avoided calling a lie “a lie,” including major media outlets directly arguing against such language. President Trump hasn’t budged from overstatement, misleading statements, and outright lies.

Notably, major media publish Trump’s lies as if they are credible, despite fact-checking exposing lie upon lie upon lie.

Early on, many opposing Trump called for media simply to call out the lies. Here is the truly bad news, however.

During my Tuesday role as caregiver for my 2-year-old grandson, I flipped through my cable channels during his nap for a brief reprieve from NickJr. I paused on CNN, even though I loath all of the 24-hour news shows.

What caught my ear was that the newscaster was repeatedly calling Trump our for lies, using the word “lie”—over and over. This, I felt, was a real new normal I had called for, but never expected.

Next, the newscaster replayed a segment from the day before focusing on a fact checker of Trump’s many, many lies. The fact checker noted a truly disturbing fact: Trump’s supporters, he explained, recognize that Trump lies, but doesn’t mind the lies; in fact, Trump’s supporters revel in those lies because, as the fact checker emphasized, this drives liberals crazy.

It is here that I must stress two points: (1) It appears those of us believing that exposing Trump as a liar would somehow derail his presidency were sorely mistaken, and (2) we are now entering a phase of U.S. history in which the long-standing slur of “liberal” is code for taking evidence-based stances, especially if those evidence-based stances swim against the current of American ideology and mythology.

Let me offer a couple example.

In my own public and scholarly work, contexts that prompt responses that discount me as a “liberal” (with false implications that I am a partisan Democrat), I have made repeated and compelling cases against corporal punishment and school-only safety measures.

Neither of these issues is both-sides debates since the research base is overwhelmingly one-sided.

Corporal punishment is not an effective discipline technique, and it creates violent youth and adults. A powerful body research prompted by the school shooting at Columbine and including studies by the Secret Service reject school-only safety measure such as security guards, surveillance cameras, active-shooter drills, and metal detectors, all of which are not deterrents and may even create violence.

Therefore, to embrace evidence-based positions on corporal punishment and school safety is the liberal or progressive (seeking change) stance, while the traditional or conservative (maintaining established practices) positions (ignoring the evidence) cling to corporal punishment and fortifying schools while refusing to address the wider influences of communities and our national mania for guns.

Let’s consider that last point more fully next.

There is an unpopular and upsetting fact driving why school-only safety measures are futile: K-12 and higher education are essentially conservative.

Despite political and popular scapegoating of all formal education as liberal, the evidence of nearly a century reveals that all forms of school more often than not reflect the communities and society they serve. In no real ways, then, do schools meet the former Secretary of Education Arne Duncan’s hollow mantra that education is the great equalizer, some sort of silver bullet for change.

Evidence shows that at different levels of educational attainment, significant gaps persist among racial categories and those gaps are even more pronounced once race and gender are included (see p. 34).

In the 1930s, a golden era for idealism about communism and socialism in the U.S. after the stock market crash, major educational thinkers such as John Dewey (a socialist) and George Counts championed the potential for progressive education (Dewey) to shape U.S. democracy, and then for social reconstruction (Counts) to reshape the nation, as Counts detailed in his Dare the School Build a New Social Order? (1932).

As an early critical voice, Counts spoke to the educational goals that appealed to me as I eventually found critical pedagogy in my doctoral program and doubled down on my early commitment to be the sort of educator who fostered change with and through my students.

Yet, here I sit in 2018, 86 years after Counts’s manifesto. And the U.S. is being led by a pathological liar supported by more and more people who directly say they don’t care about lies or evidence because it makes liberal mad.

This is the pettiness our country has wrought, despite more people today being formally educated than at any time in U.S. history.

My 35 years and counting as an educator, part as a high school teacher and now in higher education, have been a disappointing lesson that answers Counts’s titular question with a resounding “no.”

I shared with my foundations education class the proofs of a chapter I have prepared for a volume now in-press, Contending with Gun Violence in the English Language Classroom. I then briefly reviewed the evidence against in-school safety measures, prompting a student to ask what, then, should we do in schools.

Address our larger gun culture and violent communities, I explained, reminding the class that I have stressed again and again that they need to understand at least one essential lesson from our course: Schools mostly reflect communities and society, but they simply do very little to change anything.

I don’t like this message, but it is evidence-based, and I suppose, a liberal claim.

For many years, I have quickly refuted those who assume I am a partisan Democrat (I am not, never have been). I also have rejected labels of “liberal” and “progressive” for “critical” and “radical.”

But I feel the time is ripe for re-appropriating “liberal” when it is hurled as a slur.

In Trumplandia, to be fact-free is to be conservative, traditional, and to acknowledge evidence is to be liberal, progressive.

This is what the evidence reveals to those of us willing to see. Everything else is a lie.

There’s both sides for those who want it.

Recommended

College campuses are far from radical

Education, education reform, Standards, Statistics, Testing

Chicken-Little Politics and the Curse of Testing (and Standards) in South Carolina

I entered education as a high school teacher in South Carolina in the 1984-1985 academic year, the first year of a significant teacher pay raise and a pivotal ground zero in the state’s accountability era established in late 1970s legislation.

Over about four decades, SC has revised or changed educational standards six or seven times and implemented about the same number of different state and national tests.

And what hath this curse of testing and standards wrought for SC?

South Carolina students bomb the ACT, falling behind Mississippi, announces an article by Paul Bowers explaining:

South Carolina’s graduating class of 2018 came close to dead-last in the nation on the ACT college readiness test, painting a grim picture of a state that has languished near the bottom of education rankings for decades.

This year’s graduates placed 50th among the states and Washington, D.C., on the ACT, according to composite scores based on the test’s English, Reading, Math and Science sections.

Only Nevada’s students did worse.

The chicken-little politics of accountability has been fulfilled in ways that assure politicians, the public, and the media will declare schools, teachers, and students a failure. Yet again, and again, ad nauseam.

Let’s try something different here, ways to interpret better this data from the ACT.

The first key point about these scores is that SC is experiencing bureaucratic insanity—doing the same thing over and over while expecting different results.

The problem with eduction in SC has little to do with test scores, which overwhelmingly reflect what the problem is: Poverty and inequities grounded in that poverty as well as racism.

In fact, this response to the article exposes how misguided the entire process is:

While political and popular gazes remain fixed on test scores and standards (curriculum), we have failed to acknowledge that the quality—or even presence—of standards (and the concurrent curriculum) have no clear impact on measurable student outcomes.

The accountability era has not worked in SC, and it never will.

Ever-new standards and ever-new tests are simply rearranging deck chairs on the Titanic.

Here, then, are a couple more ways we can and should respond to the ACT scores.

Why is SC requiring all students to take a test to measure college readiness when a much smaller percentage of those students plan to enter college? And why this rush to prepare all students for college when it remains unaffordable for many, if not most, SC residents?

What assures that test scores on the ACT are mostly about teaching and learning, instead of poverty, racism, or even student effort (in other words, what assurance do we have that students have taken this test seriously)?

And finally, a significant failure of the chicken-little politics of test scores in SC is the misguided urge to rank (see the problems here).

What if we consider that SC is in the bottom quartile of states by poverty, and then, what if we concede that standardized tests are at least 60% and possibly over 80% linked to out-of-school factors (not any quality of schools, standards, or teaching) such as poverty and affluence? SC should be near or at the bottom of any rankings because of the state’s abysmal record of class and rank inequity as well as a very long history of underfunding and ignoring public education—especially in the most vulnerable communities.

This most recent sky-is-falling media report is our own hellish Groundhog Day experience; this article has been written dozens of times over the past four decades, and it can be recycled dozens of more times in the future.

Unlike the befuddled Phil (Bill Murray) in the movie, we actually can bring this nightmare to a stop.

If we have the political and public will, the media will be able to give this dark fairy tale a rest.

Education, Grades, Statistics, Testing

If You Are Grading, You May Not Be Teaching

1 Comment

Throughout my career of about two decades as a high school English teacher and then approaching another two decades in higher education (as a teacher educator and first-year/upper-level writing professor), I have avoided and delayed grading as well as eliminated testing from my classes.

My experiences with first-year and upper-level writing instruction have further confirmed that if you are grading, you may not be teaching.

Specifically, teaching citation and scholarly writing has revealed a problem that directly exposes why grading often works against our instructional goals.

First, let me stress again that the essential problems with grading include how traditional practices (such as assigning grades that are averaged for quarter and/or semester grades that are then averaged for course grades) tend to blur the distinction between summative and formative grades, inhibiting often the important role of feedback and student revision of assignments.

The blurring of formative and summative grades that occurs in averaging, as I have confronted often, deforms teaching and learning because students are being held accountable during the learning process (and thus discouraged from taking risks).

To briefly review the problems with grades and averaging, let me offer again what my major professor argued: Doctors do not take a patient’s temperature readings over a four-day stay in the hospital in order to average them, but does consider the trajectory of those readings, drawing a final diagnosis on the last reading (or readings). Thus, averaging is a statistical move that distorts student growth, deforms the value of reaching a state of greater understanding.

As I have detailed before, consider a series of grades: 10, 10, 85, 85, 85, 85, 85, 85, 100, 100 = 730.

The average is 73, which most teachers would assign, but the mode is 85, and if we note these grades are sequential and cumulative (10 as the first grade in terms of time, and 100 the last grade), a legitimate grade assignment could be 100.

In other words, using the same data, a teacher could assign 73, 85, or 100 to this student, and all can be justified statistically.

But another problem with grades and averaging that speaks to this post is something my students taught me when they complained about their math classes. Several students informed me that they had never passed a single math test, but had passed math courses.

The trick? Students earned bonus points for homework, etc., that were added to each test, on which students never reached a passing score.

This process means that cumulatively students never acquired so-called basic or essential math skills, but passed the courses, resulting in course credit that grossly misrepresented student learning.

Therefore, returning to my claim that grading may not be teaching, when we subtract for so-called errors to assign grades, we are allowing students to move through the learning process without actually learning the element being graded. In most cases, I believe, that strategy is teaching students that the element really doesn’t matter.

This dynamic is particularly corrosive when teaching scholarly writing and citation. Citation is one area of writing that doesn’t have degrees; you either cite fully or you don’t.

Many students, similar to the math students noted above, have never reached any level of proficiency with citing because they have mostly had points deducted for improper citation and then gone on their merry way, never having learned to cite fully.

If citation is essential, to grade and never require mastery of citation have two very negative consequences: (1) students do not attain an essential skill (and may exit formal education without the skill), and (2) students fail to understand the importance of drafting, receiving feedback, and revising.

Academic writing is challenging for developing young writers since it demands complex technical demands (such as citation and document formatting) and high expectations for content and style. Students need years and dozen of experiences reading and writing academic writing across multiple disciplines and varying conventional expectations.

But we cannot expect students to acquire the nuances of citation if we simply grade and never allow or expect them to cite fully and properly as an essential aspect of an academic writing experience.

As I make this case, I want to stress that as writing teachers we are trying to balance expectations for students and provide them low-stakes opportunities to draft with little or no consequences.

Students should have both writing assignments that demand minimum proficiency with key skills such as citation and writing contexts that foster and allow taking risks and working outside conventions.

Grading, I witness daily, inhibits both of those in ways that suggest the non-graded writing class is the best opportunity for students to learn in holistic and authentic ways that reveal themselves in student writing samples.

Because of their experiences being graded, I struggle to help students see that citation, grammar, mechanics, style, and content all work in unison either to support or erode their authority as writers and scholars.

I struggle to break through students resisting the drafting, feedback, revision process because they have been taught to submit instantly perfect work; that their identifiable flaws are the loss of points—not necessary areas to learn, grow, and excel.

As I end my thirty-fourth year teaching, I cannot stress hard enough that if you are grading, you may not be teaching, and your students likely are not learning the very things you value enough to assess.

CCSS, Coleman, College Board, comic books, Critical Pedagogy, diversity, Education, education reform, Educational Research, Equity, fascism, inequity, Martin Luther King Jr., Paulo Freire, politics, Poverty, race, racism, Statistics, Testing, VAM

The Politics of Education Policy: Even More Beware the Technocrats

Man Prefers Comic Books That Don’t Insert Politics Into Stories About Government-Engineered Agents Of War (The Onion) includes a simple picture of a 31-year-old white male with the hint of a soon-to-be Van Dyke:

The fictional “man,” Jeremy Land, explains:

“I’m tired of simply trying to enjoy escapist stories in which people are tortured and experimented upon at black sites run by authoritarian governments, only to have the creators cram political messages down my throat,” said Land, 31, who added that Marvel’s recent additions of female, LGBTQ, and racially diverse characters to long-running story arcs about tyrannical regimes turning social outsiders into powerful killing machines felt like PC propaganda run amok. “Look, I get that politics is some people’s thing, but I just want to read good stories about people whose position outside society makes them easy prey for tests run by amoral government scientists—without a heavy-handed allegory for the Tuskegee Study thrown in. Why can’t comics be like they used to and just present worlds where superheroes and villains, who were clearly avatars for the values of capitalism, communism, or fascism, battle each other in narratives that explicitly mirrored the complex geopolitical dynamics of the Cold War?”

The satire here is the whitesplaining/mansplaining inherent in the politics of calling for no politics.

It strains the imagination only slightly to understand how this commentary on comic book fanboys also parallels the persistent combination in education of calling for no politics while using policy and a narrow definition of data and evidence to mask the racial and gender politics of formal schooling.

Let’s imagine, then, instead of the fictional Land an image of David Coleman (who parlayed his Common Core boondoggle into a cushy tenure as the head of the College Board) or John Hattie (he of the “poverty and class size do not matter” cults that provide Hattie with a gravy train as guru-consultant).

A close reading of David Coleman’s mug shot reveals a whole lot of smug.

In his “visible learning” hustle, John Hattie likely prefers to keep his enormous profits invisible.

Coleman and Hattie as technocrats feed the systemic racism, classism, and sexism in formal education policy and practice by striking and perpetuating an objective pose that serves as a veneer for the normalized politics of political and economic elites in the U.S.

As Daniel E. Ferguson examines, Coleman’s Common Core propaganda, the rebranded traditional mis-use of New Criticism into “close reading,” argues:

Close reading, as it appears in the Common Core, requires readers to emphasize “what lies within the four corners of the text” and de-emphasize their own perspective, background, and biases in order to uncover the author’s meaning in the text.

However, Ferguson adds,

Critical reading, in contrast, concerns itself with those very differences between what does and does not appear in the text. Critical reading includes close reading; critical reading is close reading of both what lies within and outside of the text. For Paulo Freire, critical reading means that “reading the world always precedes reading the word, and reading the word implies continually reading the world.”

And thus, close reading serves the cult of efficiency found in the high-stakes standardized testing industry that depends on the allure of believing all texts have singular meanings that can be assessed in multiple-choice formats—a dymanic Ferguson unmasks: “The story beyond the four corners of Coleman’s video is one of a man whose agenda is served by teachers following a curriculum that requires students to read in a way assessable through standardized tests he oversees and profits from.”

Simultaneously, of course, keeping students and teachers laser-focused on text only detracts them from the richer context of Martin Luther King Jr. and the broader implications of racism and classism informed by and informing King’s radical agenda.

Simply stated, close reading is a political agenda embedded in the discourse of objectivity that whitewashes King and denies voice and agency to King, teachers, and students.

Concurrently, Hattie’s catch phrase, “visible learning,” serves the same political agenda: Nothing matters unless we can observe and quantify it (of course, conveniently omitting that this act itself determines what is allowed to be seen—not the impact of poverty or the consequences of inequity, of course).

Hattie’s garbled research and data [1] match the recent efforts in education reform to isolate student learning as the value added (VAM) by individual teachers, yet another off-spring of the cult of efficiency manifested in high-stakes standardized testing.

Just as many have debunked the soundness of Hattie’s data and statistics, the VAM experiment has almost entirely failed to produce the outcomes it promised (see the school choice movement, the charter school movement, the standards movement, etc.).

Coleman and Hattie work to control what counts and what matters—the ultimate in politics—and thus are welcomed resources for those benefitting from inequity and wishing to keep everyone’s gaze on anything except that inequity.

The misogyny and racism among comic book fanboys allows the sort of political ignorance reflected in The Onion‘s satire.  If we remain “within the four corners of the text” of Marvel’s Captain America, for example, we are ignoring that, as I have examined, “Captain America has always been a fascist. … But … Captain America has always been our fascist, and that is all that matters.”

3e281
Captain America: Steve Rogers #1 (c) Marvel

The politics of education policy seeks to point the accusatory finger at other people’s politics, and that politics of policy is served by the technocrats, such as Coleman and Hattie, who feed and are fed by the lie of objectivity, the lie of no politics.

[1] See the following reviews and critiques of Hattie’s work:

Education, Equity, inequity, Media, Poverty, Public Intellectual, race, racism, Statistics, Trumplandia

14 June 2017 Reader

How to Call B.S. on Big Data: A Practical Guide, Michelle Nijhuis

Mind the Bullshit Asymmetry Principle, articulated by the Italian software developer Alberto Brandolini in 2013: the amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it. Or, as Jonathan Swift put it in 1710, “Falsehood flies, and truth comes limping after it.”Plus ça change.

Who Is Dangerous, and Who Dies?

ERROL MORRIS: I found an innocent man who came very close to being executed. [Adams’s execution was scheduled for May 8, 1979, but Supreme Court Justice Lewis F. Powell Jr. ordered a stay only three days before he was to be strapped into the lethal-injection gurney. Ultimately, the court overturned his death sentence, but not his conviction.] I uncovered all of these appalling details 30 years ago and then opened up a newspaper recently and read about Buck. It’s as if nothing ever happened. That’s both depressing and infuriating. Mitt Romney, when he was governor of Massachusetts, was told that the death penalty is problematic because it’s fallible. You could execute an innocent person, and given our current state of knowledge, there is really no way to bring them back. Once executed, they stay executed.

CHRISTINA SWARNS: And so what was Romney’s reply?

ERROL MORRIS: He said: Oh, that’s simple. We’ll just make it infallible. We’ll make it foolproof. You said it’s fallible. We’ll just fix that.

Stop Pretending You’re Not Rich, Richard V. Reeves

So imagine my horror at discovering that the United States is more calcified by class than Britain, especially toward the top. The big difference is that most of the people on the highest rung in America are in denial about their privilege. The American myth of meritocracy allows them to attribute their position to their brilliance and diligence, rather than to luck or a rigged system. At least posh people in England have the decency to feel guilty.

In Britain, it is politically impossible to be prime minister and send your children to the equivalent of a private high school. Even Old Etonian David Cameron couldn’t do it. In the United States, the most liberal politician can pay for a lavish education in the private sector. Some of my most progressive friends send their children to $30,000-a-year high schools. The surprise is not that they do it. It is that they do it without so much as a murmur of moral disquiet.

Beneath a veneer of classlessness, the American class reproduction machine operates with ruthless efficiency. In particular, the upper middle class is solidifying. This favored fifth at the top of the income distribution, with an average annual household income of $200,000, has been separating from the 80 percent below. Collectively, this top fifth has seen a $4 trillion-plus increase in pretax income since 1979, compared to just over $3 trillion for everyone else. Some of those gains went to the top 1 percent. But most went to the 19 percent just beneath them.

50 years after the Loving verdict, a photo essay looks back on their love, Priscilla Frank

Monday, June 12, marks the 50th anniversary of the landmark United States Supreme Court decision Loving v. Virginia, which quashed anti-miscegenation laws in 16 states around the nation, ushering restrictions against interracial marriage to the wrong side of history.

The date is now remembered as Loving Day in honor of Richard and Mildred Loving, the couple who defied the state’s ability to dictate the terms of their love based on their skin color. Mildred, who was of African American and Native American descent, and Richard, who was white, wed in 1958 in Washington D.C., because interracial marriage was illegal in their native rural Virginia, as well as 15 other Southern U.S. states.

When the Lovings returned to Virginia, however, local police raided their home one early morning after being tipped off by another resident. They declared the Lovings’ marriage license invalid within the scope of the state, placing the couple under arrest.

What counts as language education policy?: Developing a materialist Anti-racist approach to language activism, Nelson Flores and Sofia Chaparro

Abstract: Language activism has been at the core of language education policy since its emergence as a scholarly field in the 1960s under the leadership of Joshua Fishman. In this article, we seek to build on this tradition to envision a new approach to language activism for the twenty-first century. In particular, we advocate a materialist anti-racist approach to language activism that broadens what counts as language education policy to include a focus on the broader racial and economic policies that impact the lives of language-minoritized communities. In order to illustrate the need for a materialist anti-racist framing of language education policy we provide portraits of four schools in the School District of Philadelphia that offer dual language bilingual education programs. We demonstrate the ways that larger societal inequities hinder these programs from serving the socially transformative function that advocates for these programs aspire toward. We end by calling for a new paradigm of language education policy that connects language activism with other movements that seek to address societal inequities caused by a myriad of factors including poverty, racism, and xenophobia.

The difficulties scholars have writing for a broad audience, Christopher Schaberg and Ian Bogost

Scholars have insights, experience and research that can help the public navigate the contemporary world, but scholarly work all too often goes unseen. Sometimes it gets sequestered behind exorbitant paywalls or prohibitively steep book prices. Other times it gets lost in the pages of esoteric journals. Other times yet, it’s easy to access but hard to understand due to jargon and doublespeak. And often it doesn’t reach a substantial audience, dooming its aspirations to impact public life.

How can scholars write for wider audiences without compromising their lives as disciplinary researchers?

The Confederate flag largely disappeared after the Civil War. The fight against civil rights brought it back, Logan Strother, Thomas Ogorzalek, and Spencer Piston

But what is less well-known is the actual history of these symbols after the Civil War — and this history sheds important light on the debate. Confederate symbols have not always been a part of American or Southern life. They largely disappeared after the Civil War. And when they reappeared, it was not because of a newfound appreciation of Southern history.

Instead, as we argue in a newly published article, white Southerners reintroduced these symbols as a means of resisting the Civil Rights movement. The desire to maintain whites’ dominant position in the racial hierarchy of the United States was at the root of the rediscovery of Confederate symbols.

Pride or Prejudice: Racial Prejudice, Southern Heritage, and White Support for the Confederate Battle Flag, Logan Strother, Spencer Piston, and Thomas Ogorzalek

Abstract: Debates about the meaning of Southern symbols such as the Confederate battle emblem are sweeping the nation. These debates typically revolve around the question of whether such symbols represent “heritage or hatred:” racially innocuous Southern pride or White prejudice against B lacks. In order to assess these competing claims, we first examine the historical reintroduction of the Confederate flag in the Deep South in the 1950s and 1960s; next, we analyze three survey datasets, including one nationally representative dataset and two probability samples of White Georgians and White South Carolinians, in order to build and assess a stronger theoretical account of the racial motivations underlying such symbols than currently exists. While our findings yield strong support for the hypothesis that prejudice against Blacks bolsters White support for Southern symbols, support for the Southern heritage hypothesis is decidedly mixed. Despite widespread denials that Southern symbols reflect racism, racial prejudice is strongly associated with support for such symbols.
College Board, Education, education reform, Educational Research, Gilles Deleuze, literacy, Michel Foucault, NCLB, Poverty, Standards, Statistics, Teaching, Testing, Writing

Reformed to Death: Discipline and Control Eclipse Education

An enduring gift of being a student and a teacher is that these experiences often create lifelong and powerful personal and professional relationships. Reminiscing about these experiences, however, is often bittersweet because we are simultaneously reminded of the great promise of education as well as how too often we are completely failing that promise.

After writing about my two years as as a co-lead instructor for a local Writing Project summer institute, the former student I discussed called me, and we found ourselves wading deeply into the bittersweet.

She has in the intervening years been a co-facilitator in the same workshop where I taught her now more than 15 years ago; she also has worked in many capacities providing teachers professional development and serving as a mentor to pre-service teachers completing education programs and certification requirements.

As we talked, the pattern that emerged is extremely disturbing: the most authentic and enriching opportunities for teachers are routinely crowded out by bureaucratic and administrative mandates, often those that are far less valid as instructional practice.

In my chapter on de-grading the writing classroom, I outlined how the imposition of accountability ran roughshod over the rise of the National Writing Project (NWP), which embodied both the best of how to teach writing and a gold standard approach to professional development.

What is best for teachers and what is best for students, however, are mostly irrelevant in the ongoing high-stakes accountability approach to education reform, a process in which discipline and control eclipse education.

Local sites of the NWP are crucibles of how the reform movement is a death spiral for authentic and high-quality teaching and learning as well as teacher professionalism.

At the core of the NWP model is a charge that teachers must experience and become expert in that which they teach; therefore, to guide students through a writing workshop experience, teachers participate in extended summer writing workshop institutes.

While NWP site-based institutes and other programs thrived against the weight of the accountability era, that appears to be waning under the weight of accountability-based mandates that are in a constant state of reform; teachers are routinely required to seek new certification while they and their students must adapt to a perpetually different set of standards and high-stakes tests.

That bureaucracy is often Orwellian since “best practice” and “evidence-based”—terminology birthed in authentic contexts such as the NWP—have become markers for programs and practices that are aligned with standards and testing, not with the research base of the field. The logic is cripplingly circular and disturbingly misleading.

This erosion and erasing of teaching writing well and effectively is paralleled all across the disciplines in K-12 education, in fact—although how writing is particularly ruined in standards- and testing-based programs and practices remains our best marker of accountability as discipline and control, not as education.

I want to end here by staying with writing, but shifting to the sacred cow of the reform movement: evidence.

High-stakes testing of writing has been a part of state accountability and national testing (NAEP and, briefly, the SAT) for more than 30 years since A Nation at Risk ushered in (deceptively) the accountability era of K-12 public education in the U.S.

What do we know about high-stakes testing as well as the accountability paradigm driven by standards and tests?

George Hillocks has documented [1] that high-stakes testing of writing reduces instruction to training students to conform to anchor papers, template writing, and prescriptive rubrics. In other words, as I noted above, “best practice” and “evidence-based” became whether or not teaching and learning about writing conformed to the way students were tested—not if students had become in any way authentic or autonomous writers, and thinkers.

My own analysis of NAEP tests of writing [2] details that standardized data touted as measuring writing proficiency are strongly skewed by student reading abilities and significant problems with the alignment of the assessment’s prompts and scoring guides.

And now, we have yet more proof that education reform is fundamentally flawed, as Jill Barshay reports:

“(T)he use of the computer may have widened the writing achievement gap,” concluded the working paper, “Performance of fourth-grade students in the 2012 NAEP computer-based writing pilot assessment.”  If so, that has big implications as test makers, with the support of the Department of Education, move forward with their goal of moving almost all students to computerized assessments, which are more efficient and cheaper to grade.

Not only does high-stakes testing of writing fail the research base on how best to teach composition [3], but also the pursuit of efficiency [4] continues to drive all aspects of teaching and learning, effectively contradicting the central claims of reformers to be pursuing seemingly lofty goals such as closing the achievement gap.

Writing instruction and assessment are prisoners of the cult of proficiency that is K-12 education reform, and are just one example of the larger accountability machine that has chosen discipline and control over education.

Reform has become both the means and the ends to keeping students and teachers always “starting again,” “never [to be] finished with anything,” as Gilles Deleuze observed [5].

Barshay ends her coverage of the IES study on computer-based writing assessment with a haunting fear about how evidence drives practice in a high-stakes accountability environment, a fear I guarantee will inevitably become reality:

My fear is that some educators will respond by drilling poor kids in the QWERTY keyboard, when the time would be better spent reading great works of literature and writing essays and creative stories.

As long as reforming and accountability are the masters, we will continue to make the wrong instructional decisions, we will continue to be compelled to make the wrong decisions.

[1] See Hillocks’s “FightingBack: Assessing theAssessments” and The Testing Trap: How State Writing Assessments Control Learning.

[2] See 21st Century Literacy: If We Are Scripted, Are We Literate?, co-authored with Renita Schmidt.

[3] See The Impact of the SAT and ACT Timed Writing Tests – NCTE.

[4] See NCTE Position Statement on Machine Scoring.

[5] See Gilles Deleuze, Postscript on the Societies of Control:

The administrations in charge never cease announcing supposedly necessary reforms: to reform schools, to reform industries, hospitals, the armed forces, prisons….In the disciplinary societies one was always starting again (from school to barracks, from barracks to the factory), while in the societies of control one is never finished with anything—the corporation, the educational system, the armed services being metastable states coexisting in one and the same modulation, like a universal system of deformation….In the disciplinary societies one was always starting again (from school to the barracks, from the barracks to the factory), while in the societies of control one is never finished with anything.

Education, education reform, Education Week, Media, Statistics, Testing

Don’t Count on Grading, Ranking Educational Quality

2 Comments

Having been a long-time advocate for and practitioner of de-testing and de-grading the classroom, I also reject the relentless obsession of mainstream media to grade and rank educational quality among states as well as internationally (see Bracey and Kohn).

As Kohn recognizes: “Beliefs that are debatable or even patently false may be repeated so often that at some point they come to be accepted as fact.”

And thus, with the monotonous regularity and mechanical lack of imagination of a dripping faucet, Education Week once again trumpets Quality Counts.

Like a college course no one wants to register for, Quality Counts 2017 gives the nation a C while no state makes an A or an F.

The appeal of all this much ado about nothing includes:

  • The U.S. has a perverse obsession with quantification that is contradicted by a people who are equally resistant to science and expertise.
  • People love the overly simplistic use of charts and interactive maps.
  • These grades and rankings always confirm the enduring narrative that public schools are failing.

However, the real problem is not how states and the nation rank, but that we persist at the grading and ranking as if that process reveals something of importance (it doesn’t) or as if that process somehow is curative (it isn’t).

How, then, does grading and ranking educational quality fail us?

  • As with regularly changing standards and high-stakes testing as part of accountability, grading and ranking educational quality is part of the larger failure of imagination, a belief in doing the same thing over and over while expecting different results. Media have been grading and ranking for decades, and the narrative of failing schools has continued; in other words, this process has no positive impact on education reform—but it feeds a media and social need to bash public schooling.
  • Anything can be quantified and ranked, and the statistics needed to quantify and rank are necessarily what drive both; thus, A-F grades and then extending the measurements so that ranking is possible become goals of the process that often distort the message of that process. For a simple analogy, in the 400-meter dash at the Olympics, the event creates finishers ranked 1-10; however, all of them are world-class and the distinction among them is minuscule, for all practical purposes irrelevant except for the need to declare winners and losers.
  • Grades and rankings of all kinds in education focus almost entirely on observable and measurable outcomes, glossing over or ignoring powerful influences on measurable student outcomes. Decades of research show that out-of-school factors account for 60-80+% of those measurable outcomes; and thus, outcome-based data of educational quality are more likely a reflection of social conditions than school-based quality. The inherent problem with using test scores, for example, for ranking and determining educational quality has been disputed by the College Board for years (see page 13).
  • Grades and rankings feed into a competition model as well as deficit ideology. These are both harmful in education because collaboration is more effective than competition and because our focus is on flaws (deficits) that we associate primarily with schools, teachers, and students, perpetuating a “blame the victim” mentality that ignores (as noted above) factors beyond the control of schools, teachers, and students (such as poverty, racism, sexism, etc.,—all of which significantly impact measurable learning outcomes).
  • And finally, grading and ranking fail because of a common misunderstanding about statistical facts as they contradict political and public expectations: large populations of humans (90% of students attend public schools) will always have a range of measurable outcomes (height, 40-yard dash times, test scores)—although also misunderstood, think the bell-shaped curve—which will appear to be a “failure” when posed against the political/public call for 100% proficiency by students. In other words, the U.S. demands that everyone be above average and then is disappointed when statistics show a range of human outcomes.

Since the mid-1800s, fueled by the Catholic church’s market fears, there has existed a media, political, and public obsession with bashing public education.

In this era of fake news and post-truth debate, as I have noted over and over, mainstream media are as culpable—if not exactly the same—as fake news and click-bait because practices such as Quality Counts by EdWeek are lazy and misleading, enduring, as Kohn noted, mostly because it is something media have always done and because these rankings feed into confirmation bias.

If quality counts, beating the grades-and-rankings drums is a sure way to insure that it will never be obtained.

If truth matters, a first step in that direction would include resisting the failed practice of grading and ranking educational quality.