Education, Educational Research, Grit, growth mindset, reading, science, Science of Reading

Understanding “Science” as Not Simple, Not Settled: Meta-Analysis Edition

December 8, 2022 plthomasedd

A powerful but often harmful relationship exists among research/science, mainstream media, and public policy.

One current example of that dynamic is the “science of reading” (SOR) movement that is driving reading legislation and policy in more than 30 states (see HERE and HERE).

Mauren Aukerman, who has posted two of three planned posts on media coverage of SOR (HERE and HERE), identifies in that second post a key failure of media: Error of Insufficient Understanding 3: Spurious Claims that One Approach is Settled Science.

For example, Aukerman details with citations to high-quality research/science: “In short, there is insufficient evidence to conclude that any single approach, including the particular systematic phonics approach often elided with ‘the science of reading,’ is most effective.” And therefore, Aukerman recommends: “Be skeptical of ‘science of reading’ news that touts ‘settled science,’ especially if such claims are used to silence disagreement.“

What makes media a dangerous mechanism for translating research/science into policy is that journalists routinely oversimplify and misrepresent research/science as “settled” when, in reality, most research/science is an ongoing conversation with data that presents varying degrees of certainty about whatever questions that research/science explores.

In education, research/science seeks to identify what instruction leads best to student learning—such as in the reading debate.

The other problem with media serving as a mechanism between research/science and policy is that journalists are often trapped in presentism and either perpetuate or are victims of fadism.

Despite no settled research/science supporting media’s coverage of the current reading “crisis,” the initial “science of reading” narrative created by Emily Hanford has now become standard media narratives without any effort to check for validity (again, I highly recommend Aukerman’s first post).

Regretfully, education (and students, teacher, parents, and society) is regularly the victim of fadism at the expense of research/science. The list of recent edu-fads that were promoted uncritically by media only to gradually lose momentum because, frankly, they simply never were valid policies is quite long: charter schools (notably no-excuses models), value-added methods for evaluating/paying teachers, school choice, Common Core, etc.

Two fads that represent well how the misuse of “science” helps this failed cycle in education are “grit” and growth mindset. Both gained their introduction to mainstream education because media portrayed the concepts are research/science-based (even justified, as “grit” was, by the Genius grant).

While schools fell all over themselves, uncritically, to embrace and implement “grit” and growth mindset, the research community gradually revealed that both concepts have some important research and ideological problems. Scholars have produced research/science that complicates claims about “grit” and growth mindset, and many critical scholars continue to call for interrogating the racist/classist groundings of both concepts.

Growth mindset has been in the news again (and discussed on social media) because two recent meta-analyses reach different conclusions; see this Twitter thread for details:

Does growth mindset work? Join me for a tale of two meta-analyses. @chrisjb1
— Beth Tipton (@stats_tipton) November 7, 2022

Tipton and co-authors, in fact, have published an analysis and commentary on this problem: Why Meta-Analyses of Growth Mindset and Other Interventions Should Follow Best Practices for Examining Heterogeneity.

The issue raised about meta-analyses parallels the exact problem with media coverage of research/science—scientific methodologies that fail due to oversimplification. See this Tweet, for example, about meta-analyses:

Perhaps it is wise to remember what the founder of meta-analysis said about computing average treatment effects for education interventions: pic.twitter.com/LLhTyW39PG
— Terry Pearson (@TPLTD) November 9, 2022

Especially in education, when individual student needs greatly impact what is “best” for teaching and learning in any given moment, Tipton’s final Tweet cannot be over-emphasized:

And please, can we stop with the “either/or”, “good/bad”, “yes/no” thinking about interventions? I’m tired of writing commentaries.
— Beth Tipton (@stats_tipton) November 7, 2022

The use of “science” in research is necessarily limiting (see HERE) when that “science” is restricted to experimental/quasi-experimental designs seeking proof of cause (does instructional approach X cause students to learn better than instructional approach Y).

While causal conclusions and research methods that address populations and controls are the Gold Standard for high-quality research/science, this type of “science” is often less valuable for the practical day-to-day messiness of teaching and learning.

Educators are better served when research/science is used to inform practice, not to mandate one-size-fits-all practice (see HERE).

The media and journalists more often than not turn research/science into oversimplified truisms that then are used as baseball bats to beat policy advocates into submission. The conversation and nuance are sacrificed along with effective policy.

The public and policymakers are left with a challenge, a way to be critical and careful when either the media or researchers present research/science.

As Aukerman warns, if journalists or researchers start down the “simple, settled” path, then they are likely not credible (or they have an agenda) because the real story is far more complicated.

Whose Voice Matters?: Reading Teacher Edition

December 7, 2022 plthomasedd

Here is a teacher voice that resonates with me in my work as a literacy teacher and scholar:

A brief consideration will indicate reasons for the considerable gap between the research currently available and the utilization of that research in school programs and methods.

It is not strange, in view of the extensive literature on language, that the teacher tends to fall back upon the textbook as authority, unmindful of the fact that the writer of the text may himself be ignorant of the basis for his study. …

This is not the time for the teacher of any language to follow the line of least resistance, to teach without the fullest possible knowledge of the implications of his medium. Before we…experiment with methods of doing specific things or block out a curriculum, let us spend some time with the best scholars in the various fields of language study to discover what they know, what they believe uncertain and in need of study. Let us go to the best sources, and study the answers thoughtfully.

This voice in many ways parallels a dominant narrative in the media about teaching reading in the US:

From how much of the media tells it, a war rages in the field of early literacy instruction. The story is frequently some version of a conflict narrative relying on the following problematic suppositions:

a) science has proved that there is just one way of teaching reading effectively to all kids – using a systematic, highly structured approach to teaching phonics;

b) most teachers rely instead on an approach called balanced literacy, spurred on by shoddy teacher education programs;

c) therefore, teachers incorporate very little phonics and encourage kids to guess at words;

d) balanced literacy and teacher education are thus at fault for large numbers of children not learning to read well.

The opening teacher voice is from Lou LaBrant, published in 1947 in the journal that would become Language Arts (NCTE). LaBrant was a classroom literacy teacher and teacher educator over a 65-year career.

The second passage is by Maren Aukerman, a scholarly analysis of the current media coverage of the “science of reading” (SOR).

Although 77 years apart, these voices and claims about teaching reading seem to suggest that concerns about reading achievement have been similar for many, many decades, and thus, placing blame for current reading achievement on specific aspects of teacher preparation and teacher practice today seems if not baseless at least misguided.

For a couple years now, I have also heard from dozens and dozens of teachers, some of whom send me real-time DMs documenting teacher PD they are receiving in SOR; these teachers identify misinformation in that PD as well as misunderstanding about reading and teaching reading by their administrators.

Often these are LETRS training sessions, or similar programs designed to emphasize phonics for teachers of reading.

Many teachers have contacted me about being reprimanded and threatened for simply asking questions about the training or pointing out the misinformation.

What is important to stress here is that these teachers—often veteran teachers who have a high level of expertise—do not have a podcast or a Facebook page amplifying their stories.

To be blunt, in today’s SOR climate, these reading teachers’ voices do not matter.

But other points need to be stressed also.

First, is the media narrative that teacher education does not include SOR or phonics instruction true?

No, and in fact, there is research that teacher education is grounded in science, but there also is an absence of research on these exact concerns (again, identical to LaBrant’s reference to the “gap” above):

It is clear that the repeated critiques of literacy teacher preparation expressed by the SOR community do not employ the same standards for scientific research that they claimed as the basis for their critiques. However, to dismiss these critiques as unimportant would ignore the reality of consequences, both current and foreseen, for literacy teacher preparation….

In contrast to the claims made by the SOR community, research in literacy teacher preparation has been extensive, scientific, and useful for guiding reform efforts….

Despite the political and media attention given to the SOR and the tools on which the SOR community relies, there is no body of evidence that reflects the SOR perspective on literacy teacher preparation by members of the literacy teacher preparation research community.
Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

Next, and really important, if we are demanding scientific research inform practice, currently the research base on LETRS simply does not show that the training improves teaching or learning to read by students:

A growing number of U.S. states have funded and encourage and/or require teachers to attend professional development using Moats’s commercial LETRS program, including Alabama, Arkansas, Kansas, Mississippi, Missouri, Oklahoma, Rhode Island, and Texas. This is despite the fact that an Institute of Education Sciences study of the LETRS intervention found almost no effects on teachers or student achievement (Garet et al., 2008).
Hoffman, J.V., Hikida, M., & Sailors, M. (2020). Contesting science that silences: Amplifying equity, agency, and design research in literacy teacher preparation. Reading Research Quarterly, 55(S1), S255–S266. Retrieved July 26, 2022, from https://doi.org/10.1002/rrq.353

We are confronted, then, with an increasingly harmful pattern: The media continues to make unsupported claims about reading failures by teachers, teacher educators, and students, leading to parental and political responses that have resulted in very harmful policies and practices (see HERE, HERE, and HERE).

And these questions remain:

Should we reform how reading is taught in the US? Of course.
Should we reform teacher education, notably focusing on literacy instruction? Absolutely.

However, the current SOR movement is doing the same thing once again while expecting different results.

And the SOR movement isn’t playing by the rules advocates and policy makers are demanding for everyone else.

If we are disqualifying reading programs popular throughout the US because of personal and corporate profit (a key claim in one podcast episode), then we must hold people and programs being implemented in their place to the same standards.

Media and journalists are making money off a false narrative, shouting “science” while using cherry-picked anecdotes to sell their story.

Corporations and program designers are making huge amounts of money off PD that isn’t supported by science at all. My home state of SC, like many states, just allocated $15 million for LETRS training.

If we are disqualifying anything that isn’t “scientific” (notably the exact same call as was codified in No Child Left Behind in 2001), then we cannot use anecdotes (especially cherry-picked anecdotes) to demand reform and cast blame.

Yet, the media continue to drive a narrative by including only voices that make that story seem true.

We could easily fill a podcast episode with teachers who have suffered through really flawed LETRS training, and at least several episodes of teachers who have had their professional autonomy stripped from them because of administrators holding them accountable for implementing a program and not teaching students (which is exactly what is happening with structured literacy as a replacement for so-called failed programs):

We recognize that some teachers using structured literacy approaches will find ways to respond to the interests, experiences, and literacy abilities of individual students; however, we are concerned about the indiscrim- inate and unwarranted implementation of the following practices:

• Directive and/or scripted lessons that tell teachers what to say and do and the implementation of les- son sequences, often at a predetermined pace (Hanford, 2018)

• Privileging of phonemic awareness and phonics as primary decoding skills (Hanford, 2018, 2019; IDA, 2019; Paige, 2020; Pierson, n.d.; Spear-Swerling, 2019)

• Use of decodable texts that do not engage multiple dimensions of reading (Hanford, 2018; IDA, 2019; Paige, 2020; Spear-Swerling, 2019)

• Specialized forms of reading instruction designed for particular groups of students as core literacy instruction for all students and teacher educators (Hanford, 2018; Hurford et al., 2016; IDA, 2019; Pierson, n.d.)

• Mandating structured literacy programs despite the lack of clear empirical evidence to support these programs

• Privileging the interest of publishers and private education providers over students.
Compton-Lilly, C.F., Mitra, A., Guay, M., & Spence, L.K. (2020). A confluence of complexity: Intersections among reading theory, neuroscience, and observations of young readers. Reading Research Quarterly, 55(S1), S185-S195. https://doi.org/10.1002/rrq.348

Throughout my forty-year career as a K-12 teacher and a university professor/teacher educator, I have a clear record advocating for students (1a) and teachers (1b).

The current megaphone allowed teachers and parents who see failure and feel failed is certainly valuable while also being anecdotes, not science, and more problematic, cherry-picked anecdotes.

I have heard much different voices, and I also know they are being ignored or often silenced with threats.

I suspect there are far more reading teachers who need better teaching and learning conditions so that their expertise can be more effective with students than those who need retraining.

Or at least if we addressed teaching and learning conditions, we would have a better context in which to decide who needs PD.

That story isn’t what sells , however.

The SOR movement has pitted teachers against teachers, teachers against administrators, and teachers against teacher educators. Those conflicts serve the interests of commercial programs, not students and teachers.

So, finally, if we drop the “science” bullying and admit that teacher voices matter, then we must hand that megaphone to all voices, not just the ones that serve the market interests of those who see the SOR movement as an opportunity to cash in (again).

If Teacher Education Is Failing Reading, Where Is the Blame?

Of Rocks and Hard Places—The Challenge of Maxine Greene’s Mystification in Teacher Education, P. L. Thomas

comic books, Daredevil, Education

Daredevil: The Collection

December 5, 2022 plthomasedd

[Header and all images property of Marvel and artists]

This origin story is set in rural Upstate South Carolina during the 1970s, and there are plenty of uncomfortable parallels with the scrawny nerd-to-hero Peter Parker (the origin story of Spider-Man, 1962, occurring a bit over a year after my birth, 1961).

This origin story isn’t about nerd-to-hero, however; it is about an anxious rail-thin teenager being diagnosed with scoliosis and stumbling into reading, drawing from, and collecting Marvel comic books.

From 1975 until I graduated high school in 1979, I managed to collect about 7000 Marvel comic books, the greatest bilk of what was published in the 1970s. One huge part of that collection was buying a collection from an ad in our local newspaper.

As I have written about often, my parents turned themselves inside out to support their son resigned to spending his adolescence wearing a full body brace to correct a crooked spine. Buying comics and even attending a comic-con in Atlanta were stressful for my working-class family, but my parents never wavered.

While my collecting—and drawing from comic books—gradually faded while I was in college and then married in the early 1980s, I held onto that collection until my then-wife and I decided to buy a townhouse before having our only child.

Here, I allowed the normal life expectations to prompt a really bad decision—selling the entire collection to a comic book store in Charlotte (who mainly wanted the X-Men titles, and the full original run of Conan) for enough money to make a small downpayment on that townhouse.

While the money for us then was enough, looking back, I essentially threw away a wonderful collection because of impatience to start the sort of life I believed I was supposed to follow.

Over the next 40 years, I was a former comic book collector—although I popped back into collecting a few times because of students I taught and the growing wider interest in superheroes grounded in films featuring Batman and then the X-Men.

Also over those 40 years, my life—as life does—changed dramatically and in ways I could have never envisions.

In 2002, I moved from K-12 teaching to higher education, and it is then, that I turned to comic book scholarship/blogging and began once again filling my office with comic books used in that work as well as starting (without any initial purpose) collection Daredevil, focusing on my favorite Alex Maleev run.

The 2010s included the greatest changes in my life. Grandchildren, another serious cycling versus car accident (on Christmas eve 2016), the death of both parents in 2017, and then a major life change in 2019 after spending two years in therapy.

This may seem trivial to many people, but a key to coming to embrace my true self, and thus, true life, was to allow myself to return to the joys of my teenage years.

For a few years now, I have recommitted to comic book collecting, focusing on Daredevil and Black Widow along with a few other Marvel (and some DC) titles.

I moved my small collection from my office into a very small apartment already overwhelmed by two occupants and way too many high-end bicycles.

But in 2022, we moved into a larger apartment allowing us to dedicate a small bedroom to those bicycles and that growing collection—along with another new avocation, Lego.

Something unexpected happened in 2022.

First, I was able to complete my Black Widow solo series collection while I also wrote an 8-blog series on Black Widow and recently submitted a book proposal on the character (currently under review).

Next, I gradually began to make huge dents in the more daunting Daredevil collection since his solo series began in 1964 and includes nearly 700 issues.

After connecting with a local comic book store, where they targeted Daredevil issues for me, I began making some large purchases and eventually believed I could complete the entire run.

A tipping point in 2022 was making the big leap to buy Daredevil 1, 2, and 3 from that store, and then realizing I had dwindled my needed issues from about 100 to just about 10.

In that final 10, I was faced with a few key issues that were experiencing the usual market inflation connected to the MCU so I was patient and watched for dropping prices at local stores and on ebay.

This post in December 2022, then, is a magical one for me, surreal as I announce with acquiring Daredevil v.1 issue 7 (the first issue with his red uniform), I have a full run of Daredevil.

Issue 7