Teacher Evaluation, VAM

VAMboozled by Empty-Suit Leadership in SC

7 Comments

Rep. Andy Patrick, R-Hilton Head Island (SC), has made two flawed claims recently, one about leadership and another about teacher evaluation (“S.C. lawmaker proposes teacher evaluation plan,” Charleston Post and Courier, December 10, 2013).*

First, and briefly, Patrick asserts that SC needs leadership for superintendent of education, discounting the importance of experience or expertise. As I will address below, Patrick’s lack of experience and expertise is, ironically, evidence that leadership is not enough. In fact, leadership begins with experience and expertise; it doesn’t replace those essential qualities.

Next, and more importantly, Patrick’s and current Superintendent Mick Zais’s pursuit of test-based teacher evaluation reform is deeply flawed and discredited by research on value added methods (VAM) of evaluating teachers.

Endorsing VAM-heavy teacher evaluation joins grade retention, charter schools, and Common Core as a series of policy decisions in SC that are countered by the research base—resulting in a tremendous waste of time and funding that should be better spent for our students and our state.

For example, Edward H. Haertel’s Reliability and validity of inferences about teachers based on student test scores (ETS, 2013) now offers yet another analysis that details how VAM fails, again, as a credible policy initiative. Haertel’s analysis offers the following:

  • First, Haertel addresses the popular and misguided perception that teacher quality is a primary influence on measurable student outcomes. As many researchers have detailed, teachers account for about 10% of student test scores. While teacher quality matters, access to experienced and certified teachers as well as addressing out-of-school factors dwarf narrow measurements of teacher quality.
  • Next, Haertel confronts the myth of the top quintile teachers, outlining three reasons that arguments about those so-called “top” teachers’ impact are exaggerated.
  • Haertel also acknowledges the inherent problems with test scores and what VAM advocates claim they measure—specifically that standardized tests create a “bias against those teachers working with the lowest- performing or the highest performing classes” (p. 8).
  • The next two sections detail the logic behind VAM as well as the statistical assumptions in which VAM is grounded, laying the basis for Haertel’s main assertion about using VAM in high-stakes teacher evaluations.
  • The main section of the report reaches a powerful conclusion that matches the current body of research on VAM:

These 5 conditions would be tough to meet, but regardless of the challenge, if teacher value-added scores cannot be shown to be valid for a given purpose, then they should not be used for that purpose.

So, in conclusion, VAM may have a modest place in teacher evaluation systems, but only as an adjunct to other information, used in a context where teachers and principals have genuine autonomy in their decisions about using and interpreting teacher effectiveness estimates in local contexts. (p. 25)

  • In the last brief section, Haertel outlines a short call for teacher evaluations grounded in three evidence-based “common features”:

First, they attend to what teachers actually do — someone with training looks directly at classroom practice or at records of classroom practice such as teaching portfolios. Second, they are grounded in the substantial research literature, refined over decades of research, that specifies effective teaching practices….Third, because sound teacher evaluation systems examine what teachers actually do in the light of best practices, they provide constructive feedback to enable improvement. (p. 26)

Haertel’s concession that VAM has a “modest” place in teacher evaluation is no ringing endorsement, but it certainly refutes the primary—and expensive—role that VAM is playing in proposals to reform teacher evaluation in SC and across the U.S.

Would SC benefit from focusing on teacher quality—as well as insuring all children have equitable access to experienced and certified teachers? Absolutely.

But current calls by leaders with no experience or expertise in education are failing that possibility by rushing to implement policy that is contradicted by a growing body of research discounting the value of VAM as a key element of teacher evaluation.

SC students, teachers, and schools cannot afford doubling-down on a failed test-based education culture, and certainly, SC cannot afford more leadership without expertise, which is what Representative Patrick is offering.

* Submitted to and unpublished in, so far, Charleston Post and Courier.

Please see VAMboozled web site for research refuting VAM.

CCSS

GUEST POST: Literature, Young Adult Fiction, and the Common Core, Lauren Girouard

1 Comment

Lauren Girouard is a student in my first year seminar (writing intensive). Lauren is an avid reader—a Neil Gaiman fan—and when she chose to write about Common Core, I asked her permission to post this as a rare voice of a student. I hope you enjoy this fine essay by a student who has come to love reading in the way we often claim we are seeking.

Literature, Young Adult Fiction, and the Common Core

Lauren Girouard

I have never been one to appreciate the adage that “we are what we read.” Undoubtedly, we are informed by what we read, we can learn from and be inspired by our choice in reading material, but are we really what we read?

Literature certainly has the ability to ignite our passions and spark our imaginations. As a child, I adored C.S. Lewis’ The Chronicles of Narnia, and while I was far too young to understand the deeper meaning behind the words I was reading, all I knew was that the world he created was beautiful. As I grew older, I was fortunate enough to have an English teacher who, while he taught all the literary standards inside the classroom, dispensed booklists of young adult fiction we should read to better appreciate the beauty and value of the written word in our lives. From a young age, I viewed literature as a beautiful notion to place on a pedestal and adore. Unfortunately, many students are not so fortunate and they struggle to achieve that same love for literature.

Goodreads, a social networking site based on sharing and categorizing literature, serves as an organizational tool for avid readers, English teachers, and authors, and features lists of books that have been frequently organized into the same category. Goodreads’ list of Popular High School Literature Books is very starkly contrasted with Goodreads’s list of Popular Young Adult Fiction Books, suggesting a widening gap between what adolescents must read in school and what they consider enjoyable literature. This disconnect between the texts most often read in a classroom setting and the books students choose to read in their free time would not be concerning if the works selected for the classroom were truly superior examples of how students should write or offered deeper insight than that offered by popular young adult fiction. However, recent findings and a blossoming in the field of adolescent literature suggests that the disparity between what young adults want to read and what they read as a part of their curriculum need not be so large.

Some will always adhere staunchly to the classics. The Argumentative Old Git suggests that literature must be taken much more seriously, that we even go so far as to diminish literature as a subject when we allow students to select their own reading material. The idea that adolescents should not be responsible for their own curriculum in the English classroom is spawned from the notion that serious literature, that is to say what is commonly accepted as classroom literature, will make students feel that literature is a graver, more important subject. Yet a recent article in “The Telegraph” suggests that students are less well-versed in literature than they used to be because they are dropping the core subject in favor of lighter fare. This runs in direct contrast to the arguments of The Argumentative Old Git and those like him who continue to fight for a curriculum rooted solidly in the literature that has become canon for high school teachers.

While Great Britain struggles to work out whether literature should be taught at all, the United States system of education is embroiled in a feud over Common Core, which would shift the study of reading in high school even further away from fiction and certainly from young adult fiction, and focus more squarely on informative texts. The program has been widely instituted, as this map from their official website shows:

Untitled

While some would prefer to make the debate over these new standards political, even going so far as to substitute Common Core for Obamacore, others are more willing to consider the pros and cons of such a program more fairly, even when they come to the same conclusion: the program is detrimental for most literature in the public education system. Common Core simply cannot seem to win on any front, even when it tries to introduce “The Hobbit” as a piece of fiction suitable for classroom reading.

If it is abundantly apparent to many that these Common Core standards, which stifle a teacher’s ability to select their own literature or allow their students to have a voice in the standard curriculum, are harmful to schools, why then do so many still insist upon keeping such a wide range of young adult literature out of the classroom? The American Library Association seems to wonder as well, going so far as to tout the literature most frequently banned in the classroom each year. The comparison of these lists to the Goodreads lists above reveal that far more Young Adult Fiction is being banned than the tried and true literary standards that so many have become comfortable with.

Some teachers continue to fight against regulated reading standards that stifle the freedom of students to choose what they read. In radical steps, some like teacher Lorrie McNeill are implementing a new reading workshop strategy where children pick their own titles for discussion in the classroom. These attempts to combine education and pleasure have not gone unnoticed, but the continued implementation of Common Core standards suggest that these methods are simply not accepted in as wide a context as would be necessary to put choice of literature in the hands of the students.

Despite the backlash against Common Core standards and curriculum, there seems to be relatively few voices championing teacher and student choice in crafting an educational list of literature for use in the classroom each year. No wonder students are increasingly opposed to even taking an additional English class, when popular young adult fiction is being stifled, whether by petitioning parents or government regulation.

CCSS, education, education reform, Standards, Testing

Faith-Based Education Reform: Common Core as Standards-and-Testing Redux

7 Comments

Let’s start with irony:

Compelling research suggests that the public in the U.S. is unique in its commitment to belief, often at the expense of evidence—leading me to identify the U.S. as a belief culture.

Additionally, while I remain convinced that the U.S. is a belief culture, I also argue that, below, the political cartoon posted at Truthout captures another important dynamic: Many committed to their own beliefs both do not recognize that they are committed to belief and belittle others for being committed to their beliefs:

By Clay Bennett, Washington Post Writers Group | Political Cartoon
By Clay Bennett, Washington Post Writers Group | Political Cartoon

And this brings me to advocacy for Common Core standards, with one additional point: Along with embracing belief over evidence, the public (along with political leadership) in the U.S. tends to lack historical context.

Placed in the century-plus commitment to pursuing new and supposedly higher standards for public schools, then, Common Core advocacy falls into only two possible characterizations:

  1. Common Core is a response to the historical failure of all the many standards movements that have come before, and thus, the success of CC depends on CC being somehow a different and better implementation of an accountability/standards/testing paradigm.
  2. CC advocacy is yet another example of finding oneself in a hole and persisting with digging despite evidence to the contrary. In other words, CC may well be yet another commitment to a reform paradigm that isn’t appropriate regardless of how it is implemented, as John Thompson details in his review of The Allure of Order:

Jal Mehta’s masterpiece, The Allure of Order, answers the question, “Why have American [school] reformers repeatedly invested such high hopes in these instruments of control despite their track record of mixed results?” He starts with the review of how the bloom fell off the NCLB rose, explaining why its results in the toughest schools have been “miserable.” In the highest poverty schools the predictable result has been “rampant teaching to the test” which has robbed children of the opportunity to be taught in an engaging manner.

Mehta explains that this “outcome might have been surprising if it were the first time policymakers tried to use standards, tests, and accountability to remake schooling from above.” The contemporary test-driven reform movement is the third time that reformers have used the “alluring but ultimately failing brew” of top down accountability to “rationalize” schools and, again, they failed [emphasis added].

These two claims are themselves evidence-based (and it will be interesting to watch as others respond, as they have to my previous work on CC, by either ignoring evidence or garbling evidence to support what proves to be faith-based commitments to CC), and thus should provide a foundation upon which to continue the debate about CC.

CC advocacy and criticism are often based on false narratives and baseless claims (see Anthony Cody for one example of this problem and Ken Libby‘s [@kenmlibby] cataloguing on Twitter #corespiracy)—again reinforcing the pervasive and corrosive consequences of faith-based, but not evidence-based debates.

Instead, we should start with an evidence-based recognition about standards-driven education reform.

For example, the existence and/or quality of standards are not positively correlated with NAEP or international benchmark test data—leading Mathis (2012) to conclude about CC: “As the absence or presence of rigorous or national standards says nothing about equity, educational quality, or the provision of adequate educational services, there is no reason to expect CCSS or any other standards initiative to be an effective educational reform by itself [emphasis in original]” (p. 2 of 5).

Therefore, CC advocacy has some principles within which it should continue if that advocacy is to be credible and thus effective:

  • Claims that CC advocacy is separate (and can be separated) from high-stakes testing must show evidence of when standards have been implemented without high-stakes tests (and how that was effective) or evidence of some state implementing CC without high-stakes tests connected. Otherwise, this is a faith-based claim.
  • Claims that accountability built on standards and high-stakes testing is an effective education reform strategy must show evidence of how that has worked in the previous state-based accountability era and then explain why those examples of success must now be replaced by the new CC set of standards. Otherwise, this is a faith-based claim.
  • CC advocacy has been endorsed as a logical next step built on the call in NCLB for scientifically based education reform; thus, CC advocates must either comply with the two points above or concede that the CC era is a break from evidence-based reform.

I am no advocate for remaining only within rational, evidence-based, and quantifiable norms for decision making, by the way, but I am convinced we must make clear distinctions between evidence and belief—and I am equally convinced that many education reformers enjoy a flawed freedom to call for evidence from their detractors while practicing faith-based reform themselves.

It is the hypocrisy that bothers me, the hypocrisy of power:

scientist evidence – Married to the Sea

Let’s acknowledge that teachers currently work under the demand of measurable evidence of their impact on students while CC advocates impose faith-based policies such as CC, new generation high-stakes testing, merit pay, charter schools, value-added methods of teacher evaluation, and a growing list of commitments to education reform at least challenged if not refuted by evidence.

CC advocates now bear the burden of either offering the evidence identified above or admitting they are practicing faith-based education reform.

Gun Control

In Memory: Sandy Hook Elementary, One Year After

1 Comment

I fear that we have become either callous or numb—either is a tragic consequence of mass shootings, even those at schools, becoming commonplace.

Here, I invite you to read from a group of pieces I wrote after and about the mass school shooting at Sandy Hook Elementary in Newtown, CT, one year ago, December 14, 2012:

“They’re All Our Children” (AlterNet)

  • Alongside discussions of mental health and gun control, we must use the lessons of Sandy Hook to reframe the debate around education reform.

Remembering Sandy Hook: Protecting Guns, Sacrificing (Some) Children

Misreading the Right to Bear Arms (AlterNet)

  • Confronting gun ownership is an argument about violence — not about autonomy and freedom.
Testing

REVIEW: De-Testing and De-Grading Schools, Bower and Thomas

1 Comment

Reviewed by J. Spencer Clark, Utah State University, which concludes:

The purpose of this book was to offer a map of the high-stakes accountability and standardization landscape, and more importantly to provide ways to navigate this landscape in positive ways. Bower and Thomas are successful in this regard and have provided a powerful critique that equally identifies powerful alternatives to high-stakes accountability. Overall, this is a fresh look at how to meld the theories behind de-grading and de-testing schools with actual classroom practice. This book could be a useful tool for instructors of pre-service methods and assessment courses, and possibly educational foundations courses at all levels, as it provides both an analysis of key aspects of a failing system of accountability and possible alternatives to it.

Bower, J., & Thomas, P. L. (2013). De-testing and de-grading schools: Authentic alternatives to accountability and standardization. New York, NY: Peter Lang USA.

De-Testing and De-Grading Schools

Testing

GUEST POST: Continu—what? Sara Newell

3 Comments

Continu—what?

Sara Newell

How do you derive meaning from a number? Should a parent or student respond differently to a 97% than to a 99%? What about a 75%? How do you know what number to assign to a student created product you’ve never seen before? As a 5th grade teacher at the Charles Townes Center for highly gifted students in grades 3-8, I felt these questions were a constant thorn in my side.

My students qualified for invitation to the center in Greenville, South Carolina based on scores in the top percentile on nationally normed tests. The current, numerical grading system has always presented quite a challenge to me as a public school classroom teacher—how do I push my students to strive for excellence without encouraging the crippling effects of perfectionism? In giving students and parents a true measure of learning, personal achievement, and goal-setting, the numerical grading system always seemed to me ineffective at best. Since I began teaching gifted students, I have been in a constant struggle to find a more effective way to provide accurate feedback about their current performance while motivating them to continue to give their best effort on whatever challenges are presented next.

The assessment issues faced in our school were exaggerated versions of the problems caused by the numerical grading system in schools across the country. The nature of our students simply intensifies the problem. For example, the vast majority of my students can ace a grade-level multiple-choice test before I even engage in the first lesson. Should they just receive “A’s”? Is that what they earned? And, if I – instead – increased the depth and complexity of my instruction to provide the appropriate intellectual challenge and a student then only mastered 92% of that material—is it “fair” to assign a less than stellar grade?

This issue becomes even more important as students begin to move into high-school level courses. How does a 92 affect their GPA when they are enrolled in high-school and honors courses beginning in 7th grade? Should they be scored less than their peers who attend mainstream schools? Teachers of gifted (and all) students face these types of problems again and again as they are asked to differentiate to meet the needs of diverse learners. How does a teacher maintain some sort of equity and still challenge students appropriately? Some schools have attempted to rectify this disparity by offering higher grade points for honors or AP classes. This does not remedy the problem—it simply magnifies the spectrum of an inaccurate ruler and introduces an additional disadvantage for college applicants whose schools do not offer this option.  The issue of quality feedback and appropriate challenge remains.

For a while, I thought the solution to the problem was that I needed to design better rubrics. If I could just break assignments down into more concrete sections, the students would see what they needed to do and would be able to demonstrate mastery in a way that provided equal access to all while challenging students appropriately. (And I could still put a number on it and feel good about it.) Unfortunately, there were still roadblocks.

In a subject-integrated inquiry-based classroom, how do you quantify “delightful,” “sophisticated,” “clever” and all of the other descriptors that address the work of students who clearly went above and beyond the scope of the assignment? The scale model in gingerbread of the Metropolitan Museum of Art or the original musical composition in response to a Langston Hughes poem received a 100% that was “worth” exactly the same amount as the student who ploddingly met the minimum requirement for each element. So, the rubric was a start, but it still lacked the depth I was seeking to truly communicate effectively with my students (not-to-mention their parents) about the quality of their work. Truly authentic assessment with feedback that can guide students into becoming independent learners still seemed out of reach.

Then, our principal brought back the idea of using continua from a school visit in Seattle. These reading, writing and math continua are based on the work of Bonnie Campbell Hill and provide a system to analyze student skill and progress over many years. The lists are simple and concise. They do not include every possible state standard but instead provide an overview of the crucial skills students need to be successful.

I jumped on these tools and piloted using them with my students almost immediately. My students completed self-evaluations, rating themselves at the “beginning,” “developing,” “proficient,” or “independent” levels described on the continua. I then added my own assessment of their skills. We used these in our student-led conferences, and I could see the beginnings of evidence-based discussions in their conversations. Students were using their writing portfolios and math assessments to provide concrete support for their evaluations. This represented a terrific shift in the way students and parents thought and talked about student work.

Instead of parent comments like, “What did you miss?” or “Great job!” I was hearing, “How did you decide you were proficient in reading fluency instead of independent?”  One parent asked his son, “I didn’t know you should be reading different genres. What are you reading right now? Is that the kind of book you always read?” These conversations were so much richer than the previous years’ event which basically consisted of students proudly showing their work while their parents made appreciative mumbles and nodded their heads.  I was excited by the beginnings of the give and take that marks a truly thoughtful discussion, but something was still missing. There was still not a way to communicate the truly exceptional or the gifted student who was playing it safe.

After musing on this initial success and talking repeatedly with a middle school colleague struggling with many of the same frustrations, we decided that we needed to create an additional continuum. The difference between the “minimum doer” and the outstanding student in our school was based not only on the ability to demonstrate skill mastery, but on the willingness to strive to apply critical and creative thinking skills. With this in mind, I pulled together a number of resources and began to hammer out a draft of a critical and creative thinking skills continuum. (I still haven’t hammered out a shorter name, though.) Dr. Richard Paul’s mini-guides on critical and creative thinking, Torrance’s work on creativity and Van Tassel-Baska’s writing on application of these skills in the classroom were all of great benefit to me as I worked. My hope was that this document would bridge the gap between the seemingly arbitrary nature of a number grade and the lightning strike of truly outstanding work. I ended up with a scale more rooted in psychology and child development than pedagogy and standards. This was initially surprising, but it became more satisfying as I realized that perhaps with this tool we might finally get to the roots of why one student was clearly outperforming another and more importantly—what to do about it.

The purpose of this creative and critical thinking skills continuum is to provide specific feedback for students and parents about the students’ current progress as well as to communicate in a straightforward way the next steps in their educational growth. Numeric grades are loaded with judgment, both objective and subjective, as well as academic stigma. Students feel that a 100% means that you are perfect while a 67% means that you are a loser. I’ve even had students tell me that even numbers are better than odd numbers (a 99% means that I am a point away from perfect—the most frustrating thing—but a 98% means that I’m solidly in the high “A” category). The focus on the number rather than what the number represents is a bizarre, yet true manifestation of the problem with attempting to quantify something as variable as knowledge and learning. Students become so focused on the number and what it “means” that they completely lose sight of the true purpose of assessment– reflection and growth. A continuum has no numbers—hence, no judgment. There is no “right” or “wrong” way to evaluate oneself with this method.

On first executing the continua in my classroom, I did not ask my students to provide evidence to support their evaluations. (That came later…) It was absolutely fascinating to see students read through and begin their self-evaluations on the critical and creative thinking continuum. I only allowed one hour of the class period for students to complete their analysis of this one-page document. However, most of my students took much more time than that. The room was silent. My students were incredibly focused on their reading and analysis. As 5th graders are still fairly ego-centric at this concrete operational stage (thanks Piaget), they seemed to feel that an assessment all about them was well worth their time. The questions students asked about concepts like “intellectual humility” and perseverance got to the core of what I had been trying to teach for years. Why is it important to continue to try to find a solution to a difficult problem? What does it mean to demonstrate originality? How do I know if I am taking an intellectual risk? These were the questions that I wanted my students to ask—and this was finally a document that set the stage to ask them.

Another revelation occurred when I reviewed these documents individually. I began to get a much more relevant picture of how each child saw him or herself. It was striking to compare the self-assessments with the list of test-score data that my principal had just sent out. (Yes, we are still in a public school. And yes, we still have to do things like set learning goals based on the number of points students “should” improve on certain tests.) Those standardized test scores have been relatively meaningless to me in the past. However, coupled with the information from the continuum self-assessments, a fascinating phenomenon was revealed. By and large, students in the top performing test score group had consistently given themselves the lowest evaluations on the continuum while the students with the lowest (comparative) test scores had marked themselves as having mastered all or almost all of the critical and creative thinking skills. The Dunning Kruger effect in action! We had a tremendous class discussion about this effect—in which less competent people in a field tend to overestimate their abilities. We analyzed how it applied to their attitudes and approaches to learning.  I began to see a shift in several students’ attitudes and performance following this one illuminating discussion.

This initial work was very inspiring. I was surprised and pleased at the effort my students put into their evaluations. The vocabulary from the continuum was popping up in our discussions again and again. Instead of “I don’t get it,” I was hearing comments like, “I need to clarify this—do you mean…?” The students were beginning to look at learning through this alternate lens. I continued to have students review the continuum and reflect on their progress as we completed units of instruction. They documented their growth and reflected on their struggles.

We also used the continuum to decide on areas of focus for the next units. I previewed with the students what I felt were the “big ideas” for learning while they made choices about skills they thought it important to develop. The quality of our communication continued to improve. Our goals were aligned—I was attempting to provide opportunities for them to improve in areas that THEY had identified as needing work. This method gave them a sense of control over their own learning.

Other teachers in my school are currently working to apply the math, reading, writing, and thinking skills continua in their classrooms. In the middle school, students are expected to provide support for their analysis as they complete their initial evaluations. In the lower grades, teachers use the continua to shift the focus from what students can’t do to what students COULD do. These continua are shared between teachers vertically to provide a long-range picture of the student’s development over time. This is something that a numeric grade based on grade-level standards fails to communicate.

At first some teachers struggled with how to make the continua relevant to their students, and all teachers recognized that the thought and effort needed to accurately utilize the continua required more time than typical “grading.” However, the value of the knowledge gained far outweighs the extra effort the analysis requires.

The next breakthrough came when I began to use the critical and creative thinking continuum in one-on-one parent conferences. For years, my conferences followed a fairly typical script. First, I would go over the previous year’s test scores. Then, I would discuss grades. The parent(s) and I would discuss any issues or “concerns,” and then I would try to end on some kind of positive note. For the parents of my highest achievers though, this was not a helpful meeting. While I’m sure they enjoyed hearing me list all of the delightful adjectives that described their child, I’m not sure that they felt that they were getting a clear picture of what their child could do to continue to grow.

The use of the continua has changed our discussions. My conferences conducted this fall focused on which elements their child was clearly demonstrating as well as areas their child could continue to develop. I was able to explain the Dunning Kruger Effect to parents who thought their child was practically perfect but who in reality was barely making an effort. I described to the parents of the perfectionists what intellectual risk-taking was and how their child could begin to do it. The conversations were so much richer than in the past, and parents did not feel that I was judging their parenting, or their children.

Instead, the focus was on attributes and evidence.  Parents were surprised and fascinated when reading their child’s reflections. The conferences now were a detailed conversation about the whole child and how he or she interacted with the world. Even more importantly, parents were now able to support our classroom objectives with greater accuracy.  One parent commented, “We were delighted to discuss and learn about the Creative Continuum. The Continuum is visual and the skill sets are clearly presented…Our meeting was one of the most informative conferences I have attended.”

Shifting our focus from a numerical grading system to a continuum-based evaluation has started to address many of the assessment issues I was facing. My students have stopped asking, “Is this for a grade?” as though that alone determines the value of an assignment. I continue to work to provide more opportunities for students to develop those critical and creative thinking skills. Knowing that I am going to be asking them to evaluate their growth—I am very conscious of the need to design learning experiences that require students to demonstrate those skills.

Most importantly, the students themselves feel a sense of ownership over their learning, and now they are making the effort to ask accurate, insightful questions about what they can do and what they still need to learn to do. Removing the focus from the number grade and putting it back on the evaluation of skills and attributes improves the quality of instruction, performance and communication. The end result is a focus on authentic student learning and success.

References

Davis, G.A., & Rimm, S.B. (2009). Education of the gifted and talented (6th ed.). Boston, MA: Allyn and Bacon.

Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence (PDF). Current Directions in Psychological Science, 12(3), 83–87.

Elder, L., & Paul, R. (2005). The miniature guide to critical thinking concepts & tools. (4th ed.). Dillon Beach, CA: The Foundation for Critical Thinking.

Hill, B. C. (2008). Retrieved from http://www.bonniecampbellhill.com/support.php

Van Tassel-Baska, J., & Stambaugh, T. (2006). Comprehensive curriculum for gifted learners. (3rd ed.). New York, NY: Pearson.

For Further Reading

Rubrics

Kohn, A. (2006). The trouble with rubrics. English Journal, 95(4), 12-15.

Wilson, M. (2007). Why I won’t be using rubrics to respond to students’ writing. English Journal, 96(4), 62-66.

Wilson, M. (2006). Rethinking rubrics in writing assessment. Portsmouth, NH: Heinemann.

Self-Assessment

Liberating Grades/Liberatory Assessment, sj Miller

Testing

What Are Tests Really Measuring?: When Achievement Isn’t Achievement

6 Comments

High-stakes standardized testing must be the most resilient phenomenon ever to exist on the planet. Joining high-stakes standardized testing in that (dis)honor would be the persistent but misleading claim that test scores are primarily achievement (and a growing future candidate for this honor is the claim that test scores by students, labeled “achievement,” are also credible metrics for “teacher quality”).

Let’s start with a couple statistical breakdowns of what test scores constitute:

But in the big picture, roughly 60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty). Observable and unobservable schooling factors explain roughly 20 percent, most of this (10-15 percent) being teacher effects. The rest of the variation (about 20 percent) is unexplained (error). In other words, though precise estimates vary, the preponderance of evidence shows that achievement differences between students are overwhelmingly attributable to factors outside of schools and classrooms (see Hanushek et al. 1998; Rockoff 2003; Goldhaber et al. 1999; Rowan et al. 2002; Nye et al. 2004).

Just 14 per cent of variation in individuals’ performance is accounted for by school quality. Most variation is explained by other factors, underlining the need to look at the range of children’s experiences, inside and outside school, when seeking to raise achievement.

Next, consider this from the UK:

Differences in children’s exam results at secondary school owe more to genetics than teachers, schools or the family environment, according to a study published yesterday.

The research drew on the exam scores of more than 11,000 16-year-olds who sat GCSEs at the end of their secondary school education. In the compulsory core subjects of English, maths and science, genetics accounted for on average 58% of the differences in scores that children achieved.

While the genetics claim is potentially dangerous, and certainly controversial, the article offers some important clarifications:

The findings do not mean that children’s performance at school is determined by their genes, or that schools and the child’s environment have no influence. The overall effect of a child’s environment – including their home and school life – accounted for 36% of the variation seen in students’ exam scores across all subjects, the study found….

Writing in the journal, the authors point out that genetics emerges as such a strong influence on exam scores because the schooling system aims to give all children the same education. The more school and other factors are made equal, the more genetic differences come to the fore in children’s performance. The same situation would happen if everyone had a healthy diet: differences in bodyweight would be more down to genetic variation, instead of being dominated by lifestyle.

Plomin said one message from the study was that differences in children’s performance were not merely down to effort. “Some children find it easier to learn than others do, and I think it’s appetite as much as aptitude,” he said. “There is a motivation, maybe because you like to do what you are good at.”

Genetics, he said, caused people to create, select and modify their environment, and so nature drives nurture, which in turn reinforces nature. A child with a gift for maths seeks friends who like maths. A child who learns to read easily might join a book club, and work through books on the shelves at home.

Additional points drawn from this research present some strong cautions about continued reliance on not only standardized tests, but also uniform national standards:

“Education is still focused on a one-size-fits-all approach and if genetics tells us anything it’s that children are different in how easily they learn and what they like to learn. Forcing them into this one academic approach is going to make some children confront failure a lot and it doesn’t seem a wise approach. It ought to be more personalised,” he said.

“These things are as heritable as anything in behaviour, and yet when you look in education or in educational textbooks for teachers there is nothing on genetics. It cannot be right that there’s this complete disconnect between what we know and what we do.”

Finally, consider this research on the disconnect between test scores and student abilities:

To evaluate school quality, states require students to take standardized tests; in many cases, passing those tests is necessary to receive a high-school diploma. These high-stakes tests have also been shown to predict students’ future educational attainment and adult employment and income.

Such tests are designed to measure the knowledge and skills that students have acquired in school — what psychologists call “crystallized intelligence.” However, schools whose students have the highest gains on test scores do not produce similar gains in “fluid intelligence” — the ability to analyze abstract problems and think logically — according to a new study from MIT neuroscientists working with education researchers at Harvard University and Brown University.

In a study of nearly 1,400 eighth-graders in the Boston public school system, the researchers found that some schools have successfully raised their students’ scores on the Massachusetts Comprehensive Assessment System (MCAS). However, those schools had almost no effect on students’ performance on tests of fluid intelligence skills, such as working memory capacity, speed of information processing, and ability to solve abstract problems….

Instead, the researchers found that educational practices designed to raise knowledge and boost test scores do not improve fluid intelligence. “It doesn’t seem like you get these skills for free in the way that you might hope, just by doing a lot of studying and being a good student,” says Gabrieli, who is also a member of MIT’s McGovern Institute for Brain Research.

So should we be shocked when students passing high-stakes reading tests in Texas admit they cannot read?:

A female classmate of Tony’s says she can’t get through the stories she reads in school unless someone explains them to her. She’s passed all her state tests, too. How? She says she uses classroom-taught “strategies” on her English reading test and that if she underlines and highlights enough and narrows down her options, she has a better chance of guessing right by playing the odds. She failed her math state test because of the word problems, so she employed her English strategies there on the retry attempt and passed.

Or that the most recent analysis of the teaching of writing in middle and high schools has found that best practice in writing hasn’t occurred because of accountability and high-stakes testing?:

Overall, in comparison to the 1979–80 study, students in our study were writing more in all subjects, but that writing tended to be short and often did not provide students with opportunities to use composing as a way to think through the issues, to show the depth or breadth of their knowledge, or to make new connections or raise new issues…. The responses make it clear that relatively little writing was required even in English…. [W]riting on average mattered less than multiple-choice or short-answer questions in assessing performance in English…. Some teachers and administrators, in fact, were quite explicit about aligning their own testing with the high-stakes exams their students would face. (Applebee & Langer, 2013, pp. 15-17)

Our educational world has been turned over wholesale to testing, despite ample evidence that test scores are many things (markers of privilege, markers of genetic predispositions, markers of teaching-to-the-test), among the least of which are student achievement and teacher quality.

If we don’t have the political will to de-test our schools, the evidence is clear that the stakes associated with testing must be greatly lessened and that the amount of time spent teaching to the tests and administering the tests must also be reduced dramatically.

educator, public scholar, poet&writer – academic freedom isn't free