Several years ago, women faculty at my university raised concerns about gender inequity across hiring, retention, and pay. The data suggested those concerns were valid so the university brought in an outside team to examine if gender inequity, in fact, existed at the university.

The university faculty was composed of fewer than 40% women (well below the percentage of women in society) and women faculty had been leaving the university at a higher rate than men faculty for several years. Although the university culture discouraged the sharing of salaries, women faculty were able to establish that women did in fact make less than men—in part, because there was also inequity of rank by gender.

These gender imbalances are common across higher education in the U.S. as well.

The external review gathered more data, mostly interviewing across campus different stakeholders in the university. That report confirmed gender inequity and offered reform strategies to address the imbalances.

Almost immediately upon its release, white male faculty questioned the review on the grounds that it did not meet the high standards of scientific inquiry (quantitative experimental/quasi-experimental research).

This scenario is playing out nationally in a similar way, but focusing on racial inequity (racism) in policing, specifically in the use of deadly force by police officers.

First, it is important to start a consideration of statistic and quantitative data by clarifying language. At the crux of a statistical analysis of gender inequity or racism (incredibly complex phenomena), we must distinguish between equality and equity.

For example, equality as a goal would dictate that universities hire the exact same number of men and women, maintain the same number of men and women at each rank, and pay men and women the exact same at those steps. Equality (in a much darker view of the world) would mean that police officers shoot and kill the same number of white citizens as black citizens (note that such a quantitative approach fails the ethical issue of whether or not police offices should kill any citizens).

Equality, however, is the wrong standard since it fails to acknowledge proportionality; this is especially important when trying to measure racism in the U.S.

Race demographics in the U.S. are significantly imbalanced since there are about 5-6 times more white people than Black people. Here is the importance of starting an investigation of racism in policing with equity.

The data on policing and race, then, become extremely complicated since police do kill more white people than Black people, but that measurement is also inequitable since the imbalance falls well below the race imbalance in society; as Bronner explains: “That’s how you get studies that show 96 out of 100,000 Black men and boys will be killed by police over the course of their lifetimes, compared to 39 out of 100,000 white men and boys — a risk that is 2.5 times higher.”

Two statistical facts (police kill more white people than Black people and police killing are racially inequitable) are simultaneously true, seem to discredit racism in policing for some people, and prove racial inequity. This last point is incredibly important and at the root of the problem with measuring racism in policing.

Once there is credible evidence of inequity (data on gender inequity at my university or policing in the U.S.), the challenge of conducting research on that inequity is identifying why the inequity exists and then establishing if that inequity is justifiable or if that inequity can and should be eradicated.

Another paradox of conducting so-called high quality research on inequity is that experimental and quasi-experimental research (designed to isolate and capture causal relationships between factors) often finds no causal significance in the data, but that doesn’t necessarily mean that the condition doesn’t exist.

This brings me to the work of Roland Fryer, who I first encountered through his research on education (charter schools and teacher quality). Measuring teaching and learning has similar problems to measuring inequity since teaching and learning are highly complex and pose real challenges for isolating relationships among factors.

Fryer’s research on education garnered a great deal of uncritical media and political attention since that research reinforced uninformed and overly simplistic views of teaching and learning among the media, the public, and political leaders.

Bruce Baker, for example, noted about Fryer’s work in education: “But, each of these studies suffers from poorly documented and often ill-conceived comparisons of costs and/or marginal expenditures.”

Here is a pattern that is essential to understand: Experimental/quasi-experimental research fails to show a causal relationship in an examination of inequity, the media rush to cover the research by misrepresenting the conclusion (“didn’t find” doesn’t mean that something doesn’t exist), and public/political biases are triggered and reinfocred.

Fryer, who seems to revel in having surprising outcomes to his research, has recently shifted to studying policing and racism, but the pattern has remained intact.

Recently, Fryer promoted a surprising study that seemed to fail to show racism in police killings of citizens, and the media jumped on board despite the research not yet being peer reviewed.

A paradox of research on inequity is that as long as a culture is inequitable all evidence that seems to disprove inequity benefits from that inequity and even the most intentionally “unbiased” research is likely tainted by that inequity.

Once other scholars, most of whom have more expertise in race and policing than Fryer, began to interrogate Fryer’s research, the “surprise” in his conclusion fell apart—in similar ways as his research on charter schools and teacher quality.

Two aspects of scholarly challenges to Fryer’s research on policing and racism are important to highlight.

First, once Fryer was challenged, he responded in a way that clearly discredits the media interpretation of his findings; Fryer wrote a rebuttal to his critics and concluded:

The time has come for a national reckoning on race and policing in America. But, the issues are thorny and the conclusions one can draw about racial bias are fraught with difficulty. The most granular data suggest that there is no bias in police shootings (Fryer (forthcoming)), but these data are far from a representative sample of police departments and do not contain any experimental variation [emphasis added]. We cannot rest. We need more and better data. With the advances in natural language processing and the increased willingness of police departments to share sensitive data, we can make progress.

Once again, probably due to the use of a non-representative sample, Fryer did not find causal proof of racism in fatal policing, but that is a statistical fact that cannot and should not be used to claim that racism does not exist in policing or in fatal police interactions with citizens.

In a response to Fryer’s response, in fact, Ross, Winterhalder, and McElreath conclude in a review of Fryer and other research that seem to fall outside the standard view that racism does impact policing:

We establish that: (1) the analyses of Ross (2015) and Fryer (2016) are in general agreement concerning the existence and magnitude of population-level anti-black, racial disparities in police shootings; (2) because of racial disparities in rates of encounters and non-lethal use-of-force, the encounter-conditional results of Fryer (2016) regarding the relative frequency of the use of lethal force by police are susceptible to Simpson’s paradox. They should probably not be interpreted as providing support for the idea that police show no anti-black bias or even an unexpected anti-white bias in the use of lethal force conditional on encounter [emphasis added]; and, (3) even if police do not show racial bias in the use of lethal force conditional on encounter, racial disparities in encounters themselves will still produce racial disparities in the population-level rates of the use of lethal force, a matter of deep concern to the communities affected.

A more fair response to Fryer (and others) is that his work—despite its weaknesses—raises challenges about the complexity of systemic racism when trying to determine how racism does or does not impact policing.

Systemic racism pervades virtually every aspect of U.S. society, therefore, teasing out and isolating racism may be nearly impossible to do (see Fryer’s emphasis on “granular data” which allows a scientist to focus on a grain of sand while ignoring the beach and the nearby ocean).

Ultimately, a more disturbing paradox may be that interrogating racism by the numbers will never allow us to consider the importance of human witnessing.

The lived experiences of women and of Black people can be silenced when numbers are allowed to trump the complexities of inequity.

“Granular data” and rigorous experimental research are neither fool-proof nor inconsequential. Scientific inquiry isn’t the problem.

The problem is there is inequity entrenched in the type of “science” that is allowed to count, and that is a cycle that itself maintains the inequity that is often nearly impossible to measure.

Fryer’s research along with the media, public, and political engagement with that research does prove one very troubling thing—a confirmation of Audre Lorde‘s warning: “[T]he master’s tools will never dismantle the master’s house.”

For Further Reading

Why Statistics Don’t Capture The Full Extent Of The Systemic Bias In Policing