One of the foundational claims throughout the history of calling for education to be “scientific” is identifying instructional practices as either scientific (or evidence-based) or not.

A key concern with such claims is that they tend to overstate in a black-and-white way that practices are scientific or not, but also, a narrow view of “evidence” is certainly a problem (see Wormeli for example).

The “science of reading” (SOR) movement, for example, has tended to use “scientific” like a baseball bat, demanding that simplistic view of “science” and heralding some practices as evidence-based and others as “contrary practices” or “Ineffective and Currently Unsupported Instructional Strategies.”

Another flaw in the SOR movement’s demand for “scientific” instruction is cherry-picking research to promote and demonize selected practices.

Look at these two examples from pro-SOR documents:

Note here that the lists are different with some overlap, raising concerns about what science matters and what agenda any organization has (NCTQ releases reports that fail basic scientific validity to push a narrow agenda, for example).

The first list from NCTQ is overly dogmatic and an example of cherry-picking masquerading as scientific.

The second list has some promise since it allows for that some practices are not supported yet, a better characterization of science—although the lit review itself suffers from limited use of evidence.

One important element in the SOR movement and the legislation as well as practices that is driven by that movement is holding SOR claims to valid expectations of “scientific” that do not suffer from oversimplification.

In short, cherry picking is not science.