fakability question...

ETS gives as their reason for not taking "noncognitive" skills into consideration in their wide array of tests like the SAT, and GRE and other more basic tests used throughout education the fact that these "noncognitive" skills cannot be easily measured without running the risk of answers being faked. So, even though they claim to recognize the importance of "noncognitive" skills, they choose to ignore them rather than apply concerted effort toward resolving the matter.

"With a strong justification for developing noncognitive assessments and ETS’s history of involvement with them, why does the organization not offer a full array of noncognitive assessments today? Why is there no noncognitive GRE or SAT subtest? The answer: Many policy makers and scientists are skeptical that noncognitive qualities can be measured reliably and in a valid way. Typically, in both research and operational use, noncognitive qualities are assessed through self-ratings. Examinees are asked questions such as, “Are you exacting in your work?” “Do you get chores done right away?” “Do you keep your emotions under control?” “Do you take time to reflect on things?” There are two problems with these kinds of ratings—the standard is not clear (i.e., relative to whom?), and they are easily faked. In almost any serious discussion of the use of noncognitive assessments, the issue of “fakability” or “coachability” comes up, and this issue is the trump card that thwarts further discussion."

In other words, they know these "noncognitive" skills are important, but they are too hard for them to reliably measure. That they choose to thence ignore these skills creates a state in which they no longer receive the emphasis they deserve. One of the important reasons they acknowledge for using "noncognitive" skills as part of standarized testing is as follows:

"An argument for noncognitive assessments is that they go beyond “academic intelligence” as Robert Sternberg (1985) puts it and tap the full range of qualities that affect and are affected by schooling. But another argument is that using noncognitive assessments may serve to reduce the test score gap, the mean difference in scores between White and Black test takers commonly observed on more narrowly focused cognitive assessments. Research suggests that there is no score gap or a reduced score gap on noncognitive assessments (Sackett, Schmitt, Ellingson, & Kabin, 2001). Combining noncognitive and cognitive test scores in a selection index would result in a reduced overall score gap."

So, in other words they are willing to ignore the full range of skills and they acknowledge the distortion in the testing of black children vs. white children but this doesn't matter enough to ETS to do anything about it. It takes a huge amount of academic arrogance to make such a major adverse impact on American education while being knowledgeable of the harm they inflict, and to thence to do nothing about it because of the vast amount of money they are raking in at the time. Skills actually fall into two convenient categories... those you can easily measure and those you cannot. Measuring one while ignoring the other is distorts the whole fabric of American education.

My own use of the term "noncognitive" is placed in quotes as even ETS recognizes the term, despite their frequent use of it is a misnnomer. They excuse themselves as follows:

"The term noncognitive, although a misnomer, is widely used in psychology and measurement. Other relevant terms are nonacademic, socioaffective, affective-motivational, and personality."

Unlike the realm of high-stakes standardized testing where there all kinds of opportunities for fraud and fakability can and do occur, when you make something from real wood, its truth of its character and intelligence is present for all to see. You cannot fake the cutting of a joint. The grain at the corners of a box was matched with care or it was not. The lid fits or does not. The work expresses care or it does not. So let's not fake our children's educations. Let's give them real things to do that promote real hands-on learning and all the important facets of human character and intellect that arise from it.

Make, fix and create...