Scoring

From EvaluationWiki

The quantitative approach to evaluation; involves the allocation of a number to represent merit/worth/significance. Of course, the key question is the validity of the allocation procedure, which is commonly extremely low. One requirement is the 'point constancy requirement' which stipulates that a point, however obtained, should represent the same amount of merit (etc.); but in most multiple=choice tests this is massively violated since: (i) blind guessing gets you a point in 20% or 25% of the times you guess, although you have zero knowledge in all such cases; (ii) the items vary considerably in difficulty, hence getting the answer right requires very different amounts of merit, but they all (on the usual rubric) get the same points (one point each). So the score allocated is not a valid indicator of merit, a fact that is well-known and exploited by those teaching classes in how to score high on the SAT without actually knowing the material. Hence the grades or ranks that are assigned on the basis of scores on such tests are invalid, often seriously so. Of course, these errors in the scoring procedure can be corrected by using a better rubric (see multiple-rating item), but the persistence of such errors and their support by the big testing outfits is a sign of the poor understanding of evaluation validity by both the testing experts and the general public.