Prediction

From EvaluationWiki

A claim about the future (cf. a retrodiction). In the professional context this will normally originate from an expert e.g., a clinical psychologist, or by inference from scientific laws or other regularities such as documented correlations. In some fields of evaluation, e.g., policy analysis, predictions are crucially important-when evaluating proposed policies, obviously the key question is what will happen if it is implemented, which of course requires a prediction. It is extremely important to be clear that evaluations of some evaluands, e.g., the performance of the first elected legislature in Japan, do not in themselves generate predictions. That is, one can evaluate a system as excellent, on the present evidence, without any commitment to its future performance. But in many cases, including most program evaluations, the client is hoping to hear from the evaluator that the program is likely to continue to perform well, and in that case, where the program is not just a historical one, the evaluation will require a prediction about future performance, which involves a specific kind of inference (and usually a specific kind of data), different from the kind that justifies the evaluation of present and past performance. Another case where the distinction is often overlooked with fatal results is in the evaluation of research proposals, researchers, and R&D centers, when the evaluation is being done to justify funding support. All of these evaluations center around the evaluation of the key researchers involved, but that evaluation in itself is not an adequate basis for evaluating the funding request; nor is it enough to evaluate the researcher plus the intellectual (e.g., scientific) content of the proposal, although these two items are often all that is addressed. But that is not enough to support the prediction of good work in the future, the relevant and essential prediction. To get at that, one must also look carefully for evidence of burnout, disruptions in personal life, a recent switch of interests, etc. One indicator of these problems is a decline in productivity in the last year or two, a factor that is rarely separated out for separate rating. But once one understands the importance of the prediction of future behavior by contrast with the evaluation of achievement to date, the latter being all that goes into the evaluation of the researcher herself or himself, it is obvious that the prediction about sustainability of the previous performance level is a separate dimension of merit that must not only be considered, and weighted heavily, but also 'barred' that is, there is a certain level of probability that must be achieved on it before one can allow performance on the other dimensions of merit to produce an overall high rating for the proposal.