Preface
From EvaluationWiki
DEVELOPMENT
1. The first edition of this glossary was assembled by Dale Farland, a staff member of The Evaluation Center at Western Michigan University, from the glossaries of the three standards books (The Program Evaluation Standards, The Personnel Evaluation Standards, and The Student Evaluation Standards, all published by Sage or its subsidiary, Corwin Press) plus the glossary that three of us did for the teacher evaluation project (CREATE) funded by the U.S. Office of Education through The Evaluation Center. Dale was able to arrange that all terms in the glossary would appear in (or be underlined in) green in all other documents on The Evaluation Center's Web site and when clicked would immediately jump to a definition in the glossary. We have retained that feature in one of the versions now up, the other being a single Word document that can be searched in one pass for other occurrences of the term sought or related terms, not possible in the other version. The first edition has been up on the Web page of The Evaluation Center for some years and was developed, as was this edition, with funding assistance from NSF. (This edition received four days' assistance from NSF and considerably more from The Evaluation Center and my own time.) During the years the first edition has been up, we have kept a record of the roughly 600 Search queries for terms that it did not contain. In this much longer version, I have added definitions for those terms that now are an established part of the lingo of professional evaluators and indicated where to find those that are essentially technical terms from statistics or other disciplines.
I also have added definitions of perhaps a hundred terms that were not requested but have, over those years, become part of the evaluation vocabulary; and I have rewritten all of the previous definitions for currency and consistency, so that they will apply throughout evaluation and not just to the students, programs, and personnel that are the subject of the original glossary.
About a dozen other glossaries of evaluation terms have been published in that time, and most have been examined in the search for omissions or improvements here. The present effort has, on the average, much longer entries, with more examples and summaries of relevant arguments than any other; and it has several times more entries. That's not too hard to do, it just takes work; the key question is whether it has fewer mistakes. It does avoid many of the mistakes in other glossaries but no doubt has its own, beyond a number picked up by some good reviewers. Evaluators are professionally committed to finding out about their errors and omissions, so please send them to me, with GLOSSARY in the title line, at michael.scriven@wmich.edu. We'll make corrections online and acknowledge sources within a few days, a big advantage of an online resource. Although I have written nearly 60,000 words toward the 5th edition of my Evaluation Thesaurus, doubling its original length, this possibility of quickly correcting errors was the deciding consideration that persuaded me to put that project aside, probably indefinitely, in favor of this one, now almost as long. But these were the main considerations:
(i) the feeling that it was time to get something comprehensive out for free access, if we want to carry the word about evaluation beyond the wealthier people in wealthy nations
(ii) the need to do something about the frequency of errors and the perfectly legitimate absence of explanations in other glossaries (I hope that their authors will immediately start work on finding the errors in this one!)
NATURE
2. This is still a glossary, one step up from a dictionary in that it covers more technical terms and technical senses of terms in its field, and one step down in that it does not cover other terms in the language. It's also one step down from a thesaurus, because it has shorter definitions although it has far more of them and less explanation, although in this case some of the entries are as long as those in most thesauruses. A thesaurus in turn is one step down from an encyclopedia or handbook, above which, in turn, are the specialized texts of the field. Glossaries are often very basic guides to technical language, and very useful in that role; but in this one we move to the other end of the allowable range, perhaps bridging the usual gap between glossaries and thesauruses. So we give more examples and, when it seems essential to understanding the term, the relevant arguments. Indeed, since it's an evaluators' glossary, in authorship as well as prospective readership, there are some evaluations in the accounts of a few topics, for example, in the entries on the major issues about causation and methodologies, where there is still much discussion in the evaluation field about the underlying logic and limitations of the quantitative/qualitative approaches to these topics. Still, for much detail, the reader will want to go elsewhere to thesauruses, handbooks, textbooks, etc. so we give some references either to validate the entries or as sources for further information. Where these are not provided, see the next segment.
RELATION TO INTERNET
3. At this stage in the development of information technology, it is often possible and surely best to avoid references that can be reached only by readers with access to an expensive library. That is one reason for moving in the direction of providing more material in this glossary and also for making it possible to say that the next level of reference in all cases where no other is specified should be to the Internet via Google. Fans of Yahoo, Exalead, Ask, or MSN know that those search engines can yield comparable and sometimes better results; but at the moment the most commonly used engine, by far, is Google, so we give it pride of place subject to later replacement.
DETAILS
4. Entries referring to semitechnical or nonstandard uses of a word or phrase, which often are put in quotes and hence would all be listed by Microsoft Word at the beginning of the glossary (e.g., "thick description"), are here placed in their normal alphabetical order, without the quotation marks, for ease of search. The justification is that this is a technical reference work, and these entries are not used with quotes in the technical literature to which this is a guide.
5. We use two terms that replace lengthy noun clauses. The term "evaluand" is used throughout to refer to whatever an evaluator is evaluating. This may be a program, product, policy, proposal, etc., depending on the subarea of evaluation in which one is working; or it may be a person (a patient, consumer, student, client, or subject) in which case the term "evaluee" is usually employed, partly to indicate the paramount relevance of considerations about the rights of human subjects.
6. Terms that essentially are technical terms from statistics or quantitative experimental design are sometimes defined because often requested, but are usually indicated as (Stat.). Terms from the tests and measurement area, which is closer to being an area of applied evaluation, are usually defined here. In both cases, if they are especially important in evaluation, we provide a definition stressing that use. The statistical terms are defined in any good statistics reference, including the glossary of many textbooks. Ten top statistics dictionaries (and their current prices) can be found at http://www.amazon.com/gp/richpub/listmania/fullview/20ZQ2ANCXGX0W/103-5322637-7314205?%5Fencoding=UTF8 .
7. Terms that are part of the English language and have no special sense in the evaluation context are not given an entry here, though they are quite often requested. Their absence should not be taken as a sign of oversight: they were probably included in the 800 requests for undefined terms that we are dealing with here; but, through the end of May 2006, for this revision of the glossary, they were still not identified as having any special sense in evaluation. The reader can enter the term or phrase (let's call it X) in Google's search box as "define:X" (don't add spaces or use the quotes), and you will get definitions from a number of sources. If you don't find what you need there, drop the "define:" and run the search again, and you will often find definitions in the context of examples that are sometimes much more useful.
8. Names of persons who have made contributions to the field are not listed as entries in a glossary by convention (unless, of course, there is an entry like "Likert scale" that was named after somebody) and can be looked up in books like Worthen, Sanders, and Kilpatrick; Alkin's Roots of Evaluation; or the Encyclopedia of Evaluation edited by S. Mathison. But you can find mentions of some people, via at least some of their publications or inventions, from an electronic search of the whole glossary if any of their publications or contributions are referred to here. (Of the dozen or so terms in here that I invented and that have passed into general use, I mention that fact in only one case and leave it to readers and reviewers to decide if that decision was a good example or a bad one.)
9. Sources for definitions are given when it seems important to document them, but not always. Where none are given, the source, as with most dictionaries, is the current practice of expert users, in this case collected by the author, partly drawing on his notes for a 5th edition of the Evaluation Thesaurus (4th edition, Sage, 1991) and on a huge exercise in searching other sources on- and offline. The draft text for this revision was also reviewed by Arlen Gullickson, former director of The Evaluation Center, another experienced evaluator familiar with current usage. Many thanks to him, not only for this valuable help, but also for considerable support for my time.
10. Terms that occur in bold face at their first occurrence in the text of an entry have a separate entry elsewhere in the glossary.
