The narrator of The ABCs of School Testing describes the need for reliability when scoring writing samples (Joint Committee on Testing Practices, 1993).
"In spite of the advantages of multiple choice tests, though, there are some things they can't do. For example, it would be pretty hard to figure out if a student were a good writer just by using a multiple choice test. So, a lot of districts and even entire states collect writing samples from their students, but scoring these samples requires judgments. It isn't a matter of it just being right or wrong, so the people who score the papers have to be trained in what to look for so they can consistently tell good samples of writing from bad ones, and agree on what score to give each paper."
Date posted: 1995