Student writing sample
[19k]
The full report, authored by Claudia Gentile, is titled Exploring New Methods for Collecting Students' School-based Writing, and was issued in April 1992 by Educational Testing Service under contract with the National Center for Education Statistics.
For ordering information on this report, write:
Education Information Branch
Prepared by Educational Testing Service for the National Assessment of Educational Progress 1990 Writing Assessment under contract with the National Center for Education Statistics, Office of Educational Research and Improvement U.S. Department of Education.
For Sale by the U.S. Government Printing Office
Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-9328
ISBN 0-16-036189-3
However, we face a problem when we try to assess the extent to which these efforts are successful. Traditional methods of evaluating students' writing (in particular, the timed essay test) are designed to measure a specific facet of writing ability--how well students can write on an assigned topic under timed conditions.[3] They are not designed to capture the range and depth of the writing processes in which students engage during process writing instruction programs.[4]
It is possible to emulate aspects of the process approach to writing within the context of traditional writing assessment methods. For example, the time allocated for writing can be increased, and can even be held over several days to allow for peer review and other classroom activities (e.g., New Brunswick, Canada Reading and Language Arts Multi-day Assessment Program).[5] However, holding an assessment over several days poses operational difficulties, increasing the costs and complexity of assessments.
Another way of establishing stronger connections between process writing curriculums and assessment methods is to adapt an instruction tool--writing portfolios--for assessment purposes.[6] Recently, schools, districts, and states have been exploring ways of using classroom writing portfolios to assess students' writing achievements. Using the writing students have produced as they engage in process writing programs establishes an immediate connection between the assessment and the writing process curriculum.[7] Recent efforts to adapt writing portfolios for assessment purposes can be classified into three types: the classroom portfolio, the combination portfolio, and the assessment portfolio.
The Classroom Portfolio
While Classroom Portfolios differ from classroom to classroom, they usually share several key characteristics. During the school year, as part of their English/language arts classwork, students collect their written work in folders. At specific points in the term, they review their work and create a portfolio by engaging in a process of reflection, selection, and description. (e.g., New York City Portfolio Project, ARTS Propel).[8]
The reflection and selection stages are guided by a set of criteria developed by teachers and/or students, based on the writing curriculum they are following.[9] These criteria often focus on the depth of student writing (writing that demonstrates the use of process strategies and writing that shows growth over time) and on the breadth of students writing (writing that illustrates the range of activities in which students have engaged).
Often the students determine how many pieces to include in their portfolios, with a minimum of three being common practice. A central element of these portfolios is the letters or statements students write explaining their selections and how their choices meet the selection criteria. This process of reviewing and evaluating one's own writing and then articulating one's decisions is considered central to the portfolio experience because it fosters students' development as writers.[10] The classroom teachers assist students throughout this process and also evaluate the portfolios. Sometimes other students, friends, and family read and comment on students' portfolios.[11] Students may collect portfolios for part of the year, the whole year, or over their whole academic careers, for one class or all classes.
The Combination Portfolio
The second type of portfolio assessment system uses a combination of approaches to collect writing from students (e.g., Vermont Portfolio Project).[12] In addition to asking students to assemble a portfolio from the work they have collected for their classes, students are asked to select a "best piece" and to include in their letter describing their portfolio an explanation of what makes this their best effort. Students may also be asked to complete a writing activity common to all students in a particular class or group. These three components--portfolio, best piece, and common piece--are then evaluated individually by one or more teachers and evaluative information is presented on each component, resulting in a profile of an individual student's writing achievements. Summary statements to students about their entire portfolios are also made by their classroom teacher, other teachers, and/or other students.
The Assessment Portfolio
The third type of portfolio assessment system involves administering several common writing activities to students (e.g., Rhode Island Portfolio Project).[13] Committees of teachers design a series of multi-day writing activities that reflect their writing curriculum. On the same days, using the same administration procedures, the teachers have their students engage in these activities. They collect the students' work in folders and have the students review their work and write letters explaining which activity yielded the best writing and from which they learned the most. A committee of teachers then meets to score the students' responses to each activity. The result is a profile of each student's achievements relative to the common tasks. This type of portfolio differs from traditional essay assessments in that the activities are designed to match a specific school's or state's curriculum and the students' work is accomplished as part of their regular classroom activities rather than under standardized assessment conditions.
The 1990 NAEP Pilot Portfolio Study
In keeping with these new developments, the National Assessment of Educational Progress (NAEP) has begun exploring alternative methods of assessing students' writing achievements--methods that focus on the writing students regularly produce as part of their classroom activities. NAEP conducted a pilot portfolio study in 1990 in order to explore the feasibility of conducting large-scale assessments using school-based writing. The main purposes of this pilot study were: (1) to explore procedure for collecting classroom-based writing from students around the country; (2) to develop methods for describing and classifying the variety of writing submitted; and (3) to create general scoring guides that could be applied across papers written in response to a variety of prompts or activities.
To this end, a nationally representative subgroup of the fourth and eighth graders who participated in NAEP's 1990 writing trend assessment was asked to work with their teachers and submit one piece of writing that they considered to be a sample of their best writing efforts. The goal was to create a "Nation's Portfolio"--a compilation of the best writing produced by fourth and eight graders in classrooms across the country.
NAEP analyzed and summarized these samples of writing along with teachers' descriptions of the assignments that produced them. In addition, NAEP compared students' school-based writings to their responses on the 1990 NAEP writing assessment to examine relationships between these two modes of assessment. This report describes the procedures used to collect, describe, and evaluate the school-based writing in this special pilot study. The 1990 writing assessment was a trend assessment--prompts that had been developed for the 1984 assessment, and readministered in 1988, were also given in 1990 in order to measure changes in students' writing achievements across the six-year period. In 1992, NAEP will continue the writing trend assessment, as well as conduct a new writing assessment comprised of informative, narrative, and persuasive writing prompts developed specifically for the 1992 assessment. While the trend writing assessment has not changed since 1984, the new 1992 writing assessment reflects recent developments in the field of writing instruction and assessment. For example, the time allocated for writing has been expanded to 25- and 50-minute periods. Also, a planning page has been included after each prompt, to encourage students to reflect and plan their responses to the topics. The 1992 assessment will also include a revised and expanded version of the 1990 pilot study and participants will be selected from among those students taking the new regular writing assessment.
Collecting Students' Writing
The Participants
Approximately 4,000 students who participated in the 1990 NAEP writing assessment--2,000 students at grade 4 and another 2,000 students at grade 8--were invited to participate in the special portfolio study. Based on traditional NAEP sampling procedures, this group would have been a nationally representative sample of the nation's fourth and eight graders.
However, only 55 percent (1,110 students) of the fourth graders and 54 percent (1,101 students) of the eighth graders and/or their teachers accepted this invitation. While these response rates provided enough papers to permit an analysis of the writing submitted on a pilot basis, as statistical samples they were too small to make generalizations about all of the nation's fourth and eighth graders' writing performances.
While the participants did not represent a national sample of students, they were from all of the major geographic regions and from various types of communities, including rural, suburban, and inner city. They represented a variety of racial/ethnic backgrounds as well as a balance between males and females (see Appendix A for details on the demographic characteristics of the participants).
Compared with the entire group of students who participated in the 1990 NAEP writing assessment, the participants of this study differed in some respects. Slightly higher percentages of the portfolio pilot study participants:
The Procedures
In the spring of 1990, at the time of the NAEP writing assessment, the English/language arts teachers of participating students were asked to help several of their students choose a sample of their own best writing from the work the students had completed so far in the 1989-90 school year. No more than 10 students from any given class were selected to participate. Teachers were asked to encourage their students to choose pieces that had involved the use of writing process strategies (such as revising successive drafts, using reference sources, consulting with others about writing). NAEP also asked teachers to attach a description of the activities that generated the students' writing and to comment on any process strategies the students used to produce their writing.
Teachers then submitted their students' writing to NAEP, along with a copy or description of the activities that generated the writing and any available drafts or prewriting samples. These pieces were used to create two national portfolios or collections of students' classroom writing--one containing the writings of fourth graders and the other containing the writings of eighth graders.
Unfortunately, due to the complex procedures NAEP employs to select students to participate in its assessments, we were unable to inform teachers at an early date which of their students would be participating in this study, with some teachers receiving only several days' notice. Thus, for the pilot, teachers and students did not have much time to review the students' writing and select best pieces. Based on this experience, a procedure for giving teachers more advance notice of the upcoming portfolio assessment was developed for the 1992 NAEP Portfolio Study. It is hoped that, by giving the participating teachers in 1992 several months' notice, the 1992 results will be representative.
Outline of this Report
This report is divided into four sections. Chapter One describes the writing received from the students and information from participating teachers about the activities that generated the writing. Chapter Two explains the procedures used to evaluate the writing students submitted as well as the results of this evaluation. Chapter Three compares the results of the NAEP 1990 writing assessment with the analysis of participants' school-based writing samples and summarizes the lessons learned from this portfolio study. The last chapter contains a set of sample papers, further illustrating how the evaluative guides can be applied and presenting a sense of the range and depth of writing we received from participating students.
Examples of the Narrative Scoring Guide
Event Description (score of 1)
Papers classified as event descriptions tell about one event. Basically, they say, "such and such happened." Some of the papers in this category give details about the setting and so appear to be more elaborate stories. However, they end with a description of a single event, rather than a series of events. The paper below, written by a fourth grader, is an example of a simple Event Description.
Papers classified as Undeveloped Stories tell about a series of events. Basically, they say, "one day this happened, then something else happened, and then another thing happened." However, they events, as well as the setting and characters, are only briefly described. The writers give very few details about each event; the story is a listing of related events.
These stories are similar to front-page newspaper reports, where the basic facts of a story are reported (who, what, when, where) but few details about why events happened are presented. For example, in the paper below, the fourth grade writer uses one sentence to describe each event.
In papers classified as Basic Stories, the writers go one step beyond a simple listing of related events. One aspect of the story (the events, the characters' goals, or the setting) is somewhat developed. However, these stories lack a sense of cohesion and completeness. Events may be presented out of sequence, some aspect of the story may be confusing due to problems with syntax, or a key event may be unclear. For example, in the paper below, the fourth-grade writer describes a series of events and, at the beginning develops a problem in some detail (a librarian who puts books away too quickly). However, the resolution to the problem, although humorous, is not well developed.
Extended Stories go beyond Basic Stories in that many of the events in these stories are elaborated to some degree. This degree of development gives a sense of a sequence of distinct story episodes. Details are given about the setting, the characters' goals, problems to be solved, and the key events. Yet, these stories may be somewhat incomplete in that the characters' goals may be left unresolved or the problem posed in the story's opening never solved. The ending may not match the beginning or the story's ending may be inconsistent with the internal logic established throughout the rest of the story. Or, as in the example below (written by an eighth grader), they may be very satisfying, yet not elaborately developed.
It is important to note that, while Extended Stories are not as elaborated or complex as are Developed Stories and Elaborated Stories, they are successful stories--all of the key story elements and events are clearly presented. They are the simplest type of complete story on this scale.
Developed Stories describe a sequence of episodes in which almost all of the events and story elements are somewhat elaborated. Yet, one aspect of these stories is not well developed, such as the ending or a crucial event. In the example below (written by an eighth grader), each episode is somewhat developed, but could be further elaborated.
No papers were considered to be Elaborated Stories. To be classifies as elaborated, stories had to present a sequence of episodes in which almost all of the events and story elements were well developed. Goals or problems introduced in the beginning were well resolved by the end, characters' motives were well developed, and the entire story was a cohesive, unified whole.
In the example below, the eight-grade writer of "The Black Rose" retells the plot of a Halloween movie. In it, the writer effectively presents each episode, leading to a spine-tingling ending. The only discordant note is the occasional s2witching of narrative voice between first person and third person. A revision of this story that included a consistent use of narrative voice would make this an example of an Elaborated Story. (As is, this story received a score of 5).
Student writing sample
[14k]
Student writing sample
[24k]
Student writing sample
[34k]
Student writing sample
[45k]
Student writing
sample [116k]