Perspectives of Hands-On
Science Teaching

David L. Haury and Peter Rillero, 1994



6. How is hands-on learning evaluated?

Teachers understand the importance of evaluation and are expressing concerns regarding appropriate practices (Symington & Osborne, 1983). In all modes of instruction evaluation has important roles in efforts to assess student learning, to discover misconceptions among students, and to determine the effectiveness of programs (Doran & Hejaily, 1992). "As classroom teachers, we can praise hands-on experiential science, but until we can demonstrate that students are learning significantly more of the fundamental thinking skills of science, we cannot say that they have truly achieved science literacy" (Tetenbaum, 1992, p.12). Here we offer a variety of views on evaluation and assessment of hands-on learning and teaching.

Teacher Responses

• Hands-on learning in my classroom is evaluated by having the students tell me why certain events or situations occurred. If the proper answers are not given, I continue to ask more questions about the experiment and I give them clues as we go to help them figure out the answer .... Then I ask for volunteers to restate in their own words the concepts that we discovered. Finally, I ask questions at random to check one more time if they learned the objectives of the class. Bertha Vargas, fourth grade teacher, Gertrude M. Bailey International School, Lowell, MA
• Hands-on learning in science can be evaluated in two ways. One way is to have the student explain to you or to the whole class the process or experiment they experienced and why it works the way it does. If he or she can explain clearly and confidently what was to be learned, then the teacher can be confident they understand the concept. The other way to evaluate is to give short quizzes or tests to see if they can explain things in a written form.

I feel both oral and written expression are important for evaluating science. They need to be able to orally explain and write what they have learned. Laura Beyer, third and fourth grade teacher, Cassady Alternative School, Columbus, OH

Developer Thoughts

• To determine if a student is able to do science, he or she must engage in performance-based assessments. In this context the student works with materials to answer questions. The assessment must be designed so that the answers cannot be obtained by any other means. The materials used should be familiar to the student, but the context should be fresh. If one is assessing the ability to measure length, the student should be asked to discover the length of an object that he or she has not seen before. A good response on the part of the student would be to select a familiar meter tape and use it properly to measure the length accurately.... To determine if a student is able to communicate adequately a series of questions are asked that require the use of the vocabulary and discussion of the concepts developed in the science activities. Larry Malone, Full Option Science Program (FOSS) Co-director, Lawrence Hall of Science, Berkeley, CA

Notes from the Literature

• "There is considerable evidence to suggest that assessment can focus learning activities in science classrooms. How teachers assess students and what they assess has a major impact on the implemented curriculum. In most instances, test-driven systems result in classroom activities which emphasize rote learning of science facts and rote learning of algorithms to solve exercises similar to those which are included on the test.... In ideal circumstances, an assessment scheme should provide students with opportunities to represent what they know about identified aspects of science. At the classroom level, teachers are encouraged to use a variety of methods to assess student knowledge acquisition. These methods include traditional pencil and paper methods, personal oral interviews, and performance tests. The desirability of using a range of techniques is based on an assumption that much of the knowledge acquired in a hands-on and minds-on science program is tacit and has not been verbalized. Accordingly, although students can apply certain knowledge when they do science, they cannot necessarily reproduce that knowledge in verbal form on a pencil and paper test or in a discussion with the teacher" (Tobin, 1990, p. 411).
• The evaluation measures of SAPA, ESS, and SCIS lagged behind the curriculum development and implementation stage. Funds for the completion of the SAPA Science Process Instrument were not available, SCIS primarily focused on conceptual objectives, and ESS made no apparent effort to evaluate specific skills (Doran, 1990).
• "The problem of assessment also constrains the spread of 'hands-on' science. It is relatively easy to test children's knowledge when they have been asked to memorize lists of data from a text. It is much harder to design tests that measure learning derived from direct experience.... The challenge before science educators is to develop better means of measuring both factual knowledge and the kinds of understanding students acquire through activities. When that task is accomplished, a major roadblock to science achievement will have been removed" (William J. Bennett (as U.S. Secretary of Education), 1986, p. 28).
• "The most widely used method of collecting information on classroom learning is the written, or paper and pencil test. These tests are extremely useful for assessing student achievement on content objectives" (Meng & Doran, 1993, p. 7). Meng and Doran (1993) also present many examples (see figure 2 and figure 3) of standardized exam multiple choice items used to assess science process skills.

  ---

  
  Figure 2. Example of Measurement Item (Item 1).

  ---

  
  Figure 3. Identifying Properties of an Object.

  ---

• "Just as reading about scientific phenomena is not the best or only way to learn science, most educators agree that filling in bubbles on a standardized test is probably not the most effective way to measure a student's potential understanding of science" (Bruder, 1993, p. 23).
• "Among the new labels being used toady is performance-based assessment. Though there are a variety of definitions, it is clear that performance-based assessment does not include multiple-choice testing or related paper-and-pencil approaches" (Haury, 1993a, p. 1).
• "Instructional activities that use concrete manipulatives help teachers address a broad range of conceptual knowledge, scientific and organizing skills, and scientific and personal attitudes. Assessing learning goals across this broad spectrum requires a repertoire of assessment strategies that promote the fit between the dominant intelligences and desired educational outcomes" (Flick, 1993, p. 6).
• "At the elementary school level especially, but continuing throughout school and beyond, informal investigation is an important part of 'hands-on' science.... The teacher's evaluation of such activities is also likely to be informal, relying mostly on unobtrusive observations. Teachers may find it useful to observe systematically individual students, small groups, or even the class as a whole. The teacher's observations should be recorded in writing, either immediately or at the end of the day, noting the time, date, and activity. These remarks may be quite brief, even cryptic, but should specify in some way what was seen, not just the teacher's judgment of its quality. If the comments are recorded on index cards, say, they can be filed easily by student name, can serve as a record of progress and attainment to be used in planning further instruction, shared with parents, used in grading, and perhaps even shared with the students themselves" (Haertel, 1991, p. 241).
• The majority of teachers (Grades K-2) responding to a survey on the use of New York State's Comprehensive Instructional Management System Science Program report that the observational approach was useful for assessing students' strengths and weaknesses (Guerrero, Eisler, & Wilcken, 1990).
• Observational checklists are easy and flexible assessment tools. For instance, to measure a student's ability to draw conclusions the following scoring rubric could be used:

---

Points     Characteristics







0          Fails to reach a conclusion







1          Draws a conclusion that is not supported by data







2          Draws a conclusion that is supported by data, but fails



           to show any evidence for the conclusion







3          Draws a conclusion that is supported by data and gives



           supporting evidence for the conclusion (Nott, Reeve, &



           Reeve, 1992, p. 45).

---

• Livingston (1992) describes the use of the Science Activity Evaluation Form to measure activity-based science process skills. The Form is derived in part from an analysis of Benjamin Bloom's Taxonomy of Educational Objectives and an analysis of middle school science activities. The following potential student behaviors are listed from low-level skills to high-level skills: draw, identify, list, locate, observe, compare, describe, distinguish, outline, state, apply, build, test, analyze, classify, computer, graph, design, infer, interpret, conclude, explain, and hypothesize. Teachers are instructed to write a student name by a skill when they observe it in class. The Science Activity Evaluation Form can be used to evaluate the work of students. It can also be used to identify activities or curricula that only engage students in predominantly low-level skills.
• Laboratory practical exams have been used to assess student attainment of science process skills. For individual assessment, each student should perform all the manipulation of equipment and materials to answer questions (Stensvold & Wilson, 1993). "Questions can be administered in a rotation pattern so that one- third of the class does one question while the others do a second and third question. This practice increases the likelihood of individual testing and minimizes the equipment necessary for any one exam activity. Questions need to be planned so that equivalent amount of time is needed for each" (Stensvold & Wilson, 1993, p. 250). An example of an activity station is based on experiences with the ESS unit, Batteries and Bulbs. A sealed box has six metal knobs on top connected by hidden circuits. "From their testing of the box, students collect and interpret their data and hypothesize all likely patterns of the unknown circuit (more than one was possible). They are also required to provide a rationale for their hypothesized circuit(s)" (Stensvold & Wilson, 1993, p. 251).
• "A behavioral objective for a task helps focus the specific skills and materials and helps to establish scoring parameters" (Doran & Hejaily, 1992). Performance tasks for skills should not be paper-and-pencil items, but should involve students in doing activities. The directions must be clear and concise; diagrams can help with clarity. Questions should be based on the process skills identified. For example, "Write your observations for ..." or "Predict what will occur when ...." In developing a scoring system, performance and not content should be stressed as the most important aspect contributing towards a grade (Doran & Hejaily, 1992).
• Keep the performance items direct and simple, and not too long. Provide diagrams and clear instructions. Use materials which are familiar to the students. Remember to consider the manageability of the item with a classroom full of students. Develop a scoring rubric before having students complete the assessment. Discuss the scoring system with other teachers and try for consensus on how to award points (Finson & Beaver, 1992).
• Kanis, Doran, and Jacobson (1990) developed a Science Process Laboratory Skills (SPLS) test as an important part of the Second IEA Science Study. This test was given to a representative sample of American fifth grade students (n=2,392) and ninth grade students (n=2,248). The following three main skill areas are assessed with this instrument: performing, investigating, and reasoning. Performing skills on the test include observing, measuring, and manipulating. The investigating skill includes planning and designing of experiments, and the reasoning skill includes interpreting data, formulating generalizations, building and revising models. There are two forms of the fifth grade instrument, Set A and Set B, both are meant to be used simultaneously. Set A contains five performing skill items, two reasoning skill items, and no investigation skill items. Set B contains five performing skill items, five reasoning skill items, and one investigation skill item.
• The specific tasks for the instrument are as follows:
• Describe and explain color change of bromothymol blue solution after blowing through a straw.
• Cite at least three similarities and differences of two plastic animal specimens.
• Determine if four objects are electrical conductors by testing in a battery-bulb circuit.
• Predict and measure the temperature of the mixture of equal amounts of hot and cold water.
• Observe and explain the dissolving of coffee crystals in water.
• Determine which seeds contain oil by rubbing them on paper. (Doran, 1990, p. 26)
• States are beginning to play a role in authentic science process skills assessment. Wisconsin will use task-oriented assessment for mathematics, science, language arts, and social studies (Project prepares task-oriented assessments, 1993). According to Darwin Kauffman, the director of Wisconsin's Department of Public Instruction "Hands-on performance assessment tasks are on the cutting edge of what is going on in schools as far as assessment" (Project prepares task-oriented assessments, 1993, p. 2). New York State developed a mandated test called the Elementary Science Program Evaluation Test (ESPET), which was first administered to 211,000 fourth grade students. The manipulative skills section of this test has attracted national and international attention. According to Susan Agruso (1993), the Project Director, the ESPET manipulative skills section "consists of five hands-on stations with a total of 15 questions designed to evaluate student ability to measure physical properties, predict an event, create a classification system, make generalizations, and draw an inference. This assessment program has had a profound effect on science education in our elementary schools. Students are especially excited about the manipulative portion, enjoying the opportunity to engage in problem-solving activities that require their handling equipment and taking data" (p. 1).
• Technology may help in overcoming obstacles to performance- based assessment in science. The Scope, Sequence, and Coordination project of the NSTA has plans to test a compact-disc interactive (CD-I) system in assessment. The system "is designed to alleviate labor intensive tasks - such as recording and analyzing student responses.... The assessment would go like this: A student works alone, using the CD-I program as a guide to carry out tasks - at various levels of complexity - with real objects and phenomena. The CD-I tracks their responses to identify patterns of preconceptions, knowledge, and problem-solving techniques" (Bruder, 1993, p. 24).
• "As educators move toward more performance-based assessment, the potential advantages offered by portfolios must be considered. Growing out of authentic classroom practices, portfolios provide a holistic view of student performance. They allow for the alignment of instruction and assessment and provide the opportunity for students to be more closely involved in reflecting upon and assessing their own growth. They also offer a vehicle for increased communication among teachers and between home and school. Although the implementation of portfolios is labor-intensive and time consuming, the gains in terms of improved education seem to warrant their consideration as part of the assessment process" (Newman & Smolen, 1991, p.32).
• "At the elementary school level especially, but continuing throughout school and beyond, informal investigation is an important part of 'hands-on' science.... The teacher's evaluation of such activities is also likely to be informal, relying mostly on unobtrusive observations. Teachers may find it useful to observe systematically individual students, small groups, or even the class as a whole. The teacher's observations should be recorded in writing, either immediately or at the end of the day, noting the time, date, and activity. These remarks may be quite brief, even cryptic, but should specify in some way what was seen, not just the teacher's judgment of its quality. If the comments are recorded on index cards, say, they can be filed easily by student name, can serve as a record of progress and attainment to be used in planning further instruction, shared with parents, used in grading, and perhaps even shared with the students themselves" (Haertel, 1991, p. 241).
• Portfolios are increasingly becoming an important tool in science teaching and learning. A portfolio is a collection of documents that contain evidence of achievement. Evidence presented in the portfolio may be worksheets, laboratory reports, raw data, first drafts, or diagrams of laboratory equipment. It is very important that the evidence is meaningful to the students and it is recommended that students attach captions to each entry and state what the evidence is and why it is evidence (Collins, 1992).
• "Assessment through student writing and portfolios provides a means for teachers to help students take a longer view of their learning in science. Portfolios are not just collections of student writing but depend upon written products by students to interpret, explain, and generally give the student's perspective of the collected works.. These works may be represented as by photographs, diagrams, drawings, and even hi-tech records such as computer documents or video and audio tapes...The portfolio itself is not the assessment. Assessment comes from teacher judgment of the materials with respect to a set of criteria" (Flick, 1993, p. 5).
• "The development of portfolios allows teachers and students to work and learn together; provides opportunities for reflection and self-assessment; helps redefining [of] traditional student and teacher roles in relation to the science curriculum; emphasizes the culture in which teaching and learning occurs; and empowers both students and teachers with respect to science learning" (Tippins & Dana, 1992).
• Swang (1993) has his sixth grade science students keep two types of portfolios - work portfolios and exhibit portfolios. Students maintain items for both portfolios in printed form and on computer disks. Swang uses the work portfolio to assess student work in progress. This includes rough drafts of steps in scientific research, class notes and research notes, data tables, notes from scientific literature on a topic of study, and notes of encouragement from the teacher. Swang uses the work portfolio to monitor student work and understanding. However, no grade is given to this portfolio. By working with the student the quality of work is improved so that it can be placed in an exhibit portfolio. The student prepares a more formalized report for this portfolio. The quality of work has already been established so the grade for the exhibit portfolio is an "A."
• Concept maps my be an effective way to assess learning from hands-on science. Markham, Mintzes, and Jones (1994) conclude from a study of college biology students that concept maps are a powerful and psychometrically sound tool for evaluation in science education.
• Hein (1987) recommends a variety of approaches be used in assessing hands-on learning, including observing students at work, examining the things they manipulate, and evaluating science- related drawing and writing. Other assessment techniques include group discussion, journaling, and student interviews (Gaffney, 1992; Tippins & Dana, 1992). Some assessment task should be done by student teams to help build group skills (Small & Petrek, 1992). It is often beneficial to have students score their peers' group work (Culp & Malone, 1992).

Summary

It is clear that the traditional paper-and-pencil, multiple-choice approach to testing cannot be used alone to adequately assess the full range of learning outcomes typically associated with hands-on learning in science. Just as hands-on learning involves activity, demonstration of experientially based learning requires some degree of active expression beyond responding to a small set of standardized test items. Recommendations include more frequent use of verbal explanations, using assessment strategies that incorporate performance tasks, developing observational checklists and scoring schemes, and compiling portfolios of student work. It is acknowledged that these enriched forms of assessment require a greater investment of time to develop, administer, and interpret, but there is also a great need to more carefully align student assessment with curricular aims, instructional practices, and performance standards.

Posted to the North Central Regional Educational Laboratory's
Pathways to School Improvement Internet server on July 10, 1995.

Contact: info@ncrel.org