Fast and accurate scoring is a key component of successful testing programs. At Measured Progress, we score student responses to open response items using our proprietary, image-based scoring system, iScore™. When we receive answer documents back from schools, we scan them, creating an image of the entire document. Every page is bar-coded to link it to the correct student, school, and district, so that we can instantly find and identify student responses and send an electronic image of each response to one or more readers for scoring. Image scoring increases quality control, reliability, and productivity. iScore can provide groups of readers at one or multiple scoring sites with access to the images. Key features of imaging student responses include the ability to  

The scoring process actually starts when our test development staff creates a rubric, which describes the attributes of a student response that would earn the student each possible score permitted for that item. This, along with other preparatory work, such as benchmarking and developing training notes, ensures that each Measured Progress client can expect accurate, high-quality data from their assessment programs. 

Benchmarking sessions are designed to identify appropriate anchor papers (or exemplars) at every score point for each field-tested, open-ended item. (The released item in this issue shows how these might look.) Prior to the benchmarking session, a set of student samples is selected for each constructed-response item prompt. The scoring staff prepares this packet of student responses, which covers the range within each score point, helping to ensure adequate selection options for training materials. During the benchmarking session, participants determine the appropriate score for each sample response and select exemplars to guide the full scoring process. At this stage, additional rubric issues are resolved, and approval is given to all training, qualification, and calibration materials. A scoring pack is created for each item, to aid in the training and qualification of scorers. 

After test administration, the completed test booklets are scanned and the images routed to the iScore system. At the scoring center, the images of student responses are assigned to trained, qualified readers, who conduct the actual scoring.  Because the readers do not have access to student names or other biographical information, all items are scored without bias. Supervisors monitor reader performance continuously and in real time by reviewing pre-scored answer books. A percentage of student responses is read by two readers. In the event of a disparity in scores, a supervisor reads the item to arbitrate a resolution to the discrepant scores. When an item is read a second or third time (to arbitrate), the second or third reader does not know the score previously assigned, which means these second or third reads are also blind reads.  

Through the use of iScore, handling of test and answer documents is kept to a minimum. Scanned student responses are stored in secure areas with restricted access by personnel. Measured Progress employees are trained in and sensitive to the need for rigorous measures to maintain the security of all test materials. 

A core group of senior managers and chief readers guide and monitor the scoring process. Chief readers have advanced degrees and practical experience in their content areas; most are veteran teachers themselves. Our Measured Progress scoring staff is augmented during peak periods with additional supervisors, content specialists, and readers. All are trained to use scoring rubrics and benchmarks developed collaboratively with educators in client states. 

Measured Progress involves clients in benchmarking and rubric reviews. In addition, state-sponsored scoring institutes have enabled thousands of teachers to draw on scoring experience to substantially improve classroom instruction and student achievement.