Innovative Test Yields Innovative Reports

Measured Progress Partners with Three States to Develop a Dual Reporting System

Measured Progress and the New England Common Assessment Program (NECAP) states have collaborated on a unique dual reporting system to provide educators with an additional dimension of data. Thanks to the new reporting system, schools and districts in the NECAP states—New Hampshire, Rhode Island, and Vermont―will be able to look at students' current and past performance. 

An Innovative Design 

The NECAP is administered in the fall and assesses students on their learning during the previous school year. NECAP states have found that there are several advantages to fall testing. In addition to simplifying the requirements of meeting federal mandates and providing results at a more useful time, "fall testing has the advantage that it occurs after students have completed the entire instructional sequence for a particular grade level," said Michael Hock, Vermont director of educational assessment. "One of the major reasons behind the decision to move to fall testing relates to exactly what you are assessing," Hock said. "Fall testing measures what I call ‘deep learning;' that is, if you include a break between when students are taught the material and when they are assessed, you start truly looking at retention." 

Because the fall-administered NECAP assesses students on the previous year's learning, test results are valuable to both students' current- and prior-year teachers. Measured Progress has worked with the NECAP states to establish a system that provides schools with assessment results for both "testing" and "teaching" years. The testing-year results include information about the students schools will be teaching during the upcoming year. The teaching-year results help schools to evaluate the performance of their students after a year of learning. 

"As an example," said Tim Crockett, vice president of client services at Measured Progress, "when students are tested in the fall of their seventh-grade year, they are really being tested on sixth-grade standards. So, their results are useful to two different schools—the school they are currently attending, where the results can be used to plan for the year ahead, and the school where they actually learned the sixth-grade material, which can use the results to adjust curriculum and improve instruction." 

Hock called the reports, "an innovation to the assessment system." He said, "We want people using these test scores to be able to take a look forward and a look back. We want them to be able to plan for the upcoming year and also to look back and measure how the curriculum is working." 

Dual Reporting 

As the NECAP partners worked together to design reports of assessment results, the states discussed how valuable it would be for schools to know not just how well-prepared their current students were for the upcoming year, but also how well-prepared their previous year's students had been. "The more we thought about reporting for both years, the more it made a lot of sense," said Harold Stephens, Measured Progress NECAP program director. 

Thus, the NECAP partners developed a plan to establish a dual reporting system. In the system, each individual student is assigned two sets of school and district coding information. Measured Progress provides codes for the testing year based on information gathered from test materials returned by schools. The states add teaching-year information based on their historical data for the students. Measured Progress uses the two codes for each student record to generate reports for each code. 

Planning and establishing the system hasn't been simple. "It has taken an enormous amount of time and effort on the part of the states and Measured Progress to put this all together," said Crockett. "A lot of the legwork is on the states' part," added Stephens, noting that the state partners expend considerable effort in gathering the student information that makes this reporting system possible. 

Despite the effort involved, there is no doubt that the process of moving to dual reporting was worthwhile. "What it comes down to is that the dual reporting system allows the teachers who had these students for a year to find out how well they did in preparing their students," said Stephens. "It was really important to the three states that schools receive reports on how well they are preparing their students to move on to the next grade level." 

Making the Reports Useful

Both Measured Progress and the states wanted to ensure that teachers and administrators would use the two sets of results with confidence. Stephens said, "Our biggest concern was making sure that the educators would be able to use the reports in a way that would be helpful to them. We didn't want anyone to be overwhelmed when they received, literally, twice as many testing reports. And so we had to educate the field." To address that need Measured Progress runs a number of report interpretation workshops annually in each state, as well as providing a service center and interpretation guides, to ensure that educators are prepared to make the best use of the test results. 

Test results are reported online, using a NECAP reporting site modified to meet each state's requirements. The reporting sites include both information for the general public and confidential information for schools and districts. With the exception of individual student results that are sent to parents and schools, all NECAP reporting is conducted online, because it is quicker and more efficient than shipping paper results, and also ensures that results are easily accessible. So far, this combination of training and online reporting has been successful; educators in the three states have found the test results to be useful. 

A Valuable Tool

Educators who access the NECAP reporting site can view results for both teaching and testing years. Among the most valuable tools on the site, according to Stephens, are the Item Analysis Reports and released item documents. Educators can identify patterns in student responses from the reports and consult the item documents to examine more detailed item information. Item Analysis Reports provide student responses for a selection of NECAP items. Educators can see from the responses whether most students answered an item correctly or incorrectly. "What can be really useful here is when a significant number of students entered the same incorrect response," said Stephens. "This is great for discovering patterns of common misconceptions and enables educators to adjust the curriculum to account for those." 

This process works for both teaching and testing year results, Stephens said, because it allows educators in the teaching year to discover curriculum areas that had not been optimally covered and to adjust instruction accordingly. Educators in the testing year are able to see the gaps in their students' knowledge in real time, enabling them to adjust instruction as necessary. 

According to Hock, another advantage of the dual reporting system is that, "Even though the students may no longer be in the building, these reports enable principals to see how the students are doing." He said, "What I have heard from the teachers who are using the reports is that they appreciate being able to see the results even for students who have moved on to other schools." 

The design of a reporting system is a crucial factor in any testing program. "In the end, you can have the best test in the world, but if the results don't give teachers useful information that is accessible to them and can help them be more effective, then the test is nothing but an NCLB scorecard," Crockett said.

Sarah Connell