Visiting Scholars at Measured Progress Promote Joint Psychometric Ventures

Bill Stout and Lou DiBello shared the promise of “the engineering science of diagnostic assessment” when they spoke at Measured Progress in February as part of the company’s Visiting Scholars program. 

Stout and DiBello, professors at the University of Illinois and co-founders of Applied Informative Assessment Research Enterprises, are working to advance psychometric rigor in diagnostic score reporting. Currently, skills diagnostics typically appear on reports as simple sub-scores and give an inaccurate assessment of a student's skill level.

Stout and DiBello are collaborating with Measured Progress’s Research & Analysis group to apply their research to the company’s assessments—both statewide, large-scale programs and the Progress Toward Standards Online Assessment (PTS3). “Our primary interest is to develop the engineering science of diagnostic assessment,” said Stout. “In addition to science and theory, we are also focused on practical issues: costs, production, sustainability, scalability, implementation, evaluation, and dissemination.” 

“Working with Measured Progress will allow us to turn years of theoretical work into developing assessments that accurately measure skills, providing an evidentiary system that will give our educators more meaningful information about each student,” added DiBello. 

Improved diagnostic feedback will be especially relevant to PTS3, an interim, classroom assessment for grades three through eight and high school. Stout and DiBello addressed how such applications could be implemented, using analysis models and software their team has already developed. 

PTS3 is intended for use as a benchmark assessment, administered periodically through the school year to see how students are progressing toward standards. Respectively, diagnostic testing is best conducted during the school year, rather than at the end, so teachers can adjust student instruction in real time.

In contrast, state testing programs are summative assessments given at the end of the school year. While it is beneficial to receive diagnostic information from such assessments as well, teachers receive the information too late to have an impact on students' achievement levels in this grade.

“We are delighted to explore these exciting new diagnostic techniques with professors Stout and DiBello,” said Louis Roussos, a Measured Progress psychometrician and former colleague of the visiting scholars. “I hope we are able to apply these methods to testing programs developed at Measured Progress in the not-too-distant future.” 

Roussos, a major contributor himself to the skills diagnosis body of research, invited Stout and DiBello to speak to staff members from all Measured Progress departments involved in test development and analysis. “For skills diagnosis to become a reality in any testing program, it will require much more than mere psychometric expertise,” he said. “It will require a true collaboration among the various teams that develop a testing program.” 

To view Stout and DiBello’s presentation and learn about previous visiting scholars and their presentations, please visit our Measurement Research site.

 Lou DiBello and Bill Stout answer questions after their presentation at Measured Progress.