We are leaders in the development of the Accessible Portable Item Protocol (APIP) Standard—an interoperability standard for test item content and for storing student access needs.Today, we are delving into the work that is needed to support the implementation of APIP.
The concepts of interchangeable parts and interoperability are abundant throughout today's society. When the ink in our printer runs out, we replace the ink cartridge, not the printer. When the fan belt on our car breaks, our mechanic does not craft a new belt but instead replaces it with any of thousands that are identical in size and shape with the one that operates our car. Without interchangeable parts, we would need to replace the entire product any time an individual component malfunctioned.
In the field of educational testing, the concepts of interchangeable parts and interoperability standards are in their infancy. On the one hand, both adaptive testing and common item banks are premised on the concept of interchangeable parts. On the other, the testing industry has yet to fully adopt communication protocols that allow any item development or data banking system to communicate with any test delivery engine in a consistent way. This lack of interoperability creates a tremendous burden for testing programs that wish to transfer test items across systems.
Additionally, as states move from paper to computer-based assessments, new approaches are needed to provide students with special needs and English language learners with supports that enable them to have access to, interact with, and respond to test questions. In a paper-based testing environment, different versions of assessment materials are created (e.g. large-print materials for students with visual impairments, test materials in multiple languages, etc.), and teachers or other individuals present alternate representations of test content to students (e.g. teachers read aloud test content to students with visual impairments). In a digital environment, alternate representations of content can be built a priori into the test items and a digital test delivery system can tailor different representational forms based on each examinee's individual needs. But in order to do so, a standard format is required for attaching accessibility information to an item.
The Innovation Lab worked with the IMS Global Learning Consortium and eight states, spearheaded by the Minnesota Department of Education, to develop APIP - an interoperability standard that allows for test items and associated accessibility information to be ported between systems. The APIP development team sought to provide assessment programs and item developers with a structure for standardizing the file format of digital test items that could also be used to specify all the information and resources required to make a test item accessible for students with a variety of disabilities and special needs, as well as English language learners.
As illustrated in the figure below, the specifications within APIP cover three critical areas:
- Content: The accessible content, which provides a wide range of accessibility supports for digital test content.
- PNP: The user Personal Needs and Preferences profile, which documents the specific accessibility needs of each student.
- Delivery: The delivery system, which combines the user needs with the content to ensure the item is accessible for the student.
APIP enables test delivery engines to tailor the presentation of items to meet individual examinees' access needs. Some of these embedded access supports require only that the delivery system have the proper functionality (e.g. magnification, alternate contrast, increased white space, and answer masking). Other access supports require system functionality and also requires item developers to place a variety of types of access information within an item, including specifications for how an item is to be presented in auditory, Braille, sign, or tactile forms. In addition, APIP allows an item developer to point to alternate versions of the item that are presented in an alternate language (e.g., Spanish), in simplified English (e.g., with negatives removed), or in some other form.
The APIP interoperability standard is now embraced by the assessment vendor community (IMS Global Learning Consortium, 2012) and referenced in multiple state, consortia, and assessment vendor Request for Proposals (Smarter Balanced, 2012; PARCC, 2012; Colorado Department of Education, 2012; Georgia Department of Education, 2012). APIP serves as the most powerful, comprehensive, and advanced vehicle for adopting an industry-recognized interoperability item format standard that enables any digital assessment to be as accessible as possible for the widest spectrum of students. Its adoption across public education has the potential to transform the teaching and learning experience for millions of students, and not solely those with special needs.
Research and Development
With funding from the federal Department of Education Grants for Enhanced Assessment Instruments program, accessibility experts in the Innovation Lab worked with interoperability experts and eight states to define accessibility needs for next-generation digital assessments and develop technical specifications and associated documentation. APIP is based on three existing interoperability standards: QTI (provides standard XML language for describing questions and tests), Access for All (defines a common way for describing a student's needs and preferences in a digital environment), and Content Packaging (used to collate all of the QTI and accessibility information and to structure it in a convenient exchange format). This powerful combination of existing standards allows APIP to be a highly efficient interoperability solution for authoring and delivering accessible tests. Upon completion of the initial technical solution, the team aggressively disseminated project findings through web presentations, white papers, and an in-person meeting in Washington, DC attended by more than 100 state representatives, vendors, and federal government representatives. APIP is currently maintained by the IMS Global Learning Consortium as an open-source, open-license standard.
APIP provides the structure, format, and language to specify the exact manner in which tailored representations are to be provided by a test delivery system for a student's specific needs. Now that this technical solution exists, in order for computer-based testing platforms to deliver high-quality alternate representations of test content in a standardized and equitable manner, a set of guidelines for how to appropriately represent content in different forms is essential. The Innovation Lab has conducted a series of projects to meet this need.
Smarter Balanced Accessibility Guidelines: Under contract with the Smarter Balanced Assessment Consortium, accessibility experts in the Innovation Lab developed guidelines for the audio representation of Smarter Balanced English language arts and mathematics assessment items and recommendations for the use of American Sign Language (ASL)-based translation for the online delivery of Smarter Balanced assessments. The resulting work products represent the most up to date and comprehensive audio and sign guidelines for assessment content that is currently available.
MeTRC: With funding from the federal Office of Special Education Programs (OSEP), the University of Oregon's Mathematics eText Research Center (MeTRC) conducted research on eText supports in collaboration with researchers across the country. The Innovation Lab led three MeTRC research studies focused on the impact of audio representation of assessment items in mathematics. Innovation Lab researchers conducted cognitive labs and an experimentally designed study to explore differences in performance and preferences when literal and interpretive audio representations were applied to practice test items.
Guidelines for Accessible Assessments Project (GAAP): With funding from the federal Department of Education Grants for Enhanced Assessment Instruments program, the Innovation Lab is working with eighteen states, spearheaded by the Maryland State Department of Education, national audio and sign experts and educators, the National Center for Educational Outcomes, and WBGH's National Center for Accessible Materials to research and develop guidelines for sign and audio representation of CCSS assessment items. Innovation Lab researchers will conduct cognitive labs and a series of experimentally designed research studies to examine the extent to which audio and sign representations, developed according to the guidelines, remove construct-irrelevant barriers for students, affect student performance, and impact the test's psychometric properties. At the conclusion of the project, the audio and sign guidelines, sample item representations, and research papers will be publicly available for use by assessment consortia, states, districts, and assessment vendors. Read more about GAAP.
Cross Vendor APIP Collaboration
Innovation Lab staff members are actively engaged in two work groups designed to address APIP implementation issues. The APIP End User Group, facilitated by IMS Global, consists of vendors and state department of education representatives. The APIP Work Group, also facilitated by IMS Global, includes several state and testing industry members, including Measured Progress, Pearson, ETS, CTB/McGraw-Hill, ACT, and CAL.
Innovation Lab staff members are also engaged in the Common Education Data Standards (CEDS) project. CEDS is housed within the National Center for Education Statistics (NCES) and develops common data standards education data elements. Part of this project includes the development of the Assessment Interoperability Framework (AIF). AIF is a collaborative effort between the Race to the Top Assessment Consortia (PARCC and Smarter Balanced) and two leading education standards organizations (SIF Association and IMS Global). One goal of AIF is to develop an architecture plan, detailing how system components and interoperability standards interact to support next-generation assessments. SIF and APIP are two standards that will be part of the AIF architecture.
The Innovation Lab is planning to expand research and development efforts to include other representations of assessment content such as refreshable Braille, tactile overlays, and supports for English language learners.