Indiana Department of Education Hires Third Party to Validate High Stakes ISTEP+ Data

Monday, June 10, 2013
Daniel Altman
Press Secretary
(317) 232-0550
daltman@doe.in.gov

INDIANAPOLIS – Today, Indiana Superintendent of Public Instruction Glenda Ritz announced that the Department of Education has hired the National Center for the Improvement of Educational Assessment (NCIEA) to determine whether testing interruptions experienced by students invalidated high stakes ISTEP+ test scores.

“Like all Hoosier parents, students, and educators, I was extremely frustrated with the alarmingly high volume of test interruptions during Indiana’s high stakes test,” said Superintendent Ritz. “These interruptions were simply unacceptable, and they call into question the validity of the testing data. These results have a large effect on schools and communities. School accountability, as well as teacher compensation and evaluation are based, in part, on this data. Because the stakes of this test are so high, the results must be beyond reproach.”

At Superintendent Ritz’s direction, NCIEA’S co-founder Dr. Richard Hill will conduct an independent analysis of the validity of the online tests taken by students that were seriously interrupted due to server errors. Dr. Hill has extensive experience in this type of analysis, and holds a Masters in Secondary Mathematics Education from St. Lawrence University and a Ph.D. in Statistics and Measurement from Syracuse University.

This analysis will be done completely independently from any analyses that are conducted by CTB-McGraw Hill. Dr. Hill will also independently review any findings by CTB-McGraw Hill. The cost of Dr. Hill’s review is not expected to exceed $53,600.

Additionally, Superintendent Ritz announced a tentative timeline for validity review and score reporting. That timeline is below.

ISTEP+ Tentative Testing and Validity Timeline

In recent weeks, Superintendent Ritz has taken several actions to work with schools regarding the administration of ISTEP+. A brief timeline of those events is below:

April 29, 2013: During the first day of online ISTEP+ testing, students throughout Indiana experienced interruptions. It was quickly discovered that the interruptions were caused by an error on CTB McGraw-Hill’s servers. After CTB’s online testing system experienced their first disruptions, Superintendent Ritz extended the testing window by two days to give schools additional flexibility to allow 482,000 students to complete online testing.

April 30, 2013: For the second consecutive day, an error on CTB McGraw-Hill’s servers caused widespread interruptions for Indiana students. In response, Superintendent Ritz suspended online testing for the remainder of the day to give CTB time to assess and fix the problems. Superintendent Ritz also extended the testing window by three additional days and asked schools to test at 50% of their normal testing load, based on the assurances of CTB. Because of these steps, the remainder of the ISTEP+ was completed with minimal interruption to Indiana students.

May 3, 2013: Superintendent Ritz conducted three separate conference calls open to all superintendents throughout Indiana to answer their questions about testing issues. Superintendent Ritz and the Department of Education also maintained constant contact with schools throughout the testing window.

May 17, 2013: ISTEP+ testing concluded for the 482,000 Hoosier children that took the test online.

May 24, 2013: The Department of Education provided schools with a list of 78,269 students that CTB indicated had testing sessions interrupted. The Department gave that list to local schools so that they could check the list against their records. On this date, the Department also issued a request for qualifications to three national companies experienced in validating test results.

June 6, 2013: Superintendent Ritz met with NCIEA’s founder Dr. Hill about reviewing the validity of online ISTEP+ testing.

June 7, 2013: Local schools throughout Indiana provided a list of additional students that experienced interruptions to the Department of Education. Students on these lists will be included in Dr. Hill’s review.

June 10, 2013: The DOE hired NCIEA as the independent third party validator of CTB’s online high stakes ISTEP+ test results. Dr. Hill immediately began Phase 1 of the validation process, which will consist of reviewing whether administrative testing problems materially affected student scores. Phase 2 of the analysis will consist of determining whether scores should be equated or adjusted based on the findings of Phase 1.