Load testing failures disrupt state exams

Inadequate software load-testing this week derailed school examinations across the US state of Florida.

From Monday, thousands of students aged 13-16 tried to log-on to computerised exams, but more than 2,000 found the connections so slow as to be unusable. Schools postponed the exams as a result. This created concerns over fairness, given that some candidates nevertheless managed to see the questions.

The failures were embarrassing for state officials, not least because Florida school superintendents had raised doubts over the load testing, which they were required to conduct themselves. Last week the Tampa Bay Times newspaper quoted one superintendent, Carlton Faulk, who questioned whether the infrastructure trials had achieved sufficient coverage: “While we believe our technology is currently ready, there has not been any true load testing mimicking the testing during the actual open testing windows.”

Others were concerned by the system’s need for stable internet connections; the limited number of compatible devices, which required students to be tested in batches; and the disruption caused by critical software updates to Java. More broadly, educators feared that the exams were more a test of students’ computer literacy than their academic knowledge, and that those with poor IT and keyboard skills were placed at a disadvantage regardless of the subject matter.

Florida’s Education Commissioner Pam Stewart announced that the log-in server seemed to be the main culprit. She said that the exam supplier, American Institutes for Research (AIR), had worked with its hosting provider Rackspace to address the problem. After software modifications, AIR performed an additional load test on the hardware that seemed to suggest improved performance.