Student Test Score Study Presented to State Board of Education
OKLAHOMA CITY (Aug. 20, 2013) – The Oklahoma State Department
of Education announced today that the independent study to determine if student
tests scores had been compromised by the two days of disruption this past
spring is complete.
The HumRRO Statistical
Investigation of Oklahoma Disruptions indicated that students appeared to do as
well on the test as students not involved in the disruption. Based on the
result of this study, State Superintendent Janet Barresi determined the state will
retain all scores of impacted students who scored proficient or advanced.
While students as a whole did
not experience depressions in scores, it is possible that some individual
students did not perform to their highest potential during the period of
disruption. Therefore, Superintendent Barresi will not report the scores of impacted
students who scored limited knowledge or unsatisfactory.
The full report was provided
by Dr. Arthur Thacker, Program Manager for Human Resources Research
Organization (HumRRO), Inc. to the department and delivered during Tuesday’s
State Board of Education’s meeting by Dr. Maridyth McBee, Assistant State
Superintendent of Accountability and Assessment.
“I’d like to thank Dr.
Thacker and HumRRO for conducting this study,” said State Superintendent Janet
Barresi. “Last year’s testing period was very difficult for our students,
teachers and testing coordinators. The difficulties they experienced were
unacceptable. It was a high priority for everyone to commission an independent
study to determine what effect the disruptions had on the student test scores.
Now that the study has concluded, we will work with the districts to take
action on behalf of students and schools where the disruptions merit such
action.
“Even though this study
suggests no systematic impact on test scores, not reporting the scores of
students who scored limited knowledge or unsatisfactory will ensure there is no
lasting impact on student performance. This is the right thing to do for
students and for schools.”
Server capacity problems by
testing vendor CTB/McGraw-Hill on April 29 and 30 caused a number of students
to be interrupted during their testing experience, making it necessary to
determine the extent to which the disruptions impacted students’ test scores.
The State Department of Education asked HumRRO to conduct a third-party
independent study to investigate the impact.
HumRRO was selected by the
SDE because the organization is recognized for their objectivity and independent approach to
evaluating the performance of testing vendors. They have also conducted similar
work for the State Department of Education during a previous challenge with a
different testing coordinator.
A state
Technical Advisory Committee of researchers and assessment experts who meet
twice per year to advise the State Department of Education on accountability
and assessment issues recommended the design for HumRRO’s disruption study.
HumRRO conducted the study independently of the SDE and CTB/McGraw-Hill.
Studies
conducted for Indiana by another organization found similar results, as did a
study conducted for Minnesota by HumRRO.
“The findings of the report
in relation to the discrepancies did not merit action in HumRRO’s opinion,”
said Superintendent Barresi. “Because the two-day incident resulted in so much
stress and chaos of the school day I wanted to assure students, teachers and
schools were not held accountable for circumstances beyond their control.”
Additionally the following
actions were taken to address discrepant scores:
- About 1,400 students received an immediate raw score
that differed from the score they received on the two-week preliminary test
score report.
-
The discrepancy occurred for two reasons:
o There appeared to be a short time period on the second
day of the interruptions when some student responses to test questions were not
saved and therefore not included on the two-week report. o CTB inadvertently left the “winter test system” as an
option for download which caused incorrect data to be saved.
Corrective action will
include:
- Students will receive the higher score between the raw
score and the two-week report score.
- Students with higher raw scores will receive a letter
from CTB indicating their proficiency level. They will be included in the
report card, but not in the school or district summary report.
- Students with a higher two-week score will receive a
traditional score report that includes performance level and performance by
content standard.
-30-
|