TCAP scores not as bright after correction
The Douglas County School District’s 2013 Transitional Colorado Assessment Program (TCAP) scores aren’t as bright as school officials initially thought, a mistake attributed to an incomplete analysis of data.
Though the changes are not dramatic and DCSD distributed the new information, some parents think the district hasn’t adequately explained the error.
A special edition of DCSD’s online community newsletter, NewsLine, emailed on Aug. 15, included a chart showing scores increased from 2012 to 2013 in all four subject areas.
A second NewsLine sent Aug. 21 included a different, updated chart, which shows reading and math scores decreased slightly from 2012 to 2013, while writing and science scores increased.
Connie Davidson, one of at least three parents who questioned the initial NewsLine release, noted the follow-up newsletter doesn't alert readers to the fact that it includes different information.
“I’m disappointed that the accurate TCAP info was buried near the bottom of this new Newsline, and that it wasn’t identified as the correction of an error,” she said.
Another parent, financial analyst John Roth, contacted DCSD systems performance officer Syna Morgan with his concerns about the intial data. Roth is married to frequent district critic Susan Meek, a former DCSD spokeswoman and onetime school board candidate who now is actively involved in helping elect four new school board members.
The decreases are not “statistically significant,” Morgan said, which is defined as a drop of 3 or more percentage points. Even with the re-calculation, the district outpaced the state average more than 12 percentage points. And on the upside, the increase in science scores was remarkable, she said.
“Statistically significant or not, we don’t want any decreases,” Morgan said. “So we pay attention to that and address it in our district improvement plan. Our increase in science was statistically significant. In all other areas, we sustained high performance. (But) in no other content areas was there an increase that was statistically significant.”
Morgan said it’s a choice whether to “celebrate the high performance of Douglas County sustaining” or to point to negatives.
“We’re very serious about taking any decrease into consideration,” she said. “However, we also know that statistically, there will be a variance from year to year of one or two percentage points.”
The difference between the first and second set of scores stems from a new DCSD data analysis tool that left out scores from students no longer in the district, according to Morgan.
The data was “based on the current year’s enrollment, not the entire population tested last year,” she said. “So that’s where the error came in.”
Morgan said she carefully reviewed the text in a draft press release from the communications department, but did not thoroughly scrutinize the graphs.
“When we investigated why there was a discrepancy in the results, that’s when we discovered the only students that were pulled into the summary were the students that were enrolled this year,” she said, adding that she was upset by the discovery.
Morgan could not estimate how many students’ scores were omitted from the initial calculation. Students in grade 3-10 participate in the annual spring TCAP.
Though DCSD’s most recently released TCAP charts now match Roth’s, he remains skeptical about the accuracy of some district-released information.
“I guess our role in this day and age is to check them,” said Roth. “I’d love for (the initial) TCAP scores to be true. But it wasn’t the case.”
Davidson, a certified public accountant, expressed a similar sentiment.
“I’ve had a lot of concerns with many different issues that are going on, so that prompted me to look at the numbers myself,” she said. “It’s very concerning to me how much control they have over what gets out to people. I just want the truth out there.”