Is CASD Misrepresenting its Test Scores?

This week, the quarterly district newsletter, "The CASD Achiever" was published online, highlighting a number of different topics from facilities changes, STEM projects by building, an exciting award, and the topic that caught my eye, test scores. In this article, called "Future Ready PA Index Highlights District Achievements," scores were shown from select buildings that showed improvement in 2018.

Are CASD schools truly making the grade? Are the curriculum changes yielding positive results? My review of state data is not so conclusive. Achievement scores overall are below state averages at the elementary level, but are above state averages at the building level for some schools, and many schools show on-target growth in at least one area. Reeceville is a notable exception.

However, the data suggests that these results are not being sustained at the secondary levels, and that even the growth rates are faltering at the high school level. Math and science scores are the most alarming.

Yet the CASD Achiever article indicated that the district met or exceeded the statewide average in mathematics in five schools and the nationwide average for growth in mathematics in eight out of ten schools. It also claimed that the district met or exceeded the statewide standard for growth in ELA in seven out of ten schools. Further the article claims that the results show a "measurable increase in academic achievement following a realignment of the mathematics and ELA curricula over the last three years." Sounds pretty good, doesn't it?

The charts that accompany the CASD article do show improvement, but the "building level scores" label on the chart is ambiguous, and is not explained further. We don't know whether these scores are for math, reading, or science, or whether they are achievement or growth scores. So let's compare these to what I got when I went to various PA department of education websites to pull the district's results.*

Elementary Schools

District Numbers

State Numbers*

4 out of 6 schools are listed as improving numbers in 2018 vs 2017 East Fallowfield - 75.18% Friendship - 69.4% Kings Highway - 70.3% Rainbow - 56% Caln - not listed Reeceville - not listed

5 out of 6 schools showed growth in math and ELA. 10 out of 12 achievement scores were below state averages. State averages: Math - 45.5% ELA - 63% East Fallowfield - outperformed one state average, underperformed the other Math - 51.9%; significant evidence of growth ELA - 58.9%; evidence of growth Friendship - performed below state averages Math - 44.9%; moderate evidence of growth ELA - 59.4%; significant evidence of growth Kings Highway - outperformed one state average, underperformed the other Math - 46.3%; significant evidence of growth Kings Highway - 56.1%; significant evidence of growth Rainbow - performed below state averages Math - 32.9%; evidence of growth ELA - 43.7%; evidence of growth Caln - performed below state averages Math - 31.1%; evidence of growth ELA - 39.2%; evidence of growth Reeceville - performed below state averages Math - 23.2%, significant evidence growth targets not met ELA - 29.9%, moderate evidence growth targets not met District - performed below state averages District PSSA Math Overall: 28.8%; evidence of growth District PSSA ELA Overall: 45.3%; moderate evidence growth targets not met

After reviewing data on the Future Ready PA Index and the PVAAS (Pennsylvania Value Added Assessment System), which is the data reference tool Dr. Donohue told me the district uses, I could not find data that matched the charts/scores provided in the district newsletter. Achievement scores remain low at four out of six elementary schools tested in 2018, but most of the elementary schools show at least some evidence of improvement, with Reeceville being the notable exception. But as a district overall, we still have achievement scores that are well below state benchmarks, and only math scores show some evidence of positive growth.

Secondary Schools

District Numbers

State Numbers*

3 out of 4 schools are listed as improving numbers in 2018 vs 2017 (CAIHS is not listed and may have been combined with CASH) North Brandywine - 54.0% South Brandywine - 53.6% Scott - not listed CAIHS - not listed CASH - 58.2%

2 out of 3 middle schools showed growth in math. Only one showed growth in ELA. Only 1 out of 6 growth targets were met at the high school level. State averages: PSSA Math - 45.5% PSSA ELA - 63% North Brandywine -  performed below state averages PSSA Math - 22.4%; evidence of growth PSSA ELA - 50.0%; evidence of growth South Brandywine - performed below state averages PSSA Math - 27.1%; evidence of growth PSSA ELA - 49.3%; moderate evidence growth targets not met Scott -  performed below state averages PSSA Math - 16.6%; moderate evidence growth targets not met PSSA ELA - 38%; moderate evidence growth targets not met State averages: Keystone Math - 65.2% Keystone Literature - 72.7% Keystone Biology - 64.4% CAIHS - performed below state averages Keystone Algebra - 6%; significant evidence growth targets not met Keystone Literature - 48%; moderate evidence growth targets not met  Keystone Biology - 26.75; significant evidence growth targets not met CASH - performed below state averages Keystone Algebra - 41.3%; evidence of growth Keystone Literature - 57.4%; moderate evidence growth targets not met  Keystone Biology - 38.6 significant evidence growth targets not met District - performed below state averages Keystone Math Overall: 14.7%; significant evidence growth targets not met Keystone Literature Overall: 35.5%; significant evidence growth targets not met Keystone Biology Overall: 19.9%, significant evidence growth targets not met

Scores get worse when we move to the secondary level. Again, it's not clear what type of metrics the district was trying to provide, but leveraging the PVAAS for all the schools that show up on that site, and the Future Ready PA Index for the 11/12 building (CASH), we see that the results remain troubling at the high school level and far different than what was represented in the district newsletter.

The data seems to suggest that, to some extent, the elementary school curriculum may be having a positive impact on student growth, but this is not consistent across buildings and student growth cannot be specifically tied to curriculum changes. However, growth rates start to falter at the secondary level.

Is it possible that advancing students to biology without first taking earth science has been too ambitious? CPM math is used at the secondary level. If this curriculum is not allowing high school students to pass the algebra keystone on their first try, after several years of foundation within the CPM program, doesn't that warrant another look at whether this program truly meets our students' needs?

Not according to the district newsletter, where the unspecified data provided seems to convince Rita Perez, Director of Pupil Services, that "we are seeing continued growth over time, and students are making progress." That's what Ms. Perez sees. What do you see?

~ Written by Liz Muirhead

* Data primarily supplied by PVAAS, with gaps filled by Future Ready Index. State averages taken from PA Dept of Education Website for Keystones, and from the Future Ready Index website for PSSAs.