The Data Is In ... But How is it Being Used

10/26/2019

At the October 22, 2019 Board meeting for USD 500 Dr. Charles Foust presented his report regarding the results of the State of Kansas evaluation of testing for the 2018-19 school year titled, “SY2018-2019 Outcome Data” The entire report can be found at a link at the bottom of this post.

Congratulations are in order for staff and administration that developed the curriculum for the district; for all the teachers who led the students through that curriculum; and for all the students who helped show improvement in many areas.

When faced with this kind of data intensive report, it is easy to pick and choose what is being focused on and what is being ignored.

It is important to use consistent language about the data. The Kansas Department of Education is using a four step scoring methodology. Here is the definition for the area of Math:
Math Assessment Results
Kansas assessment results are now reported in four levels. Level 1 indicates that a student shows a limited ability to understand and use the mathematics skills and knowledge needed for college and career readiness. Level 2 indicates that a student shows a basic ability to understand and use the mathematics skills and knowledge needed for college and career readiness. Level 3 indicates that a student shows an effective ability to understand and use the mathematics skills and knowledge needed for college and career readiness. Level 4 indicates that a student shows an excellent ability to understand and use the mathematics skills and knowledge needed for college and career readiness.
Building, district, and state-level reports provide summaries of the percent of students at each performance level. Assessment results capture a point-in-time and represent one part of a student’s and schools’ overall educational experience.

And here are the State of Kansas Results for Math as reported using this four step process:

Notice that the lowest level, Level 1, decreased from 2018 to 2019 and that is a huge advancement. But also notice that nowhere in the State definitions is the word “proficiency” used to describe the status of students.

Think of the four steps like the old A-B-C-D way of grading were Level 1 would be a D, Level 2 would be a C, etc. With this definition of measurement used by the State, approximately 44% of KCK students get a D while approximately 56% get a C or better. Room for improvement, sure and progress, sure.

The concern comes from the term “proficiency” used in the report presented to the Board of Education by the superintendent. And in the presented report it appears to say that only 18.6% of USD 500 students are “proficient” which would mean that 81.4% are not. This is a stark difference from the full state report (a link to that report is found at the end of this post). Here is a page from the superintendent report showing this rating:

It is good to have data driven discussions so long as the data is used appropriately and consistent definitions are used as well. So how do we ensure parents understand progress of the district? What is the rationale for highlighting data in such a limited way that does not appear to align with how the State Department reports results? Thinking even more broadly, are results on a single assessment the only thing that matters when determining progress of our school district?

NOTES:
The full Kansas repost can be found here:
https://ksreportcard.ksde.org/assessment_results.aspx?org_no=D0500&rptType=2
And here is the full superintendent report (shown here as a slide show) which he can be seen presenting at about minute 58 on the recent posting of the USD 500 Board meeting:

2 Comments

Data or Students?

10/27/2019 04:15:03 pm

A couple of thoughts about this post:

I largely agree with the sentiment but I think that some nuance is missing and the points are not quite stated strongly enough to drive the point home that I believe the authors are trying to make.

The congratulations that are offered are sincere, but they might be misplaced. I appreciate when someone congratulate curriculum developers and teacher for their work, but the data in question is not necessarily even correlated to curriculum or teaching. The state taste for Kansas is not an impressive one and is constantly being changed and field tested. The state didn’t decide to go with the ACT, but they are still looking into that option. David Rand always makes some good points about the data that we are looking at and the variance that is in the tests which means that the same student taking the same test on a different day, might not even score in the same Level.

Not only that, but why are we getting excited at all about increased success on a meaningless test for only Math and ELA? These tests mean NOTHING for students in, or after, school. Sure they are connected to funding in some confusing ways, but this test does not help our students think better, or go and do anything else after graduation.

Another piece that has always concerned me is the leveling factor. i.e. we will have bumps because we are focusing on the test, but it will level out quickly. Last year was the first year in a long time that we have spent any time even thinking about the test due to Dr. Foust’s (and now Dr. Hookfin’s as well) focus on the test. Dr. Lane (rightfully so) put almost zero emphasis on the test and under her administration I saw us as a district move towards focusing on the whole child and Social Emotional Learning, which are much more meaningful for students, but much harder to measure in quantitative data to then call yourself a turn-around expert. (If there happened to be anyone who wanted to do that, for example.). Dr. Lane’s administration also helped in preparing our students for college and careers via Diploma+, a model now adopted by Kauffman to ensure our students were engaged in meaningful learning.

I say all of this, not to say that teachers and curriculum do not deserve congratulations, only that we should congratulate them on things that matter, not a meaningless and functionally worthless standardized test. In fact, teachers are being blamed and micromanaged MORE after these tests being improved, not less by the current administrations. Go ask a teacher or administrator who is in KCKPS this year how the micromanagement has been. The environment is abysmal.

As far as the data and discussion itself, I really appreciate the nuance that the authors applied to the state vs. superintendent questions. All of the questions are good ones, I just wish they had gone a bit further and answered them instead of leaving them rhetorical. “What is the rationale for highlighting data in such a limited way that does not appear to align with how the State Department reports results?” That is an easy answer. A superintendent is beholden to the BOE, that individual will present data in any way that will make it look as if improvements are being made in the district. Also, if that superintendent hypothetically happens to want to use KCKPS as a jumping stone to a different district, they need any data they can get to show “turn-around” is happening.

Want to know some data that is more telling? What if the board commissioned a survey of the staff right now on how they feel that they are being treated as professionals? What if we got a questions about how likely teachers are to return next year and if they are thinking about leaving, why? The tone and environment in the district right now is one of fear and I think that that starts at the very top. Want to know how to improve student outcomes that matter? Keep qualified teachers who want to grow and improve in ways that matter and help them do that.

Another rhetorical question: “Thinking even more broadly, are results on a single assessment the only thing that matters when determining progress of our school district?” No, no, all the no! This is one of the worst metrics that any school can use to determine progress. Unless you care more about data that can be presented than you do the students themselves.

FRIEDA TRESVAN link

11/8/2019 07:14:06 am

The gaslighting and misinformation on this website is shocking. FOX News does a better job fact checking.

The word proficient comes from the STATE OF KANSAS ESEA plan but I guess your friends missed that KCKPS board meeting 11/27/18.

https://youtu.be/GYHIN9Fpv5U?t=4380

Pay close attention to minute marker 1:17:17 - 1:17:48 ...she explains the STATE DEFINITION of the KAP levels and the use of the word "Proficient".

Let's Talk About KCK School Board Activity