Friday, November 8, 2013

What's Next for Program Reviews

Recently, the Kentucky Department of Education (KDE) released the first report on Program Reviews for arts and humanities, practical living/career studies and writing programs. These Program Reviews were required by Senate Bill 1 (2009) to be included as a part of the Unbridled Learning accountability system. For more information about the results from the Program Reviews, please see the news release.  I thought readers might want to know a little more about Program Reviews and what comes next.

Why do we have Program Reviews? Senate Bill 1 wanted to ensure that Kentucky children have access to a balanced education. A balanced education includes core academic areas (math, language arts, social studies and science) and areas such as arts/humanities, practical living (health and physical education), career studies, writing, world language and K-3 programs. Kentucky has had a vision of a balanced education since the 1990 Kentucky Education Reform Act and Senate Bill 1 continued that vision. Children are much more than test scores and test scores do not tell the entire story about a class, school or district. As a former high school band director, I am very proud that Kentucky includes an emphasis on the arts in the state accountability system.

What are Program Reviews? Program Reviews are systemic reviews of a program area that includes components such as curriculum, instruction, student assessment/performance, student opportunities and access, professional development and resources. In Kentucky, we worked with many education groups and educators to develop scoring rubrics for each of the components.

How valid and reliable are Program Reviews? Any component of an accountability system must be able to answer this question. All Program Reviews go through an extensive process of piloting and field testing before becoming part of the accountability system. Extensive research is done to address questions about validity and reliability. However, now that the “real world” results are in for the first year of inclusion in the accountability system, we are taking several steps to continue to address validity and reliability issues.

     1. KDE will initiate an audit process that was developed during the 
         last two years. This audit process will include a number of 
         random schools as well as “purposeful” schools. Purposeful 
         schools will be chosen by comparing other school data to 
         the Program Reviews scores for outliers.

     2. KDE will start a research project to determine connections 
         between quality programs and their impact on student 
         achievement. Writing tends to work best since there is a    
         writing achievement score and a program review score, 
         however, we have to be cautious in this approach since 
         the writing scores are derived from either on-demand 
         writing or language mechanics. A successful writing program 
         addresses more than these two areas. In arts/humanities and 
         practical living, KDE will make some connections between 
         overall scores and Program Review scores.

     3. KDE will work to find model programs that school staff 
         can use to assist their local scoring. Schools and districts 
         would use these exemplars just like examiners use
         exemplar writing papers in training and calibrating scores. 

     4. KDE will continue to upgrade training on the rubric 
         and the scoring process.

     5. Districts will receive support and training on how to 
         conduct local audits of Program Reviews.

Within the next few weeks, KDE will update the Unbridled Learning accountability scores for schools and districts to include the results from the 2013 Program Review results. KDE will then reset the 90th percentile and 70th percentile scores for schools and districts which will serve as the baseline for comparing accountability results in 2014. Readers can find the results of their school and district Program Reviews in KDE Open House.  The revised accountability scores and targets for schools and districts will be posted in the School Report Card later this month. 

No comments:

Post a Comment