Link to Resource: The Role of the School District in School Turnaround Efforts

Authors: Nicolás Buchbinder and Elena Diaz-Bilello

This literature review investigates the evolving role of school districts in the context of school turnaround efforts, focusing on post-2015 developments under the Every Student Succeeds Act (ESSA). Traditionally, school turnaround policies emphasized isolated school-level actions, but recent perspectives advocate for a system-wide approach involving districts as pivotal facilitators of reform. The review underscores the importance of leadership, clear vision, capacity building, and collaborative frameworks for successful turnaround efforts.

Off

Link to Resource: Analysis of Colorado Multilingual Learner Redesignation Among the 2018 3rd Grade Cohort

Authors: Erik Whitfield, Benjamin R. Shear, Elena Diaz-Bilello

This study analyzes the redesignation patterns of a cohort of Multilingual Learner (ML) students in Colorado from 3rd to 8th grade (from 2018 to 2023), focusing on how redesignation varies across demographic groups. Redesignation as fluent English proficient (FEP) is a two-step process in Colorado: students must first earn a minimum score on the ACCESS exam and then produce a standardized body of evidence (BOE) demonstrating readiness to transition to mainstream English classrooms, with BOE criteria defined locally by districts and schools. By 8th grade, 69% of the 8,064 students in the cohort were redesignated as FEP, with the majority achieving this status by 7th grade. There was a significant drop in ACCESS test scores for 6th graders during the 2020-21 school year, reflecting both historical trends and pandemic-related disruptions. A large proportion of students achieving the minimum ACCESS score for redesignation each year were not redesignated, indicating the BOE requirement plays an important role in redesignation decisions. Students identified as Asian and students not eligible for Free or Reduced-Price Lunch (FRL) were more successful in meeting ACCESS criteria and achieving redesignation compared to Hispanic and FRL-eligible students. Overall female students were more likely to be redesignated as FEP by 8th grade. However, among students achieving the minimum ACCESS score necessary for redesignation each year, male students were consistently more likely than female students to be redesignated. Students with Individualized Education Programs (IEPs) had lower ACCESS scores and redesignation rates compared to those without IEPs. The findings underscore the need for ongoing monitoring of redesignation practices to address potential disparities and ensure equitable educational opportunities for ML students in Colorado.

Off

Link to Resource: Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework

Authors: Benjamin R. Shear and Kaitlin Nath

Federal law requires states to create school accountability ratings based on multiple factors including student academic achievement, academic growth, and progress in achieving English language proficiency. Many states meet these requirements by creating weighted composite scores based on indicators for each factor. When a composite score is constructed as a weighted sum of multiple indicators, the weights are called nominal weights because they represent the intended weights being given to each indicator. The actual influence of each indicator on the overall composite is referred to as the effective weight of that indicator. When the nominal and effective weights of an indicator differ, it can undermine the validity of interpretations of the composite scores. We review a widely used method for calculating effective weights and apply it to evaluate the alignment of nominal and effective weights for composite accountability scores produced in the Colorado School Performance Framework (SPF). We find the alignment of the nominal and effective weights varies across grade levels, with higher alignment for the elementary and middle school scores than for high school scores. We discuss factors that influence this alignment, such as the correlation and relative variances among the included indicators, and implications for designing school accountability composite scores. 

Off

Link to Resource: Teacher Reactions to the Phonics Content Referenced Growth Reporting Prototype: Findings from Interviews

Authors: Nicolás Buchbinder, Olivia Cox, Erik Whitfield*

*Authors listed in alphabetical order, all contributed equally to the writing of this report.

In this qualitative study, we conducted a series of think-aloud interviews with in-service elementary reading teachers as they reacted to a prototype for reporting content-referenced growth (CRG) within a commercially available diagnostic assessment, Curriculum Associates’ i-Ready Diagnostic. As in our prior work on content-referenced growth (Briggs et al., 2023), the design and structure of the reporting prototype is organized around a learning progression (LP). LPs afford rich opportunities for making sense of student growth by describing how a student's understanding of a 'big picture' concept in reading becomes more sophisticated over time with the right curricular and instructional support (Clements & Sarama, 2004; Lobato and Walters, 2017). Underlying any given LP is a theory-based hypothesis about how students develop knowledge and skill within the focal concept. However, adopting an LP approach to growth also requires adequate empirical support. In this report, we consulted an array of theories of word reading and reading development to define an LP about phonics and then validated the LP by exploring associations between its proposed levels and the difficulty of i-Ready Diagnostic items.

Off

Link to Resource: Teacher Perspectives on the Content-Referenced Growth Reporting Prototype: Findings from Interviews

Authors: Derek Briggs, Olivia Cox, Sandy Student, Erik Whitfield*

*Authors listed in alphabetical order, all contributed equally to the writing of this report.

This study aims to analyze in-service elementary teachers’ reactions to a prototype for reporting content referenced growth on a widely used commercial assessment, Curriculum Associates’ i-Ready Diagnostic. The prototype is built around an approach to growth that centers learning progressions (LPs), tools that describe the developmental path students are likely to take when learning a big picture concept in mathematics or reading (Clements & Sarama, 2004). Created through a partnership between CU Boulder and Curriculum Associates, the score reporting prototype is intended to support teachers, parents, and students in interpreting both the status of student understanding at one point in time and their growth in understanding across points in time. The purpose of this study is to gain insight into the intended use of the prototype by soliciting feedback from practicing teachers about: (a) the potential usefulness of embedding LP information into the i-Ready diagnostic report, (b), substantive interpretations supported by the LP prototype, and (c) the prototype’s usability. 

Off

Link to Resource: Development of a Reading Foundational Skills Learning Progression

Authors: Olivia Cox and Derek Briggs

A number of foundational skills contribute to students’ growth toward skilled reading. Among those is phonics knowledge, or knowledge of the relationship between alphabetic symbols in print and sounds in spoken language. We examine how phonics knowledge develops from kindergarten through third grade by considering a theory of reading development known as orthographic mapping and by examining how the difficulties of i-Ready Diagnostic phonics items change from grade to grade. Our analysis reveals distinctions in item difficulty between kindergarten, first grade, and second grade, while second and third grade items tend to be more similar in difficulty.

Off

Link to resource: Evidence of “Summer Learning Loss” on the i-Ready Diagnostic Assessment

Authors: Derek C. Briggs and Sarah Wellberg

This report provide a descriptive look at patterns of “summer learning loss” (SLL) in reading and mathematics for elementary and middle school students who use the i-Ready Diagnostic assessment system produced by Curriculum Associates. SLL in math amounts toabout 30 to 40% of school year growth depending on the grade in question. The score decreases are smaller in reading, ranging from about 10 to 20% of school year growth. 

Off

Link to Resource: Learnings from a Multi-site Case Study of Former Turnaround Schools

Authors: Elena Diaz-Bilello, Adam York, and Kaitlin Mork

In this report, CADRE researchers share results from conducting a qualitative multi-site case study at three former low performing schools located in one school district that adopted locally motivated turnaround strategies and practices to maintain higher levels of student achievement over time. Two schools are former turnaround schools that participated in the Colorado Department of Education’s (CDE) Turnaround Network (TN), and one school did not participate in the network but used many of the tools and strategies provided to TN schools. The purpose of this multi-site case study was to identify the practices, and conditions supporting those practices, that these schools learned through partnership with the TN and continue to sustain through present day.

Off

Addendum to 2021 Evaluation of the School Turnaround Network and the School Turnaround Leadership Development Program

Link to Resource: Descriptive Analyses of Connect for Success, Multi-Tiered Systems of Support, and Accountability Pathways

Authors: Kaitlin Mork, Benjamin Shear, and Elena Diaz-Bilello

This is an addendum to a report prepared for the Colorado Department of Education (CDE) by the Center for Assessment, Design, Research & Evaluation (CADRE) at CU Boulder (Shear et al., 2021). CDE requested an addendum to the 2021 CADRE report presenting similar descriptive analyses for schools that participated in the Connect for Success (CFS) program and providing initial descriptive analyses of academic outcomes for schools participating in two other Empowering Action for School Improvement (EASI) programs: The Multi-Tiered Systems of Supports (MTSS) and Accountability Pathways Planning and Implementation (Pathways).

In this addendum, we provide a brief overview of CFS, MTSS, and Pathways programs. We first describe the demographic characteristics and prior academic performance among students enrolled in all schools receiving support through these programs, which we compare to the population of all schools in the state or to schools who were eligible to participate in a CDE-offered turnaround program but did not. Next, we describe trends in the Colorado School Performance Framework (SPF) accountability ratings among participating schools in the years leading up to, during, and following participation in each of the three programs. Finally, we describe trends in average student achievement and growth scores among schools participating in these programs as measured by the Colorado Mathematics and English Language Arts state assessments administered in grades 3-8. For schools participating in CFS, we also compare the trends in SPF ratings and achievement outcomes to a group of comparison schools that were eligible to participate in the CFS but did not participate in a CDE-offered turnaround program during the studied time period. We conclude with a discussion of the results and highlight the next phase of planned work focused on estimating the effects of participating in these CDE-offered programs on student academic outcomes.

Off

Link to Resource: 

Authors: Jessica L. Alzen, Amy Burkhardt, Elena Diaz‑Bilello, Eryn Elder, Alicia Sepulveda, Audrey Blankenheim, Lily Board

Student retention is a key outcome measure for post-secondary education, but data show relatively stagnant retention rates over the past decade. Longstanding interventions such as counseling, academic advising, and mentoring provide crucial student services, but little change in retention rates suggest there is still need for additional student supports. Within the landscape of higher education, academic coaching is a relatively new, yet burgeoning intervention designed to increase student retention and success. Despite rapid growth of the intervention, little empirical work has been done to systematically describe and evaluate such programs. In this study, we provide a rich description of one academic coaching program and use a quasi-experimental design to evaluate the program’s effects on student outcomes. We investigate two research questions: 1) how does academic coaching influence key student outcomes? and 2) to what extent do these effects vary by amount of coaching received? On average, we found that students with prior semester grade point averages from 1.0—2.0 who participate in the academic coaching program earn grade point averages about 0.4 points higher during the coaching semester, are about 10% more likely to enroll in the semester following coaching, and earn about two more credits in the semester following coaching than students who choose not participate in the program. Outcomes varied minimally based on the number of coaching appointments students attended.

Off