Reports

Analysis of Colorado Multilingual Learner Redesignation Among the 2018 3rd Grade Cohort

Ichigo Takikawa — Tue, 29 Oct 2024 22:23:25 +0000

Analysis of Colorado Multilingual Learner Redesignation Among the 2018 3rd Grade Cohort Ichigo Takikawa Tue, 10/29/2024 - 16:23

Erik Whitfield , Benjamin Shear , Elena Diaz-Bilello

Link to Resource: Analysis of Colorado Multilingual Learner Redesignation Among the 2018 3rd Grade Cohort

Authors: Erik Whitfield, Benjamin R. Shear, Elena Diaz-Bilello

This study analyzes the redesignation patterns of a cohort of Multilingual Learner (ML) students in Colorado from 3rd to 8th grade (from 2018 to 2023), focusing on how redesignation varies across demographic groups. Redesignation as fluent English proficient (FEP) is a two-step process in Colorado: students must first earn a minimum score on the ACCESS exam and then produce a standardized body of evidence (BOE) demonstrating readiness to transition to mainstream English classrooms, with BOE criteria defined locally by districts and schools. By 8th grade, 69% of the 8,064 students in the cohort were redesignated as FEP, with the majority achieving this status by 7th grade. There was a significant drop in ACCESS test scores for 6th graders during the 2020-21 school year, reflecting both historical trends and pandemic-related disruptions. A large proportion of students achieving the minimum ACCESS score for redesignation each year were not redesignated, indicating the BOE requirement plays an important role in redesignation decisions. Students identified as Asian and students not eligible for Free or Reduced-Price Lunch (FRL) were more successful in meeting ACCESS criteria and achieving redesignation compared to Hispanic and FRL-eligible students. Overall female students were more likely to be redesignated as FEP by 8th grade. However, among students achieving the minimum ACCESS score necessary for redesignation each year, male students were consistently more likely than female students to be redesignated. Students with Individualized Education Programs (IEPs) had lower ACCESS scores and redesignation rates compared to those without IEPs. The findings underscore the need for ongoing monitoring of redesignation practices to address potential disparities and ensure equitable educational opportunities for ML students in Colorado.

Off

Traditional

White

Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework

Anonymous — Fri, 06 Sep 2024 14:36:51 +0000

Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework Anonymous (not verified) Fri, 09/06/2024 - 08:36

Benjamin Shear , Kaitlin Nath

Link to Resource: Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework

Authors: Benjamin R. Shear and Kaitlin Nath

Federal law requires states to create school accountability ratings based on multiple factors including student academic achievement, academic growth, and progress in achieving English language proficiency. Many states meet these requirements by creating weighted composite scores based on indicators for each factor. When a composite score is constructed as a weighted sum of multiple indicators, the weights are called nominal weights because they represent the intended weights being given to each indicator. The actual influence of each indicator on the overall composite is referred to as the effective weight of that indicator. When the nominal and effective weights of an indicator differ, it can undermine the validity of interpretations of the composite scores. We review a widely used method for calculating effective weights and apply it to evaluate the alignment of nominal and effective weights for composite accountability scores produced in the Colorado School Performance Framework (SPF). We find the alignment of the nominal and effective weights varies across grade levels, with higher alignment for the elementary and middle school scores than for high school scores. We discuss factors that influence this alignment, such as the correlation and relative variances among the included indicators, and implications for designing school accountability composite scores.

In this report, we review a widely used method for calculating effective weights and apply it to evaluate the alignment of nominal and effective weights for composite accountability scores produced in the Colorado School Performance Framework (SPF).

Off

Traditional

White

Teacher Reactions to the Phonics Content Referenced Growth Reporting Prototype: Findings from Interviews

Anonymous — Tue, 20 Feb 2024 18:48:12 +0000

Teacher Reactions to the Phonics Content Referenced Growth Reporting Prototype: Findings from Interviews Anonymous (not verified) Tue, 02/20/2024 - 11:48

Nicolás Buchbinder , Olivia Cox , Erik Whitfield

Link to Resource: Teacher Reactions to the Phonics Content Referenced Growth Reporting Prototype: Findings from Interviews

Authors: Nicolás Buchbinder, Olivia Cox, Erik Whitfield*

*Authors listed in alphabetical order, all contributed equally to the writing of this report.

In this qualitative study, we conducted a series of think-aloud interviews with in-service elementary reading teachers as they reacted to a prototype for reporting content-referenced growth (CRG) within a commercially available diagnostic assessment, Curriculum Associates’ i-Ready Diagnostic. As in our prior work on content-referenced growth (Briggs et al., 2023), the design and structure of the reporting prototype is organized around a learning progression (LP). LPs afford rich opportunities for making sense of student growth by describing how a student's understanding of a 'big picture' concept in reading becomes more sophisticated over time with the right curricular and instructional support (Clements & Sarama, 2004; Lobato and Walters, 2017). Underlying any given LP is a theory-based hypothesis about how students develop knowledge and skill within the focal concept. However, adopting an LP approach to growth also requires adequate empirical support. In this report, we consulted an array of theories of word reading and reading development to define an LP about phonics and then validated the LP by exploring associations between its proposed levels and the difficulty of i-Ready Diagnostic items.

Off

Traditional

White

Teacher Perspectives on the Content-Referenced Growth Reporting Prototype: Findings from Interviews

Anonymous — Tue, 20 Feb 2024 18:46:14 +0000

Teacher Perspectives on the Content-Referenced Growth Reporting Prototype: Findings from Interviews Anonymous (not verified) Tue, 02/20/2024 - 11:46

Derek Briggs , Olivia Cox , Sandy Student , Erik Whitfield

Link to Resource: Teacher Perspectives on the Content-Referenced Growth Reporting Prototype: Findings from Interviews

Authors: Derek Briggs, Olivia Cox, Sandy Student, Erik Whitfield*

*Authors listed in alphabetical order, all contributed equally to the writing of this report.

This study aims to analyze in-service elementary teachers’ reactions to a prototype for reporting content referenced growth on a widely used commercial assessment, Curriculum Associates’ i-Ready Diagnostic. The prototype is built around an approach to growth that centers learning progressions (LPs), tools that describe the developmental path students are likely to take when learning a big picture concept in mathematics or reading (Clements & Sarama, 2004). Created through a partnership between CU Boulder and Curriculum Associates, the score reporting prototype is intended to support teachers, parents, and students in interpreting both the status of student understanding at one point in time and their growth in understanding across points in time. The purpose of this study is to gain insight into the intended use of the prototype by soliciting feedback from practicing teachers about: (a) the potential usefulness of embedding LP information into the i-Ready diagnostic report, (b), substantive interpretations supported by the LP prototype, and (c) the prototype’s usability.

This study aims to analyze in-service elementary teachers’ reactions to a prototype for reporting content referenced growth on a widely used commercial assessment, Curriculum Associates’ i-Ready Diagnostic. Created through a partnership between CU Boulder and Curriculum Associates, the score reporting prototype is intended to support teachers, parents, and students in interpreting both the status of student understanding at one point in time and their growth in understanding across points in time.

Off

Traditional

White

Development of a Reading Foundational Skills Learning Progression

Anonymous — Mon, 19 Feb 2024 23:46:56 +0000

Development of a Reading Foundational Skills Learning Progression Anonymous (not verified) Mon, 02/19/2024 - 16:46

Olivia Cox , Derek Briggs

Link to Resource: Development of a Reading Foundational Skills Learning Progression

Authors: Olivia Cox and Derek Briggs

A number of foundational skills contribute to students’ growth toward skilled reading. Among those is phonics knowledge, or knowledge of the relationship between alphabetic symbols in print and sounds in spoken language. We examine how phonics knowledge develops from kindergarten through third grade by considering a theory of reading development known as orthographic mapping and by examining how the difficulties of i-Ready Diagnostic phonics items change from grade to grade. Our analysis reveals distinctions in item difficulty between kindergarten, first grade, and second grade, while second and third grade items tend to be more similar in difficulty.

A number of foundational skills contribute to students’ growth toward skilled reading, including phonics knowledge. In this report, we examine how phonics knowledge develops from kindergarten through third grade by considering a theory of reading development known as orthographic mapping and by examining how the difficulties of i-Ready Diagnostic phonics items change from grade to grade.

Off

Traditional

White

Evidence of “Summer Learning Loss” on the i-Ready Diagnostic Assessment

Anonymous — Tue, 27 Sep 2022 14:59:06 +0000

Evidence of “Summer Learning Loss” on the i-Ready Diagnostic Assessment Anonymous (not verified) Tue, 09/27/2022 - 08:59

Derek Briggs , Sarah Wellberg

Link to resource: Evidence of “Summer Learning Loss” on the i-Ready Diagnostic Assessment

Authors: Derek C. Briggs and Sarah Wellberg

This report provide a descriptive look at patterns of “summer learning loss” (SLL) in reading and mathematics for elementary and middle school students who use the i-Ready Diagnostic assessment system produced by Curriculum Associates. SLL in math amounts toabout 30 to 40% of school year growth depending on the grade in question. The score decreases are smaller in reading, ranging from about 10 to 20% of school year growth.

This report provide a descriptive look at patterns of “summer learning loss” (SLL) in reading and mathematics for elementary and middle school students who use the i-Ready Diagnostic assessment system produced by Curriculum Associates. SLL in math amounts toabout 30 to 40% of school year growth depending on the grade in question. The score decreases are smaller in reading, ranging from about 10 to 20% of school year growth.

Off

Traditional

White

Learnings from a Multi-site Case Study of Former Turnaround Schools

Anonymous — Fri, 12 Aug 2022 15:24:32 +0000

Learnings from a Multi-site Case Study of Former Turnaround Schools Anonymous (not verified) Fri, 08/12/2022 - 09:24

Elena Diaz-Bilello , Adam York , Kaitlin Mork

Link to Resource: Learnings from a Multi-site Case Study of Former Turnaround Schools

Authors: Elena Diaz-Bilello, Adam York, and Kaitlin Mork

In this report, CADRE researchers share results from conducting a qualitative multi-site case study at three former low performing schools located in one school district that adopted locally motivated turnaround strategies and practices to maintain higher levels of student achievement over time. Two schools are former turnaround schools that participated in the Colorado Department of Education’s (CDE) Turnaround Network (TN), and one school did not participate in the network but used many of the tools and strategies provided to TN schools. The purpose of this multi-site case study was to identify the practices, and conditions supporting those practices, that these schools learned through partnership with the TN and continue to sustain through present day.

Off

Traditional

White

Descriptive Analyses of Connect for Success, Multi-Tiered Systems of Support, and Accountability Pathways

Anonymous — Fri, 08 Apr 2022 21:45:30 +0000

Descriptive Analyses of Connect for Success, Multi-Tiered Systems of Support, and Accountability Pathways Anonymous (not verified) Fri, 04/08/2022 - 15:45

Kaitlin Mork , Benjamin Shear , Elena Diaz-Bilello

Addendum to 2021 Evaluation of the School Turnaround Network and the School Turnaround Leadership Development Program

Link to Resource: Descriptive Analyses of Connect for Success, Multi-Tiered Systems of Support, and Accountability Pathways

Authors: Kaitlin Mork, Benjamin Shear, and Elena Diaz-Bilello

This is an addendum to a report prepared for the Colorado Department of Education (CDE) by the Center for Assessment, Design, Research & Evaluation (CADRE) at CU Boulder (Shear et al., 2021). CDE requested an addendum to the 2021 CADRE report presenting similar descriptive analyses for schools that participated in the Connect for Success (CFS) program and providing initial descriptive analyses of academic outcomes for schools participating in two other Empowering Action for School Improvement (EASI) programs: The Multi-Tiered Systems of Supports (MTSS) and Accountability Pathways Planning and Implementation (Pathways).

In this addendum, we provide a brief overview of CFS, MTSS, and Pathways programs. We first describe the demographic characteristics and prior academic performance among students enrolled in all schools receiving support through these programs, which we compare to the population of all schools in the state or to schools who were eligible to participate in a CDE-offered turnaround program but did not. Next, we describe trends in the Colorado School Performance Framework (SPF) accountability ratings among participating schools in the years leading up to, during, and following participation in each of the three programs. Finally, we describe trends in average student achievement and growth scores among schools participating in these programs as measured by the Colorado Mathematics and English Language Arts state assessments administered in grades 3-8. For schools participating in CFS, we also compare the trends in SPF ratings and achievement outcomes to a group of comparison schools that were eligible to participate in the CFS but did not participate in a CDE-offered turnaround program during the studied time period. We conclude with a discussion of the results and highlight the next phase of planned work focused on estimating the effects of participating in these CDE-offered programs on student academic outcomes.

Off

Traditional

White

Academic Coaching and its Relationship to Student Performance, Retention, and Credit Completion

Anonymous — Wed, 28 Apr 2021 17:35:01 +0000

Academic Coaching and its Relationship to Student Performance, Retention, and Credit Completion Anonymous (not verified) Wed, 04/28/2021 - 11:35

Jessica Alzen , Amy Burkhardt , Elena Diaz-Bilello

Link to Resource: Academic Coaching and its Relationship to Student Performance, Retention, and Credit Completion

Authors: Jessica L. Alzen, Amy Burkhardt, Elena Diaz‑Bilello, Eryn Elder, Alicia Sepulveda, Audrey Blankenheim, Lily Board

Student retention is a key outcome measure for post-secondary education, but data show relatively stagnant retention rates over the past decade. Longstanding interventions such as counseling, academic advising, and mentoring provide crucial student services, but little change in retention rates suggest there is still need for additional student supports. Within the landscape of higher education, academic coaching is a relatively new, yet burgeoning intervention designed to increase student retention and success. Despite rapid growth of the intervention, little empirical work has been done to systematically describe and evaluate such programs. In this study, we provide a rich description of one academic coaching program and use a quasi-experimental design to evaluate the program’s effects on student outcomes. We investigate two research questions: 1) how does academic coaching influence key student outcomes? and 2) to what extent do these effects vary by amount of coaching received? On average, we found that students with prior semester grade point averages from 1.0—2.0 who participate in the academic coaching program earn grade point averages about 0.4 points higher during the coaching semester, are about 10% more likely to enroll in the semester following coaching, and earn about two more credits in the semester following coaching than students who choose not participate in the program. Outcomes varied minimally based on the number of coaching appointments students attended.

In this study, we provide a rich description of one academic coaching program and use a quasi-experimental design to evaluate the program’s effects on student outcomes. We investigate two research questions: 1) how does academic coaching influence key student outcomes? and 2) to what extent do these effects vary by amount of coaching received?

Off

Traditional

White

Evaluation of Colorado School Turnaround Network and School Turnaround Leadership Development Grants: Descriptive Analysis of 2015- 2020 Cohorts

Anonymous — Fri, 02 Apr 2021 22:50:11 +0000

Evaluation of Colorado School Turnaround Network and School Turnaround Leadership Development Grants: Descriptive Analysis of 2015- 2020 Cohorts Anonymous (not verified) Fri, 04/02/2021 - 16:50

Benjamin Shear , Elena Diaz-Bilello , Sandy Student , Medjy Pierre-Louis

Link to Resource: Evaluation of Colorado School Turnaround Network and School Turnaround Leadership Development Grants: Descriptive Analysis of 2015- 2020 Cohorts

Authors: Benjamin R. Shear, Elena Diaz-Bilello, Sanford R. Student, and Medjy Pierre-Louis

Suggested Citation: Shear, B.R., Diaz-Bilello, E., Student, S.R., and Pierre-Louis, M. (2021). Evaluation of Colorado School Turnaround Network and School Turnaround Leadership Development Grants: Descriptive analysis of 2015-2020 cohorts. Boulder, CO: The Center for Assessment, Design, Research and Evaluation (CADRE).

As part of a broader strategy to help support the lowest performing schools in the state, the Colorado Department of Education (CDE) launched the School Turnaround Network (STN) program in 2014-15 and subsequently launched the School Turnaround Leadership Development (STLD) program the following year. The STN program approaches supports using a comprehensive, holistic perspective by ensuring that entities across three levels (state, district, and school) work together to coordinate school improvement strategies for each school participating in the network. STN participants receive these supports for a three-year period. The STLD program focuses specifically on school leadership development and building the capacity of school leaders to lead and sustain difficult changes at participating schools. The theory of action for these programs and related supports, provided to the lowest performing schools through the Empowering Action for School Improvement (EASI) grants, posits that supports provided by CDE will improve student academic performance and other indicators of school quality over time. To date, 151 schools have received state support through these two programs.

The Center for Assessment, Design, Research and Evaluation (CADRE) at the �鶹��Ƶ was asked by the CDE to form a partnership and carry out studies that would address two areas of interest: 1) build on prior analyses to describe the impact of the STN and STLD programs on student academic performance as evaluated by the state summative assessments; and 2) carry out case studies of select turnaround schools to identify and learn about the work these schools have done as participants in these programs. This report focuses on the first area of interest and provides descriptive analyses of student academic performance at participating schools since the inception of both programs. For this report, we analyzed longitudinal administrative data collected by the state to track the academic performance of schools. The analyses contained in this report focus on the set of schools receiving STN and STLD supports. We also compare the academic performance of these schools to the schools that were eligible to receive these supports but did not participate in these or other EASI programs. Our analyses describe performance trends and patterns for each cohort of STN and STLD schools. We describe trends for each cohort separately rather than in a combined group due to factors such as: differences in STN and STLD program eligibility rules each year, distinct performance trends in each cohort, and the differential impacts of state testing changes on each cohort of schools during this time period.

For this report, we analyzed longitudinal administrative data collected by the Colorado Department of Education to track the academic performance of schools. The analyses contained in this report focus on the set of schools receiving School Turnaround Network (STN) and School Turnaround Leadership Development (STLD) supports.

Off

Traditional

White