Spring 2024 Newsletter
What’s New at the Center for Assessment, Design, Research and Evaluation
As of Fall 2023, CADRE now has a physical location within the new Miramontes Baca Education building (4th floor) along the southern boundary of the University of Colorado campus in Boulder. We have two new Research Associates, Adam York and Kaitlin Nath. One of our longstanding staff members, Jessica Alzen, has been promoted to Senior Research Associate. We are also excited to have a new faculty partner in Oded Gurantz, who joined the School of Education as an assistant professor in the Research and Evaluation Methodology program in the fall of 2022.
As always, CADRE staff have been keeping busy on a variety of projects.
- CADRE Director Derek Briggs recently finished a chapter (with co-authors Andy Maul and Josh McGrane) for the 5th edition of the edited volume Educational Measurement entitled “On the Nature of Measurement.” Unfortunately, the book isn’t set to be published until the end of 2024, but . Derek also enjoyed co-authoring a recently published journal article with CADRE faculty partner Ben Shear entitled “Measurement Issues in Causal Inference.” Another fruitful collaboration has been with CADRE faculty partner Lorrie Shepard and CADRE graduate student researcher Nicolás Buchbinder for a study conducted for the NAEP Validity Studies Panel. The result of this study, a white paper entitled “What Can NAEP Mathematics Subscales and Subscale Weights Tell Us 鶹Ƶ Common Core Effects?” will be released later this spring. They can’t wait to share the results. Derek continues his ongoing research on content-referenced growth reporting, described in more detail below.
- Associate Director Elena Diaz-Bilello recently co-authored a chapter with Jared Anthony for the upcoming volume by the National Academy of Education (NAEd), The Implementation and Use of Balanced Assessment Systems. This book reconceptualizes balanced assessment systems to support and inform equitable and ambitious teaching and learning. She is also writing a commentary for a set of essays addressing the implications of culturally responsive assessment for classroom assessment practices. Her commentary will be published in a new volume, Culturally Responsive Assessment in Classrooms and Large-Scale Contexts: Theory, Research, and Practice, to be released by the National Council on Measurement in Education (NCME). Elena currently collaborates with the Hawaiʻi Department of Education, the National Center for the Improvement of Educational Assessment, and the Learning Policy Institute on the statewide Performance Assessment Development Initiative (PADI) in Hawaiʻi, while continuing her dedicated partnership work with the Colorado Department of Education and several school districts in Colorado.
- Senior Research Associate Jessica Alzen finishes up two multi-year projects this year. The first is a collaboration with CADRE faculty partner, William Penuel, as well as colleagues from Northwestern University and University of California Davis. The project focuses on teacher learning regarding the Next Generation Science Standards and the pedagogical shifts necessary to fully enact curriculum aligned to those standards. The most recent paper from that collaboration discusses different ways teachers ask students to work together and how those choices can influence students’ opportunities to build science knowledge. The second project that ends this year is a collaboration with Eric Vance from the applied math department at 鶹Ƶ. A recent paper from that project considers the extent to which the Laboratory for Interdisciplinary Statistical Analysis at the university trains students to be effective interdisciplinary statistical collaborators.
- Postdoctoral Research Associate Kaitlin Nath continues three projects in partnership with the Colorado Department of Education (CDE) and local school districts. The first is a collaboration with the CDE and two districts to understand the use of performance-based assessments (PBA) and capstone projects to evaluate graduation readiness. The project focuses on the extent to which information from PBAs aligns with information from other graduation menu options and whether PBAs support equity goals. The second is a continued evaluation of a pilot grant that gives money to schools to hire additional mental health professionals and provide mental health services to students. The third is ongoing work with Colorado’s School Transformation Network, which provides targeted supports to low-performing schools across the state. So far, Kaitlin’s work with this department has included evaluating the effect of Colorado turnaround programs on student outcomes and conducting case studies to understand how schools engage in turnaround work. Within the 鶹Ƶ, Kaitlin continues work on the “CODE:SWITCH” project, which aims to train undergraduate students in the intersection of statistics and humanities. Kaitlin looks forward to continued collaboration with state partners to evaluate education initiatives.
- Research Associate Adam York continues work on partnerships with the Colorado Department of Education (CDE). One is a continuation of the evaluation of the state’s pilot K-5 Social Emotional Health grants, which funds schools to add mental health professionals that deliver social and emotional supports for students. Another is ongoing work with CDE’s Adult Education Initiatives, exploring programmatic features that support adult basic education and family literacy in Colorado. Adam recently completed a policy memo for the National Education Policy Center, titled How the Practices of Schools of Opportunity Illustrate Recent Research on Learning. The paper describes one school that detracked their curriculum, and two schools that developed culturally sustaining curriculum, noting the intersections with research on learning and implications for school leaders and policymakers.
- Faculty Partner Oded Gurantz is primarily partnering with the California Student Aid Commission (CSAC) on a number of studies examining how postsecondary policies impact family’s financial health. He is also partnering with a number of federal agencies as part of his position as a Fellow with the federal Office of Evaluation Sciences. One project is building off a prior relationship with the U.S. Department of Education, where they are examining the causal effects of the federal Return of Title IV Funds policy, which can require students who withdraw early in an academic term to return some of the grant aid to which they were entitled. This policy was the previous focus of . Oded is also engaged in a randomized control trial project currently in the field, which is studying whether small encouragements from the US Citizenship and Immigration Services office induce permanent residents to file for naturalization as U.S. citizens. The description of this project is pre-registered at https://oes.gsa.gov/projects/increasing-naturalization/.
- Faculty Partner Ben Shear continues his work aimed at improving the uses and evaluation of standardized tests with various partners. In 2023 his journal article "Causal inference and COVID: Contrasting methods for evaluating pandemic impacts using state assessments" examining the use of state assessment data to study pandemic impacts was published and was also the basis for a Provost’s Faculty Achievement Award at the University of Colorado. This project developed from a long-standing partnership with the Colorado Department of Education Accountability Unit and complements his recent co-authored article with Derek Briggs mentioned above. As a member of the Stanford Education Data Archive (SEDA) Development Team he contributed methodological expertise to support the creation of two new publicly available SEDA data products published in 2024 - SEDA2023 and SEDA 5.0. In June 2024 Ben will present new work examining the potential to incorporate matrix sampling techniques into state assessment programs for accountability and system monitoring purposes at NCSA with colleagues from the National Center for the Improvement of Educational Assessment and multiple state departments of education. In other news, Ben celebrated the birth of his first child in January 2024 and looks forward to the adventures ahead and analyzing a new type of growth percentile data.
- Faculty Partner Lorrie Shepardpublished a chapter, “Authentic Assessment Embedded in Project-Based Learning,” with Jeff Palladino, principal of Fannie Lou Hamer Freedom High School. The chapter appears in NEPC’s new book, Schools of Opportunity: 10 Research-Based Models of Equity in Action, edited by Adam York, Kevin Welner, and Linda Molner Kelley. Lorrie is awaiting the publication of her chapter on “Standards-Based Reform and Test-Based School Accountability,” coauthored with Scott Marion and Christopher Saldaña submitted over a year ago for publication in AERA’s Handbook of Education Policy Research (Vol. 2). As a member of the steering committee for an on-time volume, Lorrie will join editor Scott Marion in presenting an overview to NCME of the National Academy of Education’s new volume, Toward Balanced Assessment Systems: Considerations for Implementation and Use. Lorrie is also concluding her term (plus extra Covid years) as chair of the Professional Development Committee for the National Academy of Education. A particularly rewarding aspect of this role has been overseeing mentoring and professional development retreats for NAEd/Spencer dissertation and postdoctoral fellows.
Featured studies
Content-Referenced Reporting of Student Growth
By Derek Briggs
Over the past four years, CADRE has been involved in a Research-Practice Partnership with assessment specialists from the organization Curriculum Associates on a project that is exploring innovations in the presentation and interpretation of student growth from large-scale educational assessments. CADRE’s central innovation in this space is what we have come to describe as taking a “content-referenced growth reporting” approach...
Colorado Multi-Tiered System of Supports Partnership
By Jessica Alzen, Elena Diaz-Bilello
The Colorado Department of Education’s (CDE) Office of Learning Supports contracted with CADRE to support CDE’s efforts to improve school and district systems at the organizational level with the (COMTSS)...
K-5 SEL Pilot Grant Evaluation
By Adam York, Kaitlin Nath, Elena Diaz-Bilello
In 2022-2024 CADRE researchers partnered with the Colorado Department of Education to evaluate a state grant pilot program. This program emerged from Colorado HB 19-1017, which provided pilot K-5 Social-Emotional Learning (SEL) grants to 14 schools. One central purpose of these grants was to ensure that school health professionals (SHPs) at each elementary school could provide SEL support to all students...
Studying the role of postsecondary policies in shaping families’ financial health
By Oded Gurantz
A college degree provides substantial benefits, including lower unemployment, higher wages, and increased social mobility. However, as the price of a degree has risen, families increasingly use debt to finance a college education, with parents shouldering a larger share of the burden...
Categories:
Exclude