REL Northeast & Islands


Get event invites and news digests in your inbox.

Sign me up   No, thanks

New REL Study Compares Two Approaches for Identifying Beating-the-Odds High Schools in Puerto Rico

Posted by Yinmei Wan on December 9, 2016

Identifying beating-the-odds schools—schools that are performing better than expected given characteristics of the students enrolled—is an opportunity to uncover promising practices that other schools serving similar student populations may be able to use. Members of the Puerto Rico Research Alliance for Dropout Prevention at REL Northeast & Islands expressed interest in identifying beating-the-odds high schools in Puerto Rico. However, no single approach for identifying beating-the-odds schools exists. As a result, my REL colleague Coby Meyers, an associate professor of education at the University of Virginia, and I decided the best way to meet the alliance’s needs was to conduct a study comparing two different approaches for identifying high-poverty public high schools in Puerto Rico that were beating the odds.

On December 6, the Institute of Education Sciences (IES) published the study’s report, “A Comparison of Two Approaches to Identifying Beating-The-Odds High Schools in Puerto Rico.” The study analyzed school-level performance and demographic data to identify beating-the-odds schools from a sample of 159 high schools with at least 40 percent of their students living in low-income households (high-poverty schools). The analysis focused on the performance of the 2012/13 graduation cohort on two outcome measures: the 2012/13 cohort graduation rate and the 2011/12 grade 11 proficiency rate for reading (in Spanish, the language of instruction in Puerto Rico) and math combined.

We presented the study’s findings during a one-hour Bridge Event Webinar on December 7. The webinar also included discussions of current research on beating-the-odds schools in two other REL Northeast & Islands jurisdictions—New York and Maine.

Our study used the following two methods to identify beating-the-odds schools in Puerto Rico:

  • The status method ranked schools on the basis of their observed (actual) performance on each of the outcome measures. Schools that ranked in the top 25 percent on both the graduation rate measure and the proficiency rate measure were identified as beating-the-odds schools.
  • The exceeding achievement expectations method estimated schools’ expected performance on the two outcome measures using statistical techniques that took into account school characteristics, including students’ prior performance, school poverty rate, and other factors that might be related to school performance. The difference between each school’s actual performance and expected performance was calculated, and schools were ranked on the basis of these differences. Schools with positive difference scores in the top 25 percent on both outcome measures were identified as beating-the-odds schools.

The study findings highlight how identification of beating-the-odds schools can be affected by methodological choices. The status method identified 17 beating-the-odds schools in our sample, while the exceeding achievement expectations method identified 15. Six beating-the-odds schools were identified using both methods, resulting in a 38 percent agreement rate (overlap) between the two methods. Both groups of identified beating-the-odds high schools appeared similar to all high schools in the sample in terms of average enrollment, student racial/ethnic composition, and student-teacher ratio. However, beating-the-odds schools identified using the status method had a lower average percentage of students living in low-income households and a lower average percentage of students with disabilities than did beating-the-odds schools identified using the exceeding achievement expectations method and all schools in the sample. In addition, some of the schools identified using the status method ranked low when the exceeding achievement expectations method was used.

These findings suggest that using both methods to identify beating-the-odds schools is a good strategy because schools identified that way demonstrate high levels of “absolute” (actual) performance while also achieving higher rates of graduation and student academic proficiency on assessments than might be expected given population demographics and prior performance. The considerations presented in the report may be useful for states and districts that are developing or revising technical approaches for identifying schools that exceed performance expectations and that want to understand the limitations of these two methods.

Learn more about REL Northeast & Islands’ research publications.

Share Share