When the Northeast Educator Effectiveness Research Alliance (NEERA) created its research agenda in 2012, its core planning group wanted to study educator evaluation not as an end in itself, but as a vehicle for ensuring that schools provide the best instruction for all students. Their view of educator evaluation is that it should provide targeted and specific support to educators to do their best work in the classroom.
To advance this agenda, over the past few years a team of REL researchers have worked closely with a large urban district here in New England to study the alignment between teacher evaluation and professional learning—in a true collaboration between the district, the alliance, and the REL.
Our district colleagues wanted to learn to what extent the feedback that teachers received from the district’s new evaluation system aligned with the types of professional learning activities teachers subsequently engaged in to improve their skills and practice.
In 2012, we began by creating a data catalog of the district’s relevant data sources. We documented the data available to answer potential research questions, identified data gaps, and developed a set of potential research studies to focus on the alliance goal described above. The data catalog led to several discussions with the district about the type of data they collected and what research questions were and were not viable as a result. For example, while the district was very interested in making the link between teacher evaluation and participation in professional learning activities, the data catalog revealed that the district lacked reliable and consistent data about the range of professional learning activities in which teachers engaged.
The study examined the alignment of teacher evaluation results with professional learning by examining the feedback evaluators provided to teachers rated less than proficient in one or more standards and the subsequent professional support activities in which teachers engaged.
While teachers reported participating in a wide variety of activities, for all standards, less than 40% of respondent teachers completed all prescribed activities.
* 42% of teachers in the study population responded to the survey. The orange bars reflect the percentage of respondent teachers who participated in all prescribed activities for the standard.
One of the most exciting things about this study, and the relationship we’ve built with our colleagues in the district, is how much it is influencing the district’s evaluation and professional support systems moving forward.
In response to the findings, the district has revised its training of evaluators to provide more guidance about the nature of good written feedback and how to link it to existing professional supports. The district has also increased the alignment of its professional development offerings with the standards and elements of the evaluation rubric.
By sharing the study as it unfolded, NEERA members have had the opportunity to reflect on data challenges and consider preliminary findings and the implications of these findings. After one session, a colleague at the state level requested a meeting between the district, REL researchers, and his own state educator effectiveness team to learn more about the data the district collects, the needs the district has related to ongoing support, and the approach the district is taking to further align evaluation and professional learning. The state teams’ intention was to incorporate lessons from the district’s work into the guidance they provide to districts across the state.
Karen Shakman, PhD
Karen Shakman, an experienced researcher and evaluator, specializes in advancing the field's knowledge of educator effectiveness sytems and illuminating barriers and facilitators to sustaining K-12 education reforms. She brings significant expertise in collaborative research and works to deepen practitioners' understanding of program evaluation, research design, and data analysis.
Shakman is the lead researcher for the REL Northeast & Islands Northeast Educator Effectiveness Research Alliance (NEERA).