The Logic Model Workshop: Building the Capacity of Alliance Members and Others for Effective Program Design and Evaluation

As REL Northeast & Islands at EDC began its work with research alliances in 2012, our staff soon discovered that many district- and state-based alliance members wanted to learn more about how to use logic models to improve program or policy design, implementation, and evaluation.

A logic model is a visual representation of a theory of action that can guide individuals who are working together to design, implement, and/or evaluate a program or initiative. There is no “one way” a logic model should look, but all models include certain elements, such as a clearly identified problem to be addressed, desired outcomes, proposed strategies for achieving the desired outcomes, and assumptions about the inputs or actions that lead to the desired outcomes. Logic models help to ensure that a proposed education program or intervention truly addresses the problem identified by the group and that measures for evaluating success are integrated into the program design and roll out.

Creating logic models, however, should be understood as a process that requires deliberate and thoughtful conversations among all stakeholders invested in the new program or intervention. A logic model is typically not a static document but a guiding framework that can and should be revisited and revised as needed.

To help alliance members more fully understand and utilize logic models, a team of REL researchers, including Karen Shakman and Shelia Rodriguez, created a workshop to build stakeholders’ capacity to develop and use logic models in their work. The workshop was delivered to three alliances in 2013.

For the alliances—which in most cases convene members from across states, organizations, and disciplines around a shared common interest—the logic model workshops were an opportunity to not only learn how to develop logic models but, in some cases, also build members’ capacity to begin the process of designing a logic model for their own alliance projects.

Dr. Shakman, fellow REL researcher Jenny Scala, and Sheila Rodriguez delivered the first logic model workshop in June 2013 with the Puerto Rico Research Alliance for Dropout Prevention, which is a broad stakeholder group trying to develop an Early Warning System (EWS) for dropout prevention in Puerto Rico. Alliance Facilitator Sandra Espada-Santos invited us to lead a two-day, face-to-face workshop that not only explored what logic models are but also led the alliance members in discussing and defining the elements they wanted to include in a dropout prevention logic model for their EWS.

  The workshop helped the alliance members to identify that there were several needs to be addressed,” Espada-Santos said.

“Because of the discussions in this session, we realized we needed closer communication and collaboration between the key stakeholders. They had different definitions of what a dropout student is, and issues like that needed to be discussed with more intense communication. They also identified that studies were needed to better understand the statistics and data that were available about dropouts in Puerto Rico.  

The other two alliances that requested the logic model workshop were the Northeast Educator Effectiveness Research Alliance (NEERA) and Urban School Improvement Alliance (USIA).

These alliances were particularly interested in learning how to use logic models as a vehicle for building effective evaluation designs. NEERA has focused much of its research agenda on educator evaluation, and alliance members were interested in thinking more deeply about using theories of action to guide educator evaluation models. USIA is comprised of research directors at mid-sized districts in New England and New York, and they wanted training in how to help their instructional leaders think about setting up a proper evaluation prior to starting a new program or initiative. To address these needs, Rodriguez and Shakman presented three logic model workshops as webinars, inviting people both inside and outside the alliances to participate. Over 100 people attended these virtual workshops, which were held in October and November 2013, and January 2014. (Visit REL Northeast & Islands’ Skill-Builder Archive to view the webinars.)

Constituent groups outside REL Northeast & Islands alliances also requested logic model workshops. These groups include the following:

  • Office of Innovation and Improvement (OII), US Department of Education: In early 2014, Dr. Shakman and I led three different workshops for directors of charter schools and directors of charter management organizations.
  • Office for English Language Acquisition (OELA), US Department of Education: In October 2014 and April 2015, REL Northeast & Islands Research Alliance Manager Maria-Paz Avery and I led two workshops to help OELA staff build their capacity to assist grantees in using logic models in their projects.
  • Northeast Region Resource Center: In March 2014, Dr. Shakman led a session for state education agency staff from across the Northeast who work in early intervention, special education, and transition services for children with disabilities.

The logic model workshops were such a success that Rodriguez and Shakman used the materials as a starting point to build a toolkit that could be used by stakeholders across the region, and indeed the country, who want to conduct workshops with their colleagues in order to develop a logic model. The goal is for the toolkit is to support and build the capacity of education researchers and practitioners to develop and revise logic models on their own, without the help of people like REL researchers or other consultants, who are experts in this process. To keep the toolkit grounded in real-life needs and scenarios, a small group of regional stakeholders served as an advisory committee in developing the toolkit, reviewing materials and providing feedback along the way.

In 2015, the Institute of Education Sciences published REL Northeast & Islands’ Logic Model Workshop Toolkit. The toolkit includes a complete facilitator workbook, participant workbook, and slide deck that can be used by education researchers, district leaders, and others to lead two separate workshop sessions—the first focused on the elements of a logic model and the process for developing a logic model for a program or policy, and the second focused on how to use logic models to develop effective program evaluation questions and indicators of success. The toolkit also includes case study examples that can be used throughout the workshop to illustrate the logic modeling process.

Logic Model Workshop Toolkit

To further disseminate knowledge about logic models, REL Northeast & Islands published in September 2015 a self-paced, online professional development course focused on using logic models to develop program evaluations to support improvement. The REL also hosted a Skill-Builder Webinar last December on using the toolkit to facilitate the logic model workshop. Over 80 people participated, and survey feedback included:

  • "Speakers shared several useful tips about facilitating a workshop. Important concepts of logic models were clarified, such as the difference between outputs and outcomes."
  • "As a result of this workshop, I will be able to support schools in developing/refining their logic model specific to the [pre-K] grant and link their pre-K plan to their overall school logic model plan."

REL Northeast & Islands continues to field requests for these workshops and materials, and often uses these tools in its work with constituents in the region. In September 2016, Dr. Shakman used the toolkit to work with a member of the Connecticut State Department of Education to help her plan for an evaluation of the state’s teacher induction program. And on October 3, Dr. Avery led a team of REL staff who traveled to Augusta, Maine, to work with staff at the state department of education in developing a logic model and evaluation design for a new K–3 literacy pilot.

Looking ahead, the published logic model toolkit, coupled with the facilitator’s guide and the archive of the Skill-Builder Webinar, will allow state, district, and school leaders—as well as other regional and national stakeholders—to continue to use the tools even after the current REL contract ends on December 31.

Skill-Builder Workshop

How to Facilitate the Logic Models Workshop for Program Design and Evaluation

Toolkit Authors

Karen Shakman, PhD

Karen Shakman, an experienced researcher and evaluator, specializes in advancing the field's knowledge of educator effectiveness sytems and illuminating barriers and facilitators to sustaining K-12 education reforms. She brings significant expertise in collaborative research and works to deepen practitioners' understanding of program evaluation, research design, and data analysis.

Shakman is the lead researcher for the REL Northeast & Islands Northeast Educator Effectiveness Research Alliance (NEERA).