Change Management: Seeing the Finish Line

January 10, 2018

By Peter Lorinser (Improvement Specialist) & Juanita Zerda (Director of Implementation)

This the second of a series of blog posts on the components of the Rennie Center’s Change Management Framework.

Our Change Management Framework lays out six essential elements to guide improvement in districts, schools, and classrooms. This fall, we took a closer look at the first step: Identifying a Problem of Practice. Here, we dig into the second step in the process: Establish Projected Outcomes.

Teachers often begin planning units of study by focusing on what they hope students will know and be able to do. Visualizing what success should look like helps maintain high expectations, while providing a clear direction (or a North Star) for charting a course of action. In practical terms, establishing a set of standards for students to meet and exceed enables teachers to plot out the intermediate steps necessary to guide daily learning in pursuit of long-term goals. In addition, each intermediary step provides teachers with the opportunity to assess learning and make real-time adjustments in their teaching when lapses in understanding are revealed.

A district or school looking to make large-scale improvements should take a similar approach. Through our own improvement experiences and those of the Massachusetts Teaching and Learning Network (MassTLN), we have learned that to develop an effective improvement effort, teams must set specific outcomes. We’ve developed three essential components to help guide the process of establishing outcomes and measuring progress along the way.

 

1. Determine Summative Impact Measures: Set an ultimate outcome and choose quantifiable measures that represent its achievement through a significant and specific change in behavior or condition.

In many classrooms in the Commonwealth, you’ll find “SWBAT” written on a whiteboard each day followed by a daily learning objective. SWBAT stands for “students will be able to...” and provides a flashlight into how a teacher approaches their day-to-day work: at the beginning of class, they have the end goal in mind. Similarly, an improvement team must determine their ultimate goal and answer the essential question, “What does success look like? For whom? By when?”

For example, a school looking to enhance the use of technology in the classroom could decide that by June 30, 2019, 20 of 40 teachers who enrolled in a "technology in the classroom" summer course will successfully implement at least three "technology-focused" lessons in their classroom with 80% student satisfaction.

Moving strategically from the number of teachers engaged in the training (40 teachers) to the number of teachers that will effectively implement their learning (20 teachers) represents an aim that is reflective of reality. Setting an unrealistic outcome, such as a 100% success rate, although aspirational and admirable, inevitably leads to disappointment and discouragement. To determine realistic outcomes, teams should consider staff experience, results from similar projects, and research from the field. In our example, the school set a specific goal by identifying a set number of teachers, establishing a time constraint, and defining how they will measure success. Improvement efforts that set good summative impact measures reflect the particularities of the approach and the context in which it is operating.

 

2. Determine Formative Impact Measures: Design process and benchmarks/indicators for assessing progress against the desired long-term impact.

Meaningful change does not happen instantaneously. In the classroom, student learning builds as the year progresses and interventions are often introduced when students struggle to meet certain standards. Say, for instance, a 9th grade English teacher is leading a unit on developing a research paper. A teacher who only measures success based on students’ ability to write a full paper at the end of the unit, lacks a full account of all the learning that has occurred and the opportunity to uncover and remedy potential gaps in understanding. A better approach would be to monitor and assess student progress throughout the unit, focusing on mastery of specific learning components, such as using transitional sentences, supporting statements with evidence, and finding credible sources. Breaking down the unit in this way provides the teacher with real-time data on student learning, which can be used to strengthen lesson plans, provide constructive feedback, and, ultimately, improve student performance.

Similarly, once an improvement team specifies their overall impact measure, it is crucial that they establish intermediary milestones to measure progress. In our example of a school looking to improve the use of technology in the classroom, the work may break down into the following benchmarks:

Benchmark

Timeframe

Measurement

40 teachers complete training and at least 38 teachers report that they understand the concepts

By August 30, 2018

As measured by records of which teachers completed the training and their self-reported understanding.

35 teachers want to try it in their own classroom

By October 30, 2018

As measured by testimonials, follow-up support records, and observations.

25 teachers have applied (tested) what they learned in at least three “technology-focused” lessons.

By January 31, 2019

As measured by classroom observations, and testimonials from students and teachers.

Within this example, although 40 teachers will be trained, the project would be deemed successful if 25 out of 40 (62.5%) implemented what they learned, and of those 25, at least 20 (50%) were able to implement it in a way that had a direct impact on the classroom. The school may be able to increase these percentages following the next training session by documenting the lessons learned throughout the first implementation period.

 

3. Determine Data Collection Methods: Establish measurements using valid data collection tools.

Teachers use classwork, homework, and assessments (e.g., portfolio reviews, oral presentations, quizzes and tests) to determine whether students have mastered course content. Similarly, improvement teams should consider multiple methods for measuring progress and use data collection tools that provide substantive information for guiding action and determining next steps.

For example, if teachers engaged in the project of enhancing technology in the classroom hold a grade-level meeting to discuss their implementation efforts, this represents an initial and vital sign of shifting practice. However, data collection efforts should continue by having teachers test their implementation of technology in the classroom and surveying students to garner their feedback on the process. The combination of these three data points—engagement (meeting), execution (teacher’s testing out the practice in the classroom), and feedback (student exit slips)—tells an improvement story.

Although formative measures are crucial, it is also important that teams are not drowned in data collection. High-performing teams prioritize efficient data collection, only collecting information that will inform the effort. The best improvement data is:

  1. Easily understood (you can quickly determine what it represents);
  2. Collected frequently (you can analyze it for timely lessons learned); and
  3. Long-lived (you can detect seasonal variations because the data doesn’t disappear after a short period of time).

 

Improvement teams, like teachers, must constantly ask themselves the question, “How do we know if it’s working?” Without valid data collection methods, the answer is indiscernible. The fast-paced environment of improvement demands that data collection systems be flexible, versatile, and as simple as possible.

Like a teacher making sure students acquire the intended skills and knowledge throughout the school year, breaking down an improvement effort into milestones with corresponding data collection systems ensures an improvement team is on track toward their ultimate outcome.

**

Improvement work is complex. The various levels of outcomes that need to be developed and tracked can be complicated. Our team of specialists is here to help project managers break down their projected outcomes into manageable chunks that allow them to answer the following questions:

  • What does success look like?
  • How will we know that we are on the way to success?
  • What will we use to measure our progress along the way?

 

To further assist our Networks and other improvement teams in Establishing Projected Outcomes we have created a set of project management tools. These tools enable teams to assess their relative strengths and weaknesses. Check out our Establishing Project Outcomes rubric and stay tuned for our next blog where we will discuss the importance of detailing the nature of potential interventions.

 

Slide1_2.PNG

Slide2.PNG

 

Projected Outcomes References