Case Study: Evaluating the Kofi Annan Changemakers Initiative

Written by
Jannik Kaiser
on
March 28, 2023
Impact
Leadership

The Kofi Annan Changemakers Initiative is a year-long programme that brings together 12 selected young changemakers online and in person in Geneva, with the ambition to

“make the voices of young people count by giving them a space to dialogue with and seek counsel from experienced leaders on issues of deep concern whilst affording the opportunity to leaders to better understand the preoccupation of young people.” (Kofi Annan Foundation Website)

Unity Effect has been working with the Kofi Annan Foundation since June 2022, both to design and facilitate online and in-person sessions for the Changemakers Initiative, and to evaluate the impact of the programme. 

This case study focuses on the impact evaluation and outlines how we designed and implemented an evaluation framework, based on our approach to Regenerative Impact Evaluation.

If you are curious about how an evaluation process could look for your organisation, email us at: info@unityeffect.net or directly schedule a call with Jannik

One year with the Kofi Annan Way

The key underlying intention of the program is to bring the values and approaches underpinning the work of Kofi Annan and his colleagues to life, by fostering dialogues and creating mutual learning experiences between young and experienced leaders. 

The structure of the programme is comprised of: 

  • regular facilitated online events starting in summer, 
  • a mentoring system, by which each changemaker is matched with an expert, 
  • continuous support provided by members of the Kofi Annan Foundation, 
  • a week in Geneva: 5 days of in-person workshops, talks and side events, and the first time the group of 12 young leaders meet in person. 

With our evaluation we mostly focused on the impact of the 5 days in Geneva.

Our leading question was: 

How can we measure the changes the program and the immersive experience creates for and within the participants, and what can we learn from that in order to further improve the programme? 

The methodology

In order to apply a regenerative approach to measuring the impact of the programme, we took the following steps: 

  1. Understand and define what to measure 
  2. Designing a holistic evaluation framework that includes ways to share and reflect on the insights with the changemakers 
  3. Setting up the surveys and data collection in synergy with other activities being undertaken
  4. Evaluating the results and creating a report with key insights for the Kofi Annan Foundation
  5. Having individual coaching calls with changemakers to jointly reflect on the results. 

Each step is briefly described below. 

Understand and define what to measure 

Kofi Annan was widely recognised for his integrity, being the co-recipient of the 2001 Nobel Peace Prize for his work as Secretary-General of the United Nations. The Kofi Annan Foundation is dedicated to continuing the legacy of Kofi Annan and fostering his style of ethical leadership.

The Kofi Annan Changemakers programme addresses key capacities and core values such as ethical leadership, resilience, humility, ability to listen to others and willingness to bring people together, integrity and setting objectives.

The first step for us therefore was to better understand what’s behind each of these values. For example, what does humility mean for the programme, and how does it show up in someone’s actions? 

As a basis for this dialogue we drew on our background research and methodology to measure capacity development - which includes various core capacities, such as humility, and contains a respective definition. 

The result was an overview of the five core values, and working definitions that correspond to the programme’s understanding of each value. 

Designing a holistic evaluation framework 

To guide the development of our evaluation framework, we defined two central intentions, namely to 

  • support the Kofi Annan Foundation to better understand the impact of its Changemaker programme, and how participants were and are able to live into the five core values, and
  • provide additional support to the changemakers via individual coaching calls that draw on the analysis.

The methodology and process is based on Unity Effect’s approach to Regenerative Impact Evaluation.

“For us, measuring impact is based on curiosity, joint learning, sharing knowledge and applying methodologies that capture human complexity. It is an empowering process that reveals hidden opportunities and resources, creates a deeper understanding of yourself and others, and opens up scenarios for joint action.”

In designing the research methodology, we therefore included a range of different data sources:

  1. Pre-and post surveys with items to measure the 5 core values:  1) resilience, 2) humility, 3) ability to listen to others, 4) willingness and ability to bring people together and 5) setting objectives. The items were taken from academically validated assessments from our indicator database. 
  2. Qualitative data from before and after the week in Geneva, collected through open-ended questions in surveys. 
  3. Notes taken throughout the week.

A key aspect was to think through how the different data sources complement each other, and to integrate our surveys with the Foundation’s surveys and feedback processes. 

We will provide further details on each data source below.

More resources
Free workshops, events, resources and more, straight to your inbox each month
get our newsletter

Data Sources

Quantitative surveys

The surveys consisted of items (statements) where the changemakers evaluated to which extent they apply to them (on a scale from 1 to 10). 

We decided to select 6 items per each of the 5 values we wanted to measure. The survey therefore consisted of 30 items that the changemakers responded to before and after the week in Geneva. An example of an item is: “I feel thankful for what I have received in life” to measure humility. 

To select the statements, we drew on the item library we created, which consists of items from around 50 academically validated assessments. 

It’s important to mention that this approach measures the self-perception of participants, and not necessarily the actual embodiment of the different values. It does, however, enable us to visualise trends over time on an individual and collective level. In the case of the Changemaker programme, we identified: 

  • the highest and lowest scoring values, as well as trends within and between those values (e.g. “humility” with the highest increase), and
  • how individual changemakers see themselves compared to the group. 

Qualitative data 

As part of the surveys before and after the week in Geneva, we included open-ended questions. These were: 

  • Describe in 1-2 sentences how you see yourself as a changemaker/leader? (both before and after the week)
  • Based on your experience from the week in Geneva: What is one thing that you will do differently in your organisation / community when you return home?
  • What challenges do you anticipate arising (when doing it differently) and how will you meet them?

The answers to these questions revealed, for example, how the changemakers' perception of themselves changed (remarkedly) within such a short period of time. 

Notes from the week in Geneva

One of our team members was present throughout the week to facilitate and host sessions and support participants, and to take notes on personal observations. We agreed in advance on three “buckets” to take notes on. These were: 

  • the 5 core values, 
  • awareness of self and others, and 
  • the perceived impact of the different sessions on them.

The notes informed the analysis and, most importantly, provided a foundation for the individual coaching calls with participants. It’s important to mention that our intention behind the notes was primarily to empower the changemakers by being able to reflect their strengths and leadership qualities back to them. 

Outcomes

Report with key insights 

Combining the different data sources, we were able to not only describe general trends, but also to deduct four key insights with practical recommendations for the Kofi Annan Foundation. This included, for example: 

  • “From sparks to long-term change”: Describing how the week led to a change of perspective for many participants, and how the Foundation can support this to lead to sustained changes in participant’s actions. 
  • “Intercultural sensitivity: More than just a nice to have”: The safe space during the week in Geneva enabled the changemakers to open up and share, despite coming from different backgrounds and experiences. As part of our analysis we could show the value of this experience, recommended to make it even more explicit that intercultural sensitivity is a crucial capacity for changemakers to live into, and to consider different aspects around inclusion even more in the program design. 

We shared the report with the Foundation to support internal reflections and planning for the next steps. 

Coaching calls with changemakers

Lastly, we held 30 minute coaching calls with each participant of the programme. Their responses to the surveys and our notes formed the foundation for these calls, and we approached it as an opportunity for mutual growth and learning. Through the calls we gained additional insights into the data, as changemakers reflected on their results. It also provided the opportunity to touch base on an individual level and better understand the impact the program had had on each participant. 

Outlook 

In summer 2023 a new batch of young changemakers and leaders from around the world will start their Kofi Annan Changemakers journey. Yet the group we had the privilege to work with in the past months will still live on. The findings also supported and gave context to the Foundation's plans to establish an alumni community - to provide continuous support around the topics that matter to the changemakers and therefore “turn sparks into long-lasting change”. 

“The Kofi Annan Foundation greatly benefited from Unity Effect’s facilitation skills and caring attitude. Unity Effect also conducted an impact evaluation of the programme, providing valuable information and suggestions that will further support the initiative’s growth and positive impact on its young changemakers.” - Linda Peterhans, Kofi Annan Foundation 

For us, this process reinstated the power of approaching impact evaluation in a regenerative and holistic way. As such, it served as a tool for everyone involved to reflect, learn and grow collaboratively. At Unity Effect, we offer further approaches to guide and empower organisations to do the same. 

Reach out to us if you are interested in our work around regenerative measurement & evaluation, or our other services around (digital) facilitation, leadership and teamwork. 

Email us at: info@unityeffect.net

Or directly schedule a call with Jannik

We’d love to hear from you!

About the author
Co-founder of Unity Effect. Striving to capture the depths and intricacies of social change, and how to shape it. Having a soft spot for complexity science, impact evaluation and cat videos.
Close Cookie Preference Manager
Cookie Settings
By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage and assist in our marketing efforts. More info
Strictly Necessary (Always Active)
Cookies required to enable basic website functionality.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.