This story has also been published on the EADI Blog

Over the last couple of years, the evaluation sector has come under pressure to acknowledge the Euro-Western hegemony over knowledge, evaluation and learning practice (1). Current structures, behaviours, mindsets and mental models have led to institutional and systemic racism and discrimination of non-white people and other minoritised groups (2). This perpetuates a system that places weak or no value in non-Euro-Western knowledge, values and voices and results in non-representative practitioners and evidence, which threatens the legitimacy of evaluation as such.

Most of the attention towards fundamentally revamping the evaluation system aims at decolonising evaluation, focusing on indigenous peoples and international development. Too often, however, these efforts fail to work due to the wider lack of equity, diversity and inclusiveness in the evaluation sector – particularly in the Global North. Furthermore, individuals from non-dominant groups might be complicit in maintaining the current system - I am sharing my journey to illustrate the pervasive nature of the challenge. 

The lack of equity, diversity and inclusion in evaluation systems

There is no shortage of examples of negative biases and blind spots in the field of evaluation. In the UK, 9.3% of the total population are from Asian ethnic groups, followed by Black people with African and Caribbean ancestry. And Roma are the largest minoritised ethnic group in Europe. Yet it is a struggle to find Black, Asian or Roma evaluators and global south voices in any spaces where power concentrates and global standards are developed and held. 

Presidents and keynote speakers at evaluation conferences, webinars and other spaces convened by professional evaluation societies in the UK and Europe are dominated by white individuals from a few countries in the global North. For example, at the 2022 UK Evaluation Society’s conference, all but one keynote speaker was white and all were from a UK or US-based organisation. In the 2022 European Evaluation Society’s biennial conference, all but one keynote speaker were from a white background and all from organisations based in the UK, Belgium, Switzerland and Nordic countries – none from other European countries. 

For the top 10 most read evaluation websites, eight out of ten belong to white men, and eight of those websites are affiliated to US-based organisations. This also applies to the most known evaluation journals. For example, Evaluation, The International Journal of Theory, Research and Practice - a renowned international journal whose mission is to promote exchange between European, North American, Asian and Australasian voices - has a remarkable absence of non-white, Asian or openly LGBTQ+ voices and evaluators with disabilities in its publications (3). Moreover, 21% of all 2022 authors also published in this journal in 2023, demonstrating a small and repetitive group of non-diverse voices. And in the recent special issue called “Decolonising Evaluation” by the Journal of Multidisciplinary Evaluation, approx. 20% of authors were from a Black heritage or minoritised group but sadly none of them were from Global South organisations. 

There has been an increased uptake in story-telling approaches in evaluation, such as most significant change and outcomes harvesting, to be more culturally responsive, and inclusive and to bring epistemic pluralism to evaluation. Yet, current evaluation practices often do not fit with different cultural ways of telling stories, nor are they trauma-informed. For example, often story-based approaches demand a strict beginning/middle/end structure or some kind of turning point, which doesn’t align well with storytelling approaches found across the world, It also does not align well with how trauma-affected populations tend to provide narrative (non-linear, with gaps, due to the mental and emotional impact that occurs after traumatic experiences). 

As a result, evaluations often are not culturally responsive and, worse, they do not honour the storyteller, as stories that do not fit ‘the mould’ and are considered “weak evidence” are devalued, discounted, and depreciated. This shifts the focus to storytelling and narrative-based approaches that continue to impose colonial values, Euro-Western knowledge systems and exclude non-culturally appropriate methods, thereby perpetuating a system where minoritised and marginalised voices are not given the value and respect they deserve.

A cautionary tale: How individuals from non-dominant groups might be complicit in maintaining the current system

Even individuals and communities from non-dominant groups can be complicit, often unconsciously, in maintaining the status quo and dominant system. I know now because I was one of them.

I grew up in Mexico, in a family with intergenerational poverty history. When I was a teenager, we entered the lower-middle class. Thanks to scholarships, the private higher education I received was heavily influenced by American values and systems, and it offered a window to neoliberalism and capitalism at its best. However, I also had the privilege to study in Paris for a semester, exposing me to a more socialist version of economics and other European values. This was the first time I realised the inherent bias in different knowledge and power systems. 

I moved to the UK in 2007 and spent the first 10 years of my professional career working in the international development sector. During that time, I did my best to learn all the evaluation theories, approaches, tools and standards that were preferred, valued and rewarded. Over time I became part of the dominant system, playing a role in maintaining its non-inclusive, non-diverse and non-equitable practice by helping impose reporting systems, values, knowledge systems and standards from dominant Global North funders that did not reflect nor respect local history, values and organisational capabilities. 

Yet I often felt the dissonance with my values, lived experience and those we were supposed to be helping in the Global South, but I remained silent and unable to counter that narrative. I also did not feel I was in a position to change this and was afraid of making a career-limiting move. 

Moreover, I was desperate to fit in and to be “seen” in spaces convened by evaluation societies and the peer-reviewed journals they support. But even after I had a peer-reviewed publication in the Evaluation Journal in 2021, I found the doors closed at the UK Evaluation Society despite my repeated attempts to be invited as a guest speaker at one of the webinars they organised to amplify the authors’ findings. I realised that those who got invitations were predominantly white, often from an academic and privileged background, so I desisted. And this is not a “sour grape” claim but an example of how we also need to rethink “merit”. I recently worked with the Sanger Institute and its new programme for Black heritage academics where the notion of merit was deconstructed and its new meaning reflected in the application and selection process - making the fund more equitable, diverse and inclusive.

After a period of overt discrimination, burnout and deep reflection I was finally able to see the evaluation system for what it was. It also took setting up my own consultancy company to feel safer and to find my voice and a way to advocate for a more equitable, inclusive and diverse system. Writing this piece is one of many ways I am trying to disrupt the system and be part of its transformation. 

Making evaluation more equitable, diverse and inclusive

To make evaluation and learning more equitable, diverse and inclusive (EDI) there are many transformations required. We need to:

• Deconstruct the superiority and privilege of dominant groups, particularly Euro-Western knowledge systems and practice, addressing their inherent biases and how these are reflected in non-EDI notions of merit, selection and reward criteria and processes. 

• Increase representation of people from non-dominant groups, such as black, Asian and minoritized and marginalised groups like Roma. 

• Recognise and value diverse forms of knowledge and experiences, ensuring the inclusion of people with a truly diverse set of knowledge systems, values and ways of working – we cannot assume individuals from non-dominant groups are not complicit in maintaining the current system. 

• Reconstruct evaluation values, approaches and tools based on local conditions and diverse cultural nuances, including using trauma-informed and healing approaches in evaluation (4).

All of the above are interlinked and should be pursued in parallel. For example, increasing representation from non-dominant groups without any of the other transformations will cause more harm and mistrust. Moreover, to achieve any of these major changes, we need to take a systemic approach. This includes work at the individual, community and sector-wide level.

Funders, whether they are public or private, have a big role to play in the needed transformations as the power, and the way the way they fund and hire is at the core of the current evaluation system and practice. This evaluation report outlines how funders could pursue sector-wide EDI and systems change for social justice. 

There are also many ways in which UK and European evaluation societies could also help transform the evaluation system and its practice by:

I often see evaluators from dominant groups unsure of whether they should be part of these discussions and efforts, or how. They can be powerful advocates and allies that help open spaces and shine a light on issues and non-dominant evaluators, ideas, and values. But whether you are from a dominant or non-dominant group – we can all be part of these transformations by asking ourselves, and encouraging others to reflect on, the following questions:

1: Sources: “Special Issue: Decolonizing Evaluation: Towards a Fifth Paradigm” (2023); “Made in Africa Evaluation: Decolonising Evaluation in Africa” (2019); “Involuntary social experimentation: revisiting the case for a moratorium” (2023); “Locus of power”(2023), which builds on a model by Nan Wehipeihana; Lea Corsetti’s blog (2022)

2: Sources: 2016 Concluding Observations of the Committee on the Elimination of Racial Discrimination (CERD), the UN’s 2019 Report of the Working Group of Experts on People of African Descent; 2020 Human Rights Council Report and 2019 Equality and Human Rights Commission report.

3: When I analysed all 84 authors that published in this journal in 2022, I found that 93% of all of them are white, and that 67% of them were affiliated to a UK, USA/Canada or Australian-based organisation. I also looked at all 2023 publications to see who also published then. Consult full findings here.

4: See for example “Principles of Trauma-Informed Evaluation” and this project where photography  was used for healing and storytelling.