THE ECHO CHAMBER
Results, management and the

humanitarian effectiveness agenda

A major theme in humanitarian discourse over the last 25 years, effectiveness has been a central concern for those seeking to reform humanitarian practice and governance. Research on humanitarian effectiveness has taken the concept at face value, exploring what is effective, how to be more effective, and what the impact has been of individual initiatives. With effectiveness at the forefront of strategic discussions within Save the Children and across the humanitarian sector, the Humanitarian Affairs Team (HAT) felt that there are important questions to be asked:

  • Why has effectiveness become an organising ideal for humanitarians?
  • What is the character of the ‘humanitarian effectiveness agenda’ that has been constructed of initiatives to improve humanitarian performance?
  • Why is effectiveness understood in the way it is, and what are the implications of all this?

Setting out to investigate the forces and motivations that have shaped the humanitarian effectiveness agenda, its influence over humanitarian action, and how its contribution to notions of success relates to the circumstances and aspirations of people in countries affected by crisis, we planned seven field studies (Central America, Liberia, Niger, the occupied Palestinian territories, the Philippines, South Asia and the Syria region) to provide insight into different understandings of effectiveness and the interests involved in forming these understandings. The Echo Chamber and the accompanying Essays on Humanitarian Effectiveness are the culmination of a research project that started in October 2014 in partnership with the University of Manchester’s Humanitarian and Conflict Response Institute.

Download the complete version of The Echo Echo Chamber: Results, Management and the Humanitarian Effectiveness Agenda or just the summary.

Preface

This is not an evaluation. At least not as the term has been understood in the humanitarian sector. It is not a policy report either. You’ll not find any neatly boxed recommendations. Yes, it is about effectiveness. But you won’t see written into these pages a simple recipe for making humanitarian action more effective – evidence of what really works. This is a story about how a compulsion to understand and do ‘what works’ turned the humanitarian sector into a closed shop. It is an invitation to a reopen the debate about how humanitarian organisations understandsuccess, and about the possibilities of humanitarian action.

In July 2014 Save the Children’s Humanitarian Affairs Team (HAT) drafted a concept note for a research project on humanitarian effectiveness. With the Save the Children movement entering a new strategy cycle, the HAT had facilitated conversations within Save the Children UK about the future of the organisation’s humanitarian work and staff had focussed particularly on issues that in recent times have fallen under the banner of effectiveness: issues such as accountability, programme quality, and the participation of crisis-affected populations in humanitarian action. Meanwhile, the World Humanitarian Summit (WHS) preparatory process was underway, and the humanitarian community was being encouraged to reflect on humanitarian effectiveness as one of the four designated topics of the summit.

A major theme in humanitarian discourse over the last 25 years, effectiveness has been a central concern for those seeking to reform humanitarian practice and governance. Research on effectiveness in the humanitarian sector has invariably taken the concept at face value, exploring what is effective, how to be more effective, and what the impact has been of individual initiatives to improve effectiveness. With effectiveness at the forefront of strategic discussions within Save the Children and across the humanitarian sector, we (the HAT) felt that there were important questions to be asked: why has effectiveness become an organising ideal for humanitarians? What is the character of the ‘humanitarian effectiveness agenda’ that has been constructed of initiatives to improve humanitarian performance? Why is effectiveness understood in the way it is, and what are the implications of all this? We identified a set of assumptions that gives initiatives to enhance performance a ‘top-down’ quality – even those initiatives explicitly aimed at challenging inequalities in the humanitarian system. And, on account of the role effectiveness has come to play in definitions of success, we felt there was a need for investigation into the politics and epistemology* of effectiveness, and the institutional arrangements that underpin the humanitarian effectiveness agenda. Our research started in earnest in October 2014, in partnership with the Humanitarian and Conflict Response Institute (HCRI) at the University of Manchester; this paper is its culmination.

This paper is divided into four sections. We start by introducing the humanitarian effectiveness agenda, setting the scene from which it emerged. Then, in chapter one, we explore its roots: processes of bureaucratisation and professionalisation inside and outside the humanitarian sector that have framed developments in humanitarian performance management, particularly the revolution in management at the end of the 1970s, through which business principles were used as the basis for reforms to public sector bureaucracies in Britain and then elsewhere. Then, we discuss the birth and elaboration of the humanitarian effectiveness agenda itself, demonstrating the prominent role that commercial ideas have played in shaping humanitarian performance management. So as to focus our narrative on the humanitarian effectiveness agenda itself, we use appendices to provide more detail on the historical processes that are discussed in chapter one. In chapter two, we draw on fieldwork carried out in 12 countries to consider certain implications of the way the humanitarian effectiveness agenda has developed. With the aim of stimulating constructive debate about how to improve humanitarian action, we contend that the humanitarian effectiveness agenda has reinforced a reflexive and self-referential tendency within the humanitarian sector, creating an echo chamber in which the ideas of the sector’s dominant actors bounce off each other, validated without modification or critical interrogation. And we conclude with some suggestions as to how humanitarian agencies might go about reimagining success in terms that are more sensitive to the interests of people and institutions in crisis-affected countries, and more open to discussion.

The style of this paper is unusual for a work on effectiveness or indeed for a study produced from within a humanitarian agency. We decided to write about humanitarian effectiveness using a narrative approach, as far as possible telling stories rather than presenting a catalogue of ‘evidence’. This was partly because, let’s face it, humanitarian effectiveness, with its attention to process, bureaucracy and management, is hardly the most alluring topic. But the decision was also taken for substantive reasons. Evidence – a code that enables action with use of a known humanitarian kit – is at the heart of the humanitarian effectiveness agenda. There is an expectation in the humanitarian sector that studies about effectiveness should consist of a particular kind of evidence, presented in a particular way. There is little time for argument, exposition, and contestation. So our storytelling is a sort of dissident response to positivism* in the humanitarian sector, and to the formulaic policy-prose in which it results. Our research project has also resulted in the production of a series of essays based on field studies, with contributions from members of the HAT and HCRI, as well as independent researchers. Published alongside this paper, Essays on Humanitarian Effectiveness also has a narrative feel.

Setting out to investigate the forces and motivations that have shaped the humanitarian effectiveness agenda, its influence over humanitarian action, and how its contribution to notions of success relates to the circumstances and aspirations of people in countries affected by crisis, we planned seven field studies to provide insight into different understandings of effectiveness and the interests involved in forming these understandings. The first study was carried out in late 2014, in Bangladesh, India, and Nepal, following floods across the sub-continent. In early 2015, a second study was carried out in the Philippines, reflecting on the response to Typhoon Haiyan. At the end of the first quarter of 2015, research was carried out in Niger, looking at how humanitarian agencies have addressed slow-onset food crises. Then in the second quarter of 2015, there were studies carried out in Lebanon, Turkey and Jordan, looking at responses to conflict in Syria and its neighbouring countries; in Occupied Palestinian Territories (OPT), considering the role humanitarian agencies play in attending to a situation of long-standing hardship; in Liberia, looking at responses to the outbreak of Ebola; and in Guatemala and Honduras, looking at the burgeoning activities of humanitarian agencies in response to urban violence. All except OPT and Central America resulted in contributions to Essays on Humanitarian Effectiveness; data from both studies were used to inform the analysis presented here in The Echo Chamber; a separate publication reflecting on understandings of effectiveness in these contexts will be released by the HAT later in 2016.

Field research involved desk reviews, semi-structured interviews with key informants in capitals and areas affected by crisis, focus groups, observation, and unstructured interviews. There was variation across field studies in the emphasis placed on different methods as researchers balanced what was appropriate with what was feasible. All
the researchers working on the project engaged with Save the Children offices in the countries in which they conducted their fieldwork. Access to communities that have been affected by crisis and have received aid was mostly arranged by Save the Children colleagues, even though the project has not been focussed specifically on Save the Children’s
humanitarian activities. This was not inconsequential to the profile and disposition of people interviewed. To reduce any potential distortion of findings, researchers sought to engage with a diverse range of individuals within communities and disaggregated data by sex, age, and, where appropriate, social grouping. We were also aware of the potential
impact that the researchers’ affiliation with Save the Children might have on the answers provided by respondents. So that responses were not conditioned by positive or negative perceptions of Save the Children or by concerns related to future interactions with the organisation, researchers explained the purpose of the project to informants and told
them that their responses would be anonymous unless they expressed a desire to be quoted by name. In some instances, both in this paper and in the field studies, we use the name of an individual interviewed; in others, we make reference to their position and the name of their institution; and in others, quotations are anonymised. (This depended
on the sensitivity of the information involved and whether or not the individual was happy to be quoted). All the field research was qualitative and, though it was not inductive, we sought to allow space for the research process, and for observation and discussions with informants in particular, to determine which themes were covered in the essays and
how. Essays on Humanitarian Effectiveness provides an analysis of the impact of context on understandings of effectiveness. It also highlights infrequently acknowledged factors that shape understandings of effectiveness and others that have an impact on effectiveness according to these understandings.

Acknowledging the difficulties in moving from the particular to the general, field studies are used as supportive examples, offering insights on which we draw here, in this paper. As well as considering understandings of effectiveness, field research explored the impact in different contexts of specific initiatives to improve humanitarian performance and of the humanitarian effectiveness agenda in general. The findings in this regard are crucial to the analysis of the humanitarian effectiveness agenda in this paper. We have also carried out an extensive review of literature, not only on humanitarian effectiveness, but also on performance management, bureaucratic organisation, professionalisation, organisational change, complex systems, and various aspects of emergency response and disaster management. We have drawn on both primary sources (in particular humanitarian evaluations, but also archival information) and secondary sources, and have covered academic, journalistic and policy-focussed ‘grey’ publications. In addition to our interviews during field studies, we have conducted interviews with almost fifty people with relevant expertise from different professions and of different nationalities. We have carried out surveys of staff from across the Save the Children movement. We have tested and gathered ideas in practitioner and academic workshops, lectures and seminars, and conferences, including a conference on humanitarian effectiveness hosted by the HAT and HCRI in association with the WHS in September 2015. We have benefitted greatly from the counsel and expertise of an advisory group, comprising individuals from outside Save the Children, and a steering group made up of Save the Children colleagues. And we have drawn on our own experiences working in the humanitarian sector.

We are aware that our experiences, interests and profiles are sources of bias. The main authors involved in the project are white Western NGO workers. And the project has been mostly funded by, and managed from the headquarters of, a London-based international NGO. There is no attempt in either of the project’s publications to ‘give voice’ to others, even if we aim to give attention to issues overlooked by conventional discourse on effectiveness. Nor is there any pretence to the neutrality of the authors involved, even if the research methodology has been designed to obtain the most objective findings possible. The authors are notably, and unashamedly, present in their writings for this project. They analyse and synthesise information, offering their own perspective on histories already told and indeed proposing alternative histories. This can be seen as a rejoinder to the depoliticisation of research, which, in recent decades, has been inspired by the same ideals and interests that have shaped the humanitarian effectiveness agenda. That this paper gives particular attention to NGOs, and has a Western-, if not Anglo-centric, slant is intentional. Initiatives to improve humanitarian effectiveness have been developed primarily in ‘NGO-land’ and often in the UK, even if US and non-British European organisations have played a distinct role in the development of humanitarian performance management. This paper gives more attention to process and bureaucratic changes than it does to specific crisis responses. This too is because the humanitarian effectiveness agenda itself has been heavy on process.

While we point to problems and pose questions, we do not provide, nor do we presume to hold, all the answers. Indeed, we are suspicious of the presentation of silver bullet solutions to complex challenges. What we aim to provide is principles, or bases, upon which answers may be constructed.

In the concept note for the project, we stated our intention to propose ‘a new framework for understanding and analysing humanitarian effectiveness’ that, informed by our research, particularly our field studies, would incorporate different perspectives on effectiveness and could contribute to making the planning and implementation of humanitarian activities more driven by contextual specificities. However, as our research developed, it became clear that setting out such a framework would involve bypassing a crucial step in contextualising humanitarian aid. It became clear that the centrality of effectiveness to notions of success was itself reflective of an assumption about the objectivity of humanitarian knowledge, and therefore a barrier to contextualisation. What we propose here, then, is that, in order to open up conceptions of success, it is necessary for humanitarian agencies to challenge the politics and culture that have shaped the humanitarian effectiveness agenda. For them to truly contextualise their work, it will be necessary, in fact, to take steps towards restructuring the political economy of humanitarian aid. We conclude by offering some ideas about how this might be done.

Our history of the humanitarian effectiveness agenda and the forces that have shaped it is not exhaustive. But we seek to contribute to the growing body of literature that challenges ‘the idea that there are no alternatives to particular practices or concepts by drawing out the conditions under which these practices and concepts emerged’. In this
case, that entails reflecting on the interests, ideology and events that have placed effectiveness at the forefront of humanitarian discourse and practice. We consider how conditions might be created for alternative practices and concepts to be acknowledged, valued and incorporated into the planning and implementation of humanitarian activities.

This report has been produced as a discussion paper and does not reflect Save the Children policy.

Photo credit: Hedinn Halldorsson/Save the Children.