The Impact Initiative, together with the Institute of Development Studies (IDS), the Research and Policy in Development programme at the Overseas Development Institute and International Institute for Environment and Development, brought researchers, practitioners and policymakers together in March 2017 for a research symposium to discuss: "If evidence really matters what can we do about it?"
At the event we interviewed two development professionals, Jude Fransman (Open University) and Sarah Cook (UNICEF Office of Research - Innocenti), asking them to share their thoughts on what good and credible evidence is and what its value is in international development.
What is good evidence?
‘[Evidence is] the body of knowledge - a source of materials - research findings, data, analysis that can contribute collectively to answering a question, for giving us directions for policy and practice, to mobilise a body of knowledge around a particular issue,’ explains Sarah.
This definition of evidence provides a useful foundation to think about what makes evidence ‘good quality’. Both Jude and Sarah argue that ‘rigour’ is an essential component and Jude adds that this needs to be balanced with ‘relevance’ and ‘responsiveness’, for evidence to be at its best.
Sarah also talks about ‘transparency’ as a key criteria, including around how research agendas are set, preferred methodologies, and in relation to which issues and voices are pushed to the fore (and which are pushed back) as a result.
What is credible evidence?
The Oxford English dictionary describes credibility as ‘the quality of being trusted and believed in.’ Sarah states, ‘…being credible means being transparent enough that we understand where the data has come from […] as well as the methods used, and thus how dependable the results are’. Jude suggests that the purpose of evidence defines its credibility:
‘For a researcher, evidence could serve as exploratory and explanatory role - it could be focused on solving a particular problem, it could be focused on evaluating a programme or an organisational practice, or it could be developmental, so have a capacity building aim within an organisation, or a skills development aim with the individual involved.’
The notion that credibility is very much determined by context is shared by Sarah as well. Factors such as where and by whom evidence is presented, as well as the audience receiving it, shape whether evidence is seen as credible or not. So credibility is a subjective concept, as Sarah argues:
‘…credible knowledge, is also about politics, power relations, audiences, and processes of legitimacy that mean that different groups may or may not accept that evidence.’
Similarly, Jude argues that notions of credibility can be shaped by institutional structures, processes and agendas; by research practices and artefacts; by researcher identities; and by established ‘accounts’ or discourses around, for example, ‘rigour’, ‘impact’, and ‘partnerships’.
What is the value of good and credible evidence in international development?
As it has transpired through Jude and Sarah’s interviews, good and credible evidence is determined by its purpose and by transparency in the values that underpin it. These qualities of evidence are important in international development (as in other fields) as they make explicit the rationale, motivations and power dynamics driving the research agenda and invite critical reflection upon this.
Jude talks about an ESRC-funded seminar series she has been convening (rethinkingresearchpartnerships.com) that looks at the nature of evidence and its relationship to participation in research partnerships between NGOs and academics. Through the analysis of seven case studies of partnerships, it has examined how evidence is shaped by contexts and purposes, how power works within partnerships to shape evidence and distribute participation; and how alternative configurations of participation can lead to the transformation of power and production of new and better evidence through partnerships.
Successful and balanced partnerships between NGOS and research institutes have a number of common features and Jude highlights an essential element:
‘[relationships] based on trust and long-term sustainable understanding and commitment to particular values, ideologies are important […] that’s when we can see new really exciting pieces of evidence coming to the fore.’
However, this is often constrained by opportunistic and superficial ‘collaborations’ in response to short-term and one-off funding opportunities.