The King's Fund report reveals tensions in government evaluation of social programmes

Government commitment to using sound evidence to inform social policy may be at odds with its drive to tackle the root causes of health inequality and social exclusion, according to Finding Out What Works, a King's Fund report published today.

Co-author Anna Coote, The King's Fund director of health policy said:

'Existing evidence is too thin and inconclusive to provide a reliable basis for tackling health inequalities. The government has invested substantial sums in finding out 'what works', for example £20m for Sure Start3 and £16m for the New Deal for Communities, but complex locally-based initiatives such as these are notoriously hard to evaluate. There are serious tensions between policy-makers, researchers and local practitioners - because they all have different interests. This gives rise to confusion and can get in the way of effective social change.'

The report welcomes new attempts to strengthen the evidence base, improve evaluation methods and disseminate findings, but points to key problems:

  • There is a gap between the rhetoric of 'evidence-based policy' and what happens in communities where policies are implemented.
  • Complex local interventions do not lend themselves easily to methods of evaluation that try to trace cause and effect.
  • Research findings about 'what works' are often used selectively, to serve political ends.
  • Practitioners working at local level often find the formal 'evidence base' irrelevant to their needs; many feel that their own experience and expertise are not taken into account.

The report draws on new research into the experience of policy makers, academic researchers engaged in programme evaluation and local practitioners. One senior government official commented:

'Evidence-based policy and cutting-edge innovation seemed to me rather a contradiction in terms.' A researcher described what happened when evaluation findings were sent to Whitehall: 'They get the reports, they crawl all over them. They take out all the bad messages and then they publish them. Whether they look at them to improve policy, I don't know. 'Give us some good news!' That's what they say.'

The King's Fund chief executive Niall Dickson adds:

'At the moment there is no agreed framework for evaluating complex social programmes. Politicians favour 'quick wins' while senior civil servants seek clear results that satisfy ministers. Researchers pursue academic credibility and practitioners in the field want to secure funding and get help with improving their work locally. This report is intended to stimulate reflection and debate and to improve our understanding of how all this works and how we build knowledge and act upon it.'

The report also draws on a three-year transatlantic seminar series organised jointly by The King's Fund, the Rockefeller Foundation and the Aspen Institute in the US. In their conclusions, the authors call for:

  • More open and inclusive discussion about the unresolved tensions and dilemmas outlined in the paper.
  • Stronger efforts to develop a learning culture in government, among evaluators and among local practitioners.
  • Priority to be given to building knowledge, rather than just to promoting evidence-based policy and practice.
  • Knowledge to be built by integrating the experience of practitioners and local residents with the findings of researchers, and a more explicit recognition of the trade-offs required by the political context of the day.

The King's Fund is holding a seminar, Finding Out What Works: Building knowledge about complex community based initiatives, on Monday 22 November 2004 (see Notes to Editors).

For further media information or interviews please contact Beverley Cohen on tel: 020 7307 2632 or Daniel Reynolds on tel: 020 7307 2581 at The King's Fund media and PR office.

Notes to editors: 

1. Finding Out What Works by Anna Coote, Jessica Allen and David Woodhead is published by the King's Fund costing £15. Please call 020 7307 2591 to order a copy or download it from our online bookshop.

2. The King's Fund is holding a seminar, Finding Out What Works: Building knowledge about complex community-based initiatives, on Monday 22 November 2004, 5-8pm at the King's Fund, 11-13 Cavendish Square, London, W1G 0AN. Please call Beverley Cohen on 020 7307 2632 if you would like to attend.

3. Since 1997, the Government has invested heavily in a range of ambitious programmes, ranging from Health Action Zones to the National Strategy for Neighbourhood Renewal, which are designed to be implemented with local communities. SureStart aims to improve the health and well-being of very young children and their families, and the New Deal for Communities seeks to end social exclusion by improving health, education, employment, housing and community safety in disadvantaged neighbourhoods.

4. This report is the result of three years' research including interviews with the government staff who are commissioning programmes, the practitioners implementing them and the academics evaluating them, as well as recent UK and US case studies.

5. Finding Out What Works is part of the King's Fund's Putting Health First programme, which seeks to develop an effective health system giving priority to preventing illness and reducing health inequalities, not just to providing health services.

6. The King's Fund is an independent charitable foundation working for better health, especially in London. We carry out research, policy analysis and development activities, working on our own, in partnerships, and through grants. We are a major resource to people working in health, offering leadership and education courses; seminars and workshops; publications; information and library services; and conference and meeting facilities.