Why co-produce? Accounting for diversity in citizens’ motivations to engage in neighbourhood watch schemes.

Carola van Eijk, Trui Steen & Bram Verschuere

In local communities, citizens are more and more involved in the production of public services. To list just a few examples: citizens take care of relatives or friends through informal care, parents help organizing activities at their children’s school, and neighbours help promoting safety and liveability in their community. In all these instances, citizens complement the activities performed by public professionals like nurses, teachers, neighbourhood workers and police officers; this makes it a ‘co-productive’ effort. But why do people want to co-produce? In our recently published article in Local Government Studies we try to answer that question by focusing on one specific case: local community safety. One of the main conclusions is that citizens have different incentives to co-produce public services, and local governments need to be aware of that.

Simultaneous to the international trend to emphasize citizens’ responsibilities in the delivery of public services, there are also concerns about the potential of co-production to increase the quality and democratization of public service delivery. One important question pertains to who is included and excluded in co-production processes. Not all stakeholders might be willing or feel capable to participate. So, acknowledging the added value of citizens’ efforts and the societal need to increase the potential benefits of co-production, it is important to better understand the motivations and incentives of citizens to co-produce public services. A better insight not only can help local governments to keep those citizens who are already involved motivated, but also to find the right incentives to inspire others to get involved. Yet, despite this relevance, the current co-production literature has no clear-cut answer as the issue of citizens’ motivations to co-produce only recently came to the fore.

In our study, we focus on citizens’ engagement in co-production activities in the domain of safety, more specifically though neighbourhood watch schemes in the Netherlands and Belgium. Members of neighbourhood watch teams keep an eye on their neighbourhood. Often they gather information via citizen patrols on the streets, and report their findings to the police and municipal organization. Their signalling includes issues such as streetlamps not functioning, paving stones being broken, or antisocial behaviour. Furthermore, neighbourhood watch teams often draw attention to windows being open or back doors not being closed. Through the neighbourhood watch scheme, the local government and police thus collaborate to increase social control, stimulate prevention, and increase safety.

The opinions of citizens in co-producing these activities and their motivations for getting engaged in neighbourhood watch schemes are investigated using a ‘Q-methodology’ approach. This research method is especially suitable to study how people think about a certain topic. We asked a total of 64 respondents (30 in Belgium and 34 in the Netherlands) to rank a set of statements from totally disagreement to fully agreement.

Based on the rankings, we were able to identify different groups of co-producers. Each of the groups shares a specific viewpoint on their engagement, emphasizing for example more community-focused motivations or a professional attitude in the collaboration with both police and local government. To illustrate, in Belgium one of the groups identified are ‘protective rationalists’, who join the neighbourhood watch team to increase their own personal safety or the safety of their neighbourhood, but also weigh the rewards (in terms of safety) and costs (in terms of time and efforts). In Netherlands, to give another example, among the groups identified we found ‘normative partners’. These co-producers are convinced their investments help protect the common interest and that simply walking around the neighbourhood brings many results. Furthermore, they highly value partnerships with the police: they do not want to take over police’s tasks but argue they cannot function without the police also being involved.

The study shows that citizens being involved in the co-production of safety through neighbourhood watch schemes cannot be perceived as being similar to each other. Rather, different groups of co-producers can be identified, each of these reflecting a different combination of motivations and ideas. As such, the question addressed above concerning why people co-produce cannot be simply answered: the engagement of citizens to co-produce seems to be triggered by a combination of factors. Local governments that expect citizens to do part of the job previously done by professional organisations need to be aware of the incentives people have to co-produce public services. Their policies and communication strategies need to allow for diversity. For example, people who co-produce from a normative perspective might feel misunderstood when compulsory elements are integrated, while people who perceive their engagement as a professional task might be motivated by the provision of extensive feedback.

 

Foto Carola %28bijgesneden%29.jpgCarola van Eijk holds a position as a PhD-candidate at the Institute of Public Administration at Leiden University. In her research, she focusses on the interaction of both professionals and citizens in processes of co-production. In addition, her research interests include citizen participation at the local level, and crises (particularly blame games).

 

trui02a

Trui Steen is Professor ‘Public Governance and Coproduction of Public Services’  at KU Leuven Public Governance Institute. She  is interested in the governance of public tasks and the role of public service professionals therein. Her research includes diverse topics, such as professionalism, public service motivation, professional-citizen co-production of public services, central-local government relations, and public sector innovation

 

Bram Verschuere 2.jpg

Bram Verschuere is Associate Professor at Ghent University. His research interests include public policy, public administration, coproduction, civil society and welfare policy. 

Troubled Families: How experimenting could teach us “what works?”

Jason Lowther

 

In this blog on 3rd Feb, I explored the formal Troubled Families Programme (TFP) evaluation and looked at the lessons we can learn in terms of the timing and data quality issues involved. This week I want to consider how designing a more experimental approach into this and future programmes could yield lots more insight into what works where.

The idea of an “experimental” approach to policy and practice echoes enlightenment period thinkers such as Francis Bacon (1561—1626), who promoted an empirical system built on careful experimentation. Donald Campbell’s ideas[1] on ‘reforms as experiments’ argued that social reforms should be routinely linked to rigorous experimental evaluation. ‘Social engineering’ built on ‘social experiments’ became a popular concept in the USA and social science.

Social experiments in America included work in response to a concern that providing even modest income subsidies to the poor would reduce motivation to find and keep jobs. Rossi and Lyall (1976) showed that work disincentives were in fact less than anticipated. In the field of prison rehabilitation, Langley et al. (1972) tested whether group therapy reduced re-offending rates. The results suggested that this approach to group therapy did not affect re-offending rates.

Unfortunately, meaningful experiments proved more difficult than anticipated to deliver in the field, and even robust experiments were often ignored by policy makers. As a result, until recently this experimental approach fell out of favour in social policy, except in the field of medicine.

The term ‘evidence-based medicine’ appears to have been first used by investigators from a US university in the 1990s where it was defined as ‘a systemic approach to analyze published research as the basis of clinical decision making.’ The evidence-based medicine movement considered experiments – specifically, collections of Randomised Controlled Trials (RCTs) subject to systematic reviews – as the “gold standard” of proof of whether interventions “work” or not.

Randomised controlled trials are sometimes not easy to undertake in social policy environments, but they can be done and they can provide surprising results. Starting in 2007, Birmingham City Council evaluated three evidence-based programmes in regular children’s services systems using RCTs[2]. We found that one programme (Incredible Years) yielded reductions in negative parenting behaviours among parents, reductions in child behaviour problems, and improvements in children’s relationships; whereas another (Triple-P) had no significant effects.

What was interesting for practitioners was that the children in all the trials had experienced improvements in their conduct. Only by use of a formal “control” group were we able to see that these “untreated” children were also improving, and so we were able to separate out the additional impacts of the intervention programmes.

There are a number of lessons from this and other past experience that can help practitioners wanting to deliver robust trials to test whether innovations are working (or not). The most important point is: build the evaluation testing into the design of the programme. The Troubled Families Programme could have built an RCT into the rollout of the programme – for example, selecting first year cases randomly from the list of families who were identified as eligible for the scheme. Or introducing the scheme in some council areas a year earlier than others. Or councils could have done this themselves by gradually rolling out the approach in different area teams.

Sandra Nutley and Peter Homel’s review[3] of the New Labour government’s Crime Reduction Programme stressed the importance of balancing the tensions between fidelity to “evidence based” policy (to maximise the chance of impact) and innovation (to ensure relevance to the local context), short-term wins and long-term learning, and evaluator independence (to ensure rigour) versus engagement (to help delivery).

In my final blog on the TFP next time, I explore the potential for “theory-based” approaches to evaluation helping us to understand “what works and why?” in this and other policy areas.

Campbell, D. T. and Russo, M. J. (1999) Social experimentation. Sage Publications, Inc.

Langley, M., Kassebaum, G., Ward, D. A. and Wilner, D. M. 1972. Prison Treatment and Parole Survival. JSTOR.

Nutley, S. and Homel, P. (2006) ‘Delivering evidence-based policy and practice: Lessons from the implementation of the UK Crime Reduction Programme’, Evidence & Policy: A Journal of Research, Debate and Practice, 2(1), pp. 5-26.

Rossi, P. H. and Lyall, K. (1976) ‘Reforming public welfare’, New York: Russell Sage.

Sanderson, I. (2002) ‘Evaluation, policy learning and evidence‐based policy making’, Public administration, 80(1), pp. 1-22.

White, M. (1999) ‘Evaluating the effectiveness of welfare-to-work: learning from cross-national evidence’, Evaluating Welfare to Work. Report, 67.

[1] Campbell, D. T. and Russo, M. J. (1999) Social experimentation. Sage Publications, Inc.

[2] Little, Michael, et al. “The impact of three evidence-based programmes delivered in public systems in Birmingham, UK.” International Journal of Conflict and Violence (IJCV) 6.2 (2012): 260-272.

[3] Nutley, S. and Homel, P. (2006) ‘Delivering evidence-based policy and practice: Lessons from the implementation of the UK Crime Reduction Programme’, Evidence & Policy: A Journal of Research, Debate and Practice, 2(1), pp. 5-26.

 

 

lowther-jason

 

Jason Lowther is a senior fellow at INLOGOV. His research focuses on public service reform and the use of “evidence” by public agencies.  Previously he led Birmingham City Council’s corporate strategy function, worked for the Audit Commission as national value for money lead, for HSBC in credit and risk management, and for the Metropolitan Police as an internal management consultant. He tweets as @jasonlowther

Councils going bankrupt? Don’t be Scilly!

Chris Game

 

Remember the old London bus joke: you wait for ages, then three come along at once? Well, some local government finance anoraks have been waiting ages for a 114, and now two 24s arrive, almost in mini-convoy.

Not buses, of course – but key sections of Acts of Parliament. Section 114 of the Local Government Finance Act 1988 requires a council’s Chief Finance Officer (CFO) to issue a s114 Notice reporting to all elected members an actual or impending seriously unbalanced budget.

As indicated, they’re infrequent, issued only in what CIPFA terms (p.3) “the gravest of circumstances” – current spending way beyond budget, reserves virtually exhausted, no imminent solution. Their impact too is grave – effectively freezing spending until councillors agree measures to achieve a balanced budget. But … they do keep the crisis in-house.

The alternative – being required to wash your dirty linen in public – is the dreaded Section 24 of the Local Audit and Accountability Act 2014. Here, a council’s external auditors append a written s24 ‘Recommendation’ to their Annual Audit letter, “copied to the Secretary of State”. The recommending and copying may sound chummy, but it’s the bullet-shaped chumminess of a Mafia ‘message job’. S24 Notices are very nasty, and happily very rare – or were.

Recently the Isles of Scilly Council received one, which is interesting in itself, since not everyone’s clear what it actually is: not part of the already enormous Cornwall Council, but a sui generis unitary authority – in a class of its own.

It’s considerably smaller than my own authority, Birmingham City Council (bear with me, it’s not that daft a comparison), with one-480th of its population – though, interestingly, one councillor for every 109 residents, compared to Birmingham’s post-2018 ration of one for every 10,900. And, as a genuine unitary, it has Birmingham’s major functions, including an airport, plus fire services, water supply, sea fisheries and coastal defence.

Unfortunately, Scilly’s unique status doesn’t exempt it from the austerity pressures and grant cuts faced by all English councils. It’s suffering badly, and its external auditors concluded that – with £3 million needed to pay staff and suppliers, no council tax income for the last two months of the financial year, and reserves down to £0.5 million – the law required Section 24, ‘recommending’ in terms (p.37) that the Council get its whole financial act together, extremely pronto.

Its impact, warning notwithstanding, can be surmised from the reaction of the eminent former Labour leader of the previous council to have received a s24 missive, just last November from coincidentally the same auditors, Grant Thornton. “The most concerning audit letter I have seen in all my [36] years on the council” was Cllr Sir Albert Bore’s verdict – the council being Birmingham, and the equivalent budget black hole not £3 million but pushing £38 million.

Just as Aristotle’s single swallow did not a summer make, two s24s don’t themselves make a systemic winter crisis. They’re surely, though, a sign – given that not one such report was issued to any council during the whole four-year 2010 spending review period, and we’ve now had two in two months.

But a sign of what? That’s rhetorical. I’m categorically not a local finance expert, and in this blog’s limited remaining space there are no answers – just three observations.

  1. Why no s114s?

 In the past we’d see them at least occasionally. Now there are rumoured sightings – e.g. in Northamptonshire – yet what have materialised are the two s24s, the proverbial nuclear option. It’s been suggested that the (post-1988 Local Finance Act) statutory duties of Social Services Directors mean a s114-prompted total spending freeze could prevent, say, a vulnerable child being placed in care. But CIPFA Chief Executive Rob Whiteman has rejected this interpretation.

  1. Please, not the B-word

There’s unfortunately no way of avoiding the media headline, but Scilly, Birmingham and probably any financially struggling English councils aren’t about to ‘go bankrupt’ in the sense in which the word is commonly understood. UK councils, unlike US local government or our own national government, are statutorily required to set each year a balanced budget. Running a deficit of the smallest fraction of Detroit’s nearly $400 million in 2013 is simply not possible – indeed, a very s(c)illy idea.

3. But watch for those ‘statutory duties’

So not bankruptcy in the normal legal sense, but almost daily signs and public warnings – Conservative Surrey’s seriously contemplated 15% council tax referendum, tax hikes all round of approaching 5%, hitherto protected adult care budgets now being cut and 13% of responding councils in the MJ/LGIU 2017 State of Local Government Finance Survey –reporting “a danger they would no longer have enough funding to fulfil their statutory duties in the coming year”.

Which would mean facing legal challenge for failing to meet those statutory duties and/or declaring ‘technical insolvency’. Not ‘bankruptcy’, note; but, as the famous duck test puts it: if it looks like, walks like, and quacks like a duck ….

 

Chris Game - pic

Chris Game is a Visiting Lecturer at INLOGOV interested in the politics of local government; local elections, electoral reform and other electoral behaviour; party politics; political leadership and management; member-officer relations; central-local relations; use of consumer and opinion research in local government; the modernisation agenda and the implementation of executive local government.

In tech we trust: A teacher’s perspective on INLOGOV’s e-learning (r)evolution

Dr. Abena F. Dadze-Arthur

 

INLOGOV’s first online Masters

It was a historic moment for INLOGOV – even by the standards of the Institute’s long and eventful history. For the first time ever, INLOGOV was to design and deliver an online International Masters in Public Administration (MPA). The new MPA was to be delivered wholly online with students doing all their classroom activities outside the traditional classroom, at a distance from their school or college, and supported by interactive technology tools. The programme was to be targeted at adult learners across the world, specifically those already working in the public sector, who wish to study part-time while maintaining their career paths. It marked INLOGOV’s accession to the new club of educational institutes partaking in an evolution of a rather revolutionary nature, which is interchangeably termed e-learning, distance learning or online education.

A worldwide (r)evolution

Indeed, in offering the new MPA, along with two other online postgraduate programmes, the University of Birmingham joined the ranks of other prestigious tertiary educational institutes that have embraced the challenge of delivering e-learning courses. After all, the business case is compelling. Online programmes are rapidly becoming not only an inevitable but also rather lucrative part of mainstream education. The worldwide market for e-learning already reached $35.6 billion in 2011, and generated estimated revenues of some $51.5 billion in 2016, boasting growth rates of 17.3% in Asia, 16.9% in Eastern Europe, 15.2% in Africa and 14.6% in Latin America.

International organisations, such as UNESCO and the International Council for Open and Distance Education, conclude that the revolutionary explosion of e-learning is down to two key factors: First, and perhaps most obviously, the technological advances of the Internet and web-based technologies offer learners and teachers a considerable range of affordable tools and resources. These enable novel approaches to networked learning, change the ways in which knowledge is being imparted and open up new means of engagement. Second, online education allows professionals all over the world to upskill and pursue further qualifications while continuing to work in their jobs. Moreover, working professionals from Africa and Asia are now able to overcome the inadequacies and asymmetries of local educational provisions by enrolling in e-courses delivered by internationally renowned universities.

But who teaches the teachers?

Indeed, a whole new way of learning that is free from the constraints of time, space and pace – but also a whole new way of providing education! Helpfully, from the learner’s perspective, there is a lot of information out there about the ways in which this type of education differs from traditional ‘brick-and-mortar’ programmes, and what to consider when registering for e-courses. However, from the teacher’s perspective, surprisingly little has been published about designing and creating online education, and finding practical solutions for pedagogical and technical challenges. Tasked with authoring and tutoring the very first module of INLOGOV’s online MPA, my co-convener and I felt like two fishes out of water. Although, as university teachers and researchers at INLOGOV, we have had much experience of designing and delivering public management programmes for mid-career public servants on a ‘face-to-face’ basis (both in the classroom ‘on-campus’ and ‘in-house’ for sponsoring client organisations in the public and voluntary sectors), the new online MPA seemed a daunting endeavour. Preparing and providing an ‘online’ distance-learning module for a more diverse international group of practitioners, drawn from a wider range of public service contexts and experiences, certainly raised new and partly unexpected challenges for us that called for fresh approaches. Since then we have delivered the module twice – and learned something about how ‘to do’ online education.

‘Doing’ online education from a teacher’s perspective

Working in partnership with the international education and publishing group Deltak-Wiley, we realized early on the need to research and write much of the MPA programme anew. Our existing PowerPoints, although helpful in visually highlighting or synthesizing complex arguments presented in a classroom lecture, were unsuited and reductionist for this mode of teaching. Hence, we spent weeks producing fully scripted learning materials of a high quality and publishable standard, which also featured animated videos and interactive diagrams, timelines and theoretical models that could be expanded or collapsed at the click of a mouse.

Mindful of the international nature of the student group for whom the programme is intended, we recognized the need to ‘internationalise’ our curriculum. This was achieved by several means: we added new literature on international public management and governance in the reading lists; we included a variety of contemporary examples of public management from around the world; we produced a series of short, BBC-documentary-style videos featuring practitioners and researchers from across the globe who discussed their particular experiences of public management and governance in their respective home countries; and we used an array of photo images to portray global diversity in public service delivery.

Encouraging critical reflection in relation to the students’ own experiences of working in public management posed another challenge that required fresh thinking. We tackled this issue by including weekly formative assignments, which asked the participants to share and discuss issues and examples from their own country contexts in ‘Discussion Forums’. These forums enabled us to get course participants critically to engage in the activities, rather than simply to absorb ideas from the text, animated videos, or short film clips. In addition, it firmly placed the students at the heart of learning, thus achieving learner-centricity.

Not having the classroom interaction meant that we needed to find different ways by which the students could develop rapport with, and respect for, one another and so learn from each other. We solved this challenge by making the ‘Discussion Forums’ interactive. In order to attract a good grade, our online students were not only expected to ‘post’ their contributions (by particular deadlines) on the ‘Discussion Forums’, but they also had to respond to the ‘posts’ of at least two others (by further deadlines). For the formal assessment of these postings, we chose to use two criteria: a) ‘intellectual contribution’ to the discussion, and b) ‘contribution to the learning community’ focusing particularly on responsiveness to colleagues’ ‘posts’. We thereby incentivized the learners carefully to read each other’s contributions and offer thoughtful and thought-provoking feedback, constructive advice and mutual support – all of which led to the development of a strong learning-community. We also built in two synchronous sessions, which are specified times when students and instructors hold virtual ‘meetings’ online in real-time. However, what can be done so easily in a face-to-face classroom environment proved rather difficult online. Following initial difficulties in identifying meeting dates and times that suited every student in three different continents and time zones, we encountered even more serious problems during the meetings with both the audio (delays and echoes) and the webcams (requiring too much bandwidth). Clearly, online instructors are not the only ones who still need to mature – the technology does too!

The most important lesson

With our first cohort of students soon due to graduate from INLOGOV’s online MPA, we are expecting systematic feedback and more lessons on what worked, or not, about our online teaching from the participants’ perspective. Not to mention that, as we mature as online teachers and are given the opportunity to tweak and adjust our online classes, and deliver them to new and different student cohorts, our insight and understanding as e-learning providers is bound to increase. However, since I first set out, armed with skepticism and furrowed brows, to join the e-learning (r)evolution, the biggest lesson I personally have learned is how rewarding and of high educational quality an online course can be.

 

If you are interested in more details about our authoring and tutoring of the first module of INLOGOV’s online Masters in Public Administration (MPA), please download our chapter by clicking here.

 

Untitled.png

Following a ten-year professional career as a public policy specialist working for various governments across the world, Dr Abena F. Dadze-Arthur switched to an academic career in public administration, and is currently a lecturer at INLOGOV. Abena teaches courses on various aspects of public management and governance. Her research mainly focuses on non-western and post-western public management approaches.

First, do no harm – An assessment of the Housing and Planning White Paper

Anthony Mason gives an initial assessment of the white paper on housing and planning in England

First impressions are not always very reliable.  When Sajid Javid replaced Greg Clark as Secretary of State for Communities and Local Government following the post-vote governmental putsch last year (sorry, change of Prime Minister following the referendum), local government figures were very wary.  Clark had, and still has, a reputation for understanding local government and can connect the local to the national in discussions around the cabinet table in a way that few of his colleagues are able to. Javid, however, was an unknown quantity – said to feel that the DCLG role was a demotion and giving every indication that he was unexcited by the move.

Yet, for those of us specifically interested in housing policy, Clark – alongside his spiky and confrontational housing minister, Brandon Lewis, presided over some rotten housing policies, as I suggested in this place last year.  Indeed, the Housing and Planning Act 2016 will, I suspect, go down in legislative history as the Dangerous Dogs Act of housing policy in England.  Gratifyingly, a number of the craziest measures enabled by that Act have proved so difficult to implement that the “new” government has simply shunted them into a siding and (we hope) left them there to rot.

And now comes the first comprehensive white paper on housing policy in England for almost a generation. Bearing the less than poetic title-as-ambition of Fixing our broken housing market.  Javid and his refreshingly rounded housing minister Gavin Barwell, set out in 104 pages and many supporting papers their ambitions to do just that.  To their great credit, Javid and Barwell have spent many weeks on careful consultation with local government, sector interests, and Number 10 before getting this far; delaying the publication of the white paper somewhat while doing so.  They have even persuaded the PM to pen a lengthy introduction to the paper – presumably in the hope of corralling rural Conservative NIMBYs into line.

The white paper sets out many proposals and poses 38 carefully framed policy questions for response (by 2nd May 2017, if you’d like to contribute).  But in quick summary, it:

  • Acknowledges that England needs around 250,000 new homes each year going forward. This was expressed as “between 225,000 and 275,000 homes” – and is up from the oft-quoted 200,000 previously accepted (but never consistently realised)
  • Proposes that each local authority will have to draw up and regularly review an “honest assessment” of local housing need – methodology to follow.
  • Says that developers could be forced to build within two years of planning consent, or see that consent lapse. At the moment, permission usually lapses after three years.  The paper also proposes new compulsory purchase powers for councils where sites lie undeveloped – details to follow.
  • Suggests an expanded and more flexible affordable homes programme, for housing associations and local authorities, with £7.1bn of (already announced) funding. It drops the “old” government’s fixation with starter homes in favour of a more balanced approach.
  • Encourages building rates at higher density – including of higher buildings – to make best use of land (and to avoid having to give a view on releasing green belt).
  • Dodges the question of future housing association and council rent levels after George Osborne’s compulsory rent reductions “We will provide clarity over future rent levels. In return, we expect them to build significantly more affordable homes over the current parliament.” Is what ministers promise.
  • Says that smaller building firms will be given assistance to expand, including support for off-site construction (where components are fabricated off-site and factory-assembled). It also encourages “build to let” where private companies build large-volume rental flats for tenants.
  • Continues a focus on leaseholds, proposing what it calls “an end to leasehold abuse” where home buyers are locked into leases with spiralling ground rents.

Most of us acknowledge the general need for new homes while protesting loudly if those homes are to be built near to us – and for years, housing policy in England has tried not to upset voters and yet deliver new homes.  And the white paper has had to throw titbits in all directions to keep sector interests at bay.  Local authorities are both excoriated for planning failures and mildly encouraged to build new homes.  Those who worship at the altar of home ownership will be pleased that there is a threat to close a loophole that has allowed councils building homes through wholly-owned companies to avoid the right to buy.  Those who see renting as the most realistic way forward will be pleased that much of the white paper acknowledges this reality and makes gentle proposals for longer tenancies.  Big developers are both criticised for not building out sites as well as encouraged by some anti-planner language.

But ministers have failed to resolve some longstanding conundrums – and a couple of new ones – in their paper:

  • Successive governments have tried to combine bottom-up and top-down policies on housing which appear to conflict in their efforts to encourage and coerce. For example, communities and parishes have been given more control over developments and yet principal councils are still required to provide new homes.  Housing associations should develop more and yet have no control over the rents they can charge for these new homes.
  • Government has long had an intellectual tendency to support developers over planners – even though planning consents have been running ahead of homes built for some time. This white paper at last begins to recognise that not all is well, with our developers while avoiding the obvious response: councils’ potential contribution to building at scale.
  • There is a cherished belief that brownfield sites can provide the majority of our new homes, but these sites no longer match need. Not surprisingly, they are disproportionately in cities, but not all housing need is city-based.  The white paper avoids the question of building on the green belt, even though, in our own city, we’ve faced a highly charged debate about this topic.
  • A further concern is around labour and skills. We’ve long worried that not enough UK youngsters express any desire to work in the building industry.  This is now compounded by fears of the actual or apparent impact of Brexit on the non-UK workforce.

The fundamental question that the paper avoids is whether any combination of our present arrangements for building can ever deliver the amount of housing we need; as the answer to that question may be too hot to handle.  It’s old evidence now, but the Calcutt review of the housebuilding industry commissioned a decade ago set out a straightforward graphic showing who has built what in the UK in the years since the Second World War (see figure one on page 10).

untitledThis evidence was summarised in a beautifully simple graphic (above) by the University of Sheffield School of Architecture.  It evidences that the three decade long gap in our housing provision is simply because we’ve stopped building council houses.  The answer to the fundamental question would seem to be to let councils (and housing associations) build again at some scale in order to supplement the relatively fixed-but-declining contribution of private developers.

The title to this post is a common misquotation of the Hippocratic Oath.  It suggests that a first duty for medics is not to do harm “Primum non nocere” – and the new white paper seems to pass that test, at least.  If a second duty is “then to do good” – then I’m not yet convinced that the paper will achieve that in any significant way.

Anthony Mason

 

 

Anthony Mason is an Associate at INLOGOV and works mostly on local government systems and organisation and on improving public sector partnerships.  His early career was in local government followed by more than 20 years in PwC’s public sector consultancy team.

 

 

Troubled Families: Two Secrets to Great Evaluations

Jason Lowther

In this blog last week I explored the (rather flimsy) evidence base available to the developers of the original Troubled Families Programme (TFP) and the potential for “theory of change” approaches to provide useful insights in developing future policy. This week I return to the formal TFP evaluation and look at the lessons we can learn in terms of the timing and data quality issues involved.

The first secret of great evaluation: timing

The experience of the last Labour Government is very instructive here. New Labour appeared as strong advocates of evidence-based policy making, and in particular were committed to extensive use of policy evaluation. Evaluated pilots were completed across a wide range including policies relating to welfare, early years, employment, health and crime. This included summative evaluations of their outcomes and formative evaluations whilst the pilots were underway, attempting to answer the questions “Does this work?” and “How does this work best?”

Ian Sanderson provided a useful overview of Labour’s experience at the end of its first five years in power[i]. He found that one of the critical issues in producing great evaluations (as for great comedy), is timing. Particularly for complex and deep-rooted issues (such as troubled families), it can take a significant time for even the best programmes to have an impact. We now know the (median) time a family remained on the TFP programme was around 15 months.

It can also take significant time for projects to reach the “steady state” conditions, which they would work under when fully implemented. Testing whether there are significant effects can require long-term, in-depth analysis. This doesn’t fit well with the agenda of politicians or managers looking to learn quickly and sometimes to prove a point.

Nutley and Homel’s review[ii] of lessons from New Labour’s Crime Reduction Programme found that “projects generally ran for 12 months and they were just starting to get into their stride when the projects and their evaluations came to an end” (p.19).

In the case of the Troubled Families Programme, the programme started in April 2012, and most of the national data used in the evaluation relates to the 2013-14 financial year. Data on exclusions covered only those starting in the first three months of the programme, whereas data on offending, benefits and employment covered families starting in the first ten months of roll-out.

We know that 70% of the families were still part-way through their engagement with the TFP when their “outcomes” were counted, and around half were still engaged six months later.

It’s now accepted by DCLG that the formal evaluation was run too quickly and for too short a time. There just wasn’t time to demonstrate significant impacts on many outcomes.

The second secret: data quality

Another major element of effective evaluation is the availability of reliable data. Here the independent evaluation had an incredibly difficult job to do. The progress they have made is impressive – for the first time matching a wide range of national data sets, local intelligence and qualitative surveys. But at the end of the day the data quality base of the evaluation is in places poor.

The evaluation couldn’t access data on anti-social behaviour from national data sets, as this is not recorded by the police. This is unfortunate given that the strongest evidence on the effectiveness of TFP-like (Family Intervention) programmes in the past concerns reducing crime and anti-social behaviour[iii].

A chunk of data came from the 152 local authorities. This data was more up to date (October 2015), although only 56 of the councils provided data – which enabled matching to around one quarter of TFP families. The evaluation report acknowledges that this data was “of variable quality”. For example, the spread of academy schools without a duty to co-operate meant there are significant gaps in school attendance data. This will be a serious problem for future evaluations unless academies’ engagement with the wider public service system is assured.

In summary, the TFP evaluation covered too short a period and, despite heroic efforts by DCLG and the evaluators, was based on data of very variable quality and completeness.

Next time we will explore the “impact” evaluation in more detail – looking at how designing a more experimental approach into this and future programmes could yield more robust evaluation conclusions of what works where.

[i] Sanderson, Ian. “Evaluation, policy learning and evidence‐based policy making.” Public administration 80.1 (2002): 1-22.

[ii] Nutley, Sandra, and Peter Homel. “Delivering evidence-based policy and practice: Lessons from the implementation of the UK Crime Reduction Programme.” Evidence & Policy: A Journal of Research, Debate and Practice 2.1 (2006): 5-26.

[iii] DfE, “Monitoring and evaluation of family intervention services and projects between February 2007 and March 2011”, 2011, available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/184031/DFE-RR174.pdf

 

 

lowther-jason

 

Jason Lowther is a senior fellow at INLOGOV. His research focuses on public service reform and the use of “evidence” by public agencies.  Previously he led Birmingham City Council’s corporate strategy function, worked for the Audit Commission as national value for money lead, for HSBC in credit and risk management, and for the Metropolitan Police as an internal management consultant. He tweets as @jasonlowther