Dr Foster’s day out in the sun: the use and abuse of hospital mortality rates

Chris Game

It was an odd happenstance that Dr Foster – a gentleman best known for his rain-ruined, nursery rhyme expedition to Gloucester – should have his proverbial 15 minutes of contemporary news fame in the middle of last week’s heat wave.

The unfortunate doctor, you may recall, went to Gloucester in a shower of rain. Ignoring the truly excruciating rhyme ahead, he stepped in a puddle, right up to his middle, and never went there again.

The news to which this doggerel relates is, of course, allegedly failing hospital trusts, and specifically Hospital Standardised Mortality Ratios (HSMRs) – the widely used measures of hospital death rates developed, publicised, defended and refined by today’s equally fictional Dr Foster.

The doctor and his HSMRs took rather a beating in last week’s report by Prof Sir Bruce Keogh, National Medical Director for the NHS in England: “However tempting it may be, it is clinically meaningless and academically reckless to use such statistical measures to quantify actual numbers of avoidable deaths” (p.5).

Strong words, and justified – because this is precisely what had happened the previous weekend, with numerous media claims that the report would be about 13,000 ‘needless deaths’ at the 14 NHS hospitals selected, because of their high mortality rates, for special investigation. It wasn’t.  The report contained no such numbers, and instead provided detailed, focused recommendations to assist the improvement of the hospitals’ serious but not irremediable problems.

Sir Bruce’s report had been calculatedly hijacked, but who he held chiefly responsible – Ministers and their advisers, the media, even some collusive involvement of Dr F himself – was unclear. The outcome, sadly, was unmistakeably clear. Health Secretary Jeremy Hunt’s parliamentary presentation of the report became a shameful partisan blame-fest – so depressing for so important a topic that, as a completely non-expert observer but low-key Dr Foster fan, I was moved to attack my keyboard.

I remember well my own first encounter with Dr Foster in January 2001.  I was teaching a course here at Birmingham University on policy research methods, and in, of all places, a two-part Sunday Times supplement, there appeared some near-perfect raw material for a student assignment: the first ever listing of standardised ‘death rates’ (HSMRs) for England’s or any other nation’s hospitals.


So what, my students discussed, were these ‘metrics’, and what did they really measure?  What did they include, and exclude?  Who’d collected and analysed the data? How did they relate to other possible measures of a hospital’s care and performance? What was the range, and where were the highest and lowest ratios – that ‘Where?’ question providing an additional reason for my recalling that first Dr Foster’s Good Hospital Guide.

A hospital’s Standardised Mortality Ratio is usually presented as a percentage: the recorded deaths in hospital from most (but not all) diseases, as a percentage of the number that would normally be expected, after taking account of, or standardising for, a wide range of factors concerning the patients and the nature and severity of their illnesses.

HSMRs’ other key feature, consistently misunderstood, is that they measure hospitals not against some objective clinical standard, but against each other. An HSMR of 100 is the national average; below 100 means fewer deaths than statistically expected; over 100 means more. Not needless, preventable or avoidable deaths, not deaths from incompetent care, simply more than statistically expected. Even if all hospitals were good, half would still have ratios of 100+ and look ‘bad’ – and vice versa.

The Dr Foster Guides and website emphasise these points scrupulously. A high HSMR should be treated as a warning: a risk, but not proof, of failings in care, and reason for further investigation, with attention focusing mainly on ‘outliers’ – those outside, especially if repeatedly outside, the normal range. University Hospitals Birmingham NHS Foundation Trust’s HSMRs, though consistently over 100, are thus less immediately concerning than the 130+ ratios of Basildon & Thurrock (2005-09) and Mid Staffordshire (2005-07).

However, as Sir Brian Keogh noted, in the dash for political advantage or media headlines, the temptation to elbow aside these literally health warnings is powerful indeed. So, although those first hospital ratios weren’t listed in league table format, they were quickly sorted into one and the range calculated.

It was wide and, although all mortality rates have fallen significantly in the past decade – and, of course, the HSMR baseline adjusted accordingly – it remains so today. Then, University College London Hospitals had the lowest ratio of 68, and most of the low ratios were in London and the South-East. But two of the three highest were on our proverbial doorstep in the West Midlands: Walsall Hospitals Trust with 119 and Sandwell with 117.

My recollection is that these hospitals and trusts, not to mention their patients, had little advance notification of their figures. Certainly, there were widespread protests – by those assuming that, if this was a ‘Good Hospital Guide’, high-ratio hospitals must be ‘bad’. However, despite their susceptibility to such misinterpretation, HSMRs were here to stay. Which begged the obvious question: who was this pioneering but troublesome Dr Foster?

As already indicated, there is no actual Dr Foster. The name was the whimsical invention of two journalists involved in producing the 2001 Sunday Times supplements. But, if there were a real doctor, the only possible candidate is someone you may well have seen recently on your TV screens, Professor Sir Brian Jarman.

A one-time GP who by the 1990s had become a distinguished Imperial College academic, he developed the ‘Jarman Index’ – a formula for distributing government funding to the nation’s hospitals – which gradually evolved into the HSMR, a formula for identifying a hospital’s share of responsibility for its death rates. It was a major statistical advance, but the then Health Secretary was nervous and refused Jarman permission to publish individual hospitals’ HSMRs.

He took his stats, therefore, to two journalists rather more committed to the idea that transparent, debatable research findings and more informed patients had key roles to play in improving health care: the Sunday Times’ Tim Kelsey and the Financial Times’ Roger Taylor. The outcomes were swift and far-reaching: the first of the now annual Dr Foster Good Hospital Guides, and Dr Foster Intelligence – an initially private company that since 2006 has been half-owned by the Department of Health (another controversial development) and is today an internationally renowned provider of healthcare information.

And the drivers of almost all this growth, and indeed of the career progression of the key actors, have been HSMRs – which might surprise some of my 2001 students, who had no difficulty identifying what they saw as potential weaknesses.

Yes, HSMRs are a purely statistical exercise – no visits, inspections, interviews or case notes. Yes, if the indicators in the formula change, so too could the ratios. Yes, they record only in-hospital deaths, and not even all of them. Yes, they surely could be manipulated – by discharging terminally ill patients into hospices or ‘the community’, or (as three West Midlands trusts were later accused of doing) by stretching the ‘admitted for palliative care’ code and thereby raising the expected death rate. And yes, it does seem a rather blunt way of measuring quality of care – or indeed the overall performance of a large hospital.

To their credit, many hospitals’ response to a high HSMR has been to work with the Dr Foster team, to try to understand better the causes and thereby bring the ratio down. Walsall, for example, reduced its HSMR in five successive years, down to 103 by 2005/06.

There have also, though, been continuous criticisms of both HSMR methodology and interpretation – from health care professionals, the media and academia – particularly after 2007, when some of Dr Foster’s statistical ratios contradicted the inspection-based assessments of the Care Quality Commission.

There followed the first Francis Inquiry into the Mid Staffordshire NHS Foundation Trust, and with it the development and official approval of a new, more comprehensive mortality measure – the Summary Hospital-level Mortality Indicator (SHMI) – covering all, instead of most, in-hospital patient deaths, plus those occurring up to 30 days after discharge from hospital.

The two measures sound similar, and frequently they produce broadly similar results, as shown in the 2012 Dr Foster Guide. Birmingham’s HSMR is 112, its SHMI 105; Sandwell & West Birmingham 99 and 97; Coventry & Warwickshire 103 and 107; Walsall 117 and 113; Royal Wolverhampton 100 and 103.

But they can differ significantly – and did for several of the 14 trusts investigated in the Keogh Report. You might think that the Government, having finally found in SHMIs a more comprehensive mortality measure than HSMRs, which most statisticians and clinicians seem to accept as more reliable, would use it to select the hospital trusts it wished to have investigated.

Wrong!  The supposedly failing trusts were picked because of being high ‘outliers’ for two consecutive years (2010/11 and 2011/12) on either of the two measures. So Tameside and Basildon/Thurrock, for example, were included apparently because of their higher than expected SHMIs, but Burton and Sherwood because of higher than expected HSMRs.

We’re into circumstantial evidence here. But, suppose you were a Government keen to rubbish Labour’s NHS record and frighten patients and electors into viewing further privatisation more favourably. It surely wouldn’t seem a bad tactic to maximise the number of allegedly  failing ‘killer’ hospitals – 14 is nearly one in 10 of England’s acute hospital trusts – and feed the media scare stories about thousands of ‘avoidable’ deaths. Or has my imagination run away with me?


Chris Game is a Visiting Lecturer at INLOGOV interested in the politics of local government; local elections, electoral reform and other electoral behaviour; party politics; political leadership and management; member-officer relations; central-local relations; use of consumer and opinion research in local government; the modernisation agenda and the implementation of executive local government.

Picturing place: citizen participation in the age of social media

Katherine Tonkiss

The INLOGOV blog has featured a number of insightful pieces on citizen participation in recent months. Most recently, Laurens de Graaf reflected on the limited role of citizens in participatory projects, where they typically act as information sources for elected representatives rather than decision-makers themselves. Previously, Catherine Durose argued in favour of alternative modes of citizen participation in order to move away from often empty, ‘tick-box’ consultation processes. Further, Catherine Jackson-Read reflected on whether local government in its current form can work effectively in collaboration with citizens.

What these posts have in common is a consensus that facilitating effective citizen participation is a significant challenge for local government, and that authorities should look to more novel approaches to facilitating participation beyond the traditional meeting in the drafty village hall.

These posts sprang to mind when I came across a campaign being run by Birmingham City Council’s Fair Brum partnership, ‘Place Matters’. The purpose of this project is to facilitate the participation of citizens in shaping Birmingham’s neighbourhood strategy by submitting photographs of their neighbourhood via social media. The focus is on ‘what is distinctive about different neighbourhoods and what local people value in their local environment’.

Photographs should answer one of the following questions:

  1. What do you like about your area?
  2. What makes your area unique or distinctive?
  3. What would you change about your area?

This novel campaign relates to the idea of ‘place’ in two very interesting ways.

First, the campaign involves a notion of place strongly grounded in the neighbourhood. The idea of citizens telling their local authority and its partners about their neighbourhood in terms of what it is like to live there doesn’t just involve relaying information to assist decision-making, but actually resconstructs what place means to the citizen in their immediate locality and how they interact with that place. In doing so, this creates a vision of place from the ways in which people understand and interpret their lived environment.

Secondly, and conversely, the campaign involves a very expansive notion of place. The act of photographing the neighbourhood and uploading it via social media is a clear step away from engaging citizens in that drafty village hall, and rather opens up the ability to convey ideas about place from the home – very much along the lines of the Gov 2.0 model that Tom Barrance wrote about a couple of weeks ago. It also opens up the possibility of participation to those without English language skills, or to those who are otherwise unable to engage in traditional processes of local democracy. Previous research I have been involved in has highlighted how traditional models of citizen participation can further exclude some of the most underrepresented groups, and alternatives such as this offer the opportunity to overcome such barriers.

I acknowledge that it will still exclude those who don’t use social media, however this is part of a raft of engagement activities and so there will, presumably, be other ways of engaging that don’t necessarily rely on having a Twitter account.

The results of this exercise will be insightful for local authorities and academic researchers alike, in terms of whether it does address that all too common issue that participation activities become tokenistic opportunities to obtain information rather than to engage citizens in decision-making processes. It will be important for the partnership to demonstrate a link between these participation activities and meaningful citizen input into the decision-making process about the neighbourhood strategy. If successful, the exercise will offer fascinating insights both into Birmingham as a city and into citizen participation in the neighbourhood.


Katherine Tonkiss is a Research Fellow at INLOGOV.  She is currently working on a three year, ESRC funded project titled Shrinking the State, and is converting her PhD thesis, on the subject of migration and identity, into a book to be published later this year with Palgrave Macmillan.  Her research interests are focused on the changing nature of citizenship and democracy in a globalising world, and the local experience of global transformations.  Follow her Twitter feed here.

The ‘Detroit question’ and Parish Councils

Ian Briggs

The agenda on public service integration continues apace: discrete organisations working together in partnership has been the ideal in the past, but we are now in a world where the boundaries between organisations are becoming blurred and important questions are being asked again around the overall structure of our public services.

We recently hosted an interesting roundtable for Police and Crime Commissioners here at INLOGOV where the debate somewhat expectantly turned to the potential advantages of gathering the so called ‘Blue Light’ services together. Indeed, there has been a lot of progress in this field: certain first responder paramedic services can and are delivered by fire fighters and there are other examples of this integration which make a great deal of sense. It makes sense economically and, importantly, from the public perspective it makes sense too. Little attention is paid by a citizen or service user as long as the need is met and the service adds value. So we can safely say that some useful, realistic and economically advantageous integration projects are now well underway.

However, there is one aspect of integration that is rarely discussed but appears to be gathering a bit more attention recently. It goes something like this…. “As a District Council we are a billing authority, we send out the council tax bills and residents can see proportionately where their council tax is being spent. They can see that a proportion is retained by us; they can see a fair bit goes to the county council, the Fire service and police and little bit at the end is called the precept for the Parish Council. When we look at the totality for the parish and town councils precept it adds up to a fair sum. Why is it that when we can hardly afford the electricity bill to keep the lights on in our district council, the parishes and town councils have a load of loot?”

The councillor in this conversation also goes on to say that they have invested a great deal of time and energy in reshaping services and entered into complex partnering arrangements to bring efficiency benefits for us and the county, but “we seem to have overlooked the role of parish and town councils”. Looking recently at one multi tier area in the West Midlands that is heavily and actively parished suggests that the total precept adds up to quite a few million pounds; a rough calculation also suggests that for most town and parish councils they have discretionary control over more than half of their budgets (according to CLG the total English council precept for town and parish councils is over £540m -2013/13).

Decisions on spend, often set against some reasonably realistic and thoughtful parish plans, do deliver real and appreciated benefits for local communities but those parish plans, whilst submitted for approval to the district council (the billing authority in question), are administratively accepted and gather dust just to be treated as another bit of ‘administrivia’.

Granted, some councils are working well with their parish and town councils. They go beyond mere consultation and actively engage them in priority setting and are working towards much stronger integrated working, but that is not the case in many instances. There must some mileage in extending this debate to have a near seamless integration of priorities and social outcomes that dig deep into the work that parish councils do. Why is it that when the parish council budget is set it often accounts for a myriad of small contracts that are judged upon their worthiness but are rarely bound into the higher order outcomes that higher tier councils are working towards? It is logical to think that where a parish council has responsibility for open spaces and its efficacy in the management of those places is judged upon how effectively the grass is cut that those same open spaces are a resource that can have a significant role in meeting outcomes around public health, wellbeing, youth activities and a whole host more that are sought in higher tier councils.

The problem here is twofold – firstly it is rare for local parish and town councils to be engaged with these issues and secondly it is equally rare for many higher tier councils to even try to do so. There are also the attendant issues of poor integration of expenditure and budgets.

Although I have tried to bring together neighbouring parishes to coordinate and share their basic contracts (as a kind of horizontal integration that brings an economy of scale) little has been achieved and there has been little progress in taking this debate upward too.  As we are becoming more outcomes focused it is somewhat surprising that more is not done to encourage an integrative approach to what parish councils are doing on the ground and what the higher tier councils are seeking to achieve in this respect.

There have been some recent initiatives that have sought to bring this about – Selby in North Yorkshire should be commended for their approach to grouping parish and town councils and one county in the West Midlands that we are aware of has undertaken some preliminary work to engage with parish ad town councils better. However, where there have been some attempts to bring about this vertical integration, some resistance can be found too. It centres on the lack of focus on larger more holistic outcomes to be found in some, though perhaps not all parish and town councils and the potential lack of administrative and client skills they can call upon. But there may also be a lack of will on behalf of higher tier councils to stimulate this debate – it can be a tall order to effectively engage with a varied and disparate group of organisations that quite rightly guard their independence and local connectivity and in some cases district councils can have upwards of hundreds of parishes to deal with – engaging with them can be resource intensive in the extreme.

So, as some councils are facing highly uncertain futures – there is some polemic in the media about a handful of councils pulling up the shutters soon – it might be time to open this debate up and look at where there could be useful approaches to vertical integration in local government. It might be the first step in avoiding a UK Detroit.


Ian Briggs is a Senior Fellow at the Institute of Local Government Studies. He has research interests in the development and assessment of leadership, performance coaching, organisational development and change, and the establishment of shared service provision.

Examining citizen participation: theory and practice

Laurens de Graaf

As a researcher of citizen participation I often discuss the functioning of local democracy with, among others, councillors, officers and citizens. These discussions are showing that knowledge of democratic theory in the field is not often very present.

Partly, this is understandable –if the field consisted of political scientists only, would democracy function at all? But it seems as if limited knowledge about democracy creates some practical problems. To put it more precisely, the perspective on democracy appears to depend on the slogan: ‘where you sit is where you stand’. Councillors see themselves as guardians of democracy, because they are (the only ones) elected, and are the representatives of the people. Officers don’t often understand the (seemingly) irrational decisions councillors make and see democracy often as frustrating for their policy process. Citizens are distant observers and only a few committed citizens are actually participating in democratic processes.

Councillors and officers have been aiming for (more) citizen participation since the 1990s. But what effect does citizen participation have on local democracy?

Citizen participation is vital to democracy

Citizen participation is usually seen as a vital aspect of democracy. Many theorists claim that citizen participation has positive effects on the quality of democracy. Theories of participatory democracy, deliberative democracy and social capital assert that citizen involvement has positive effects on democracy. It contributes to the inclusion of individual citizens in the policy process, it encourages civic skills and civic virtues, it leads to rational decisions based on public reasoning, and it increases the legitimacy of the process and outcome. These aspects are summarized in the table below.

Aspects of democracy Clarification Theoretical Perspective
Inclusion Allow individual voices to be heard (openness; diversity of opinions) Social capital & Deliberative democracy
Civic skills and virtues Civic skills (debating public issues, running a meeting) and civic virtues (public engagement and responsibility, feeling a public citizen, active participation in public life, reciprocity) Participatory democracy & Social capital
Deliberation Rational decisions based on public reasoning (exchange of arguments and shifts of preferences) Deliberative democracy
Legitimacy Support for process and outcome Participatory democracy

Table: Aspects of citizen participation and democracy; a framework for analysis

What councillors and officers are telling me is that they are not fully aware of all these different aspects, but like the overview. It helps them to reflect on democracy from different angels.

Local participatory policymaking in the Netherlands

My article – co-authored by Ank Michels – examines the probability of these claims for local participatory policymaking projects in two municipalities in the Netherlands. However, I think that the claims can also be applied to local democracy in the UK and other countries. The article focuses on the relations between citizens and government from a citizens’ perspective.

The findings show that the role of citizens in participatory projects is limited, serving mainly to provide information on the basis of which the government can then make decisions. Nevertheless, the article argues that citizen involvement has a number of positive effects on local democracy: not only do people consequently feel more responsibility for public matters, it increases public engagement, encourages people to listen to a diversity of opinions, and contributes to a higher degree of legitimacy of decisions. One negative effect is that not all relevant groups and interests are represented. The article concludes that for a healthy democracy at the local level, aspects of democratic citizenship are more important than having a direct say in decision-making.

Reflecting on the functioning of (your local) democracy can be a fruitful exercise once in a while. The framework of analysis that was presented here may help, among others, councillors, officers and citizens to understand democracy more broadly and empathise with (each) other’s perspectives and roles.

A full account of this research is available in my recent article with Ank Michels: ‘Examining Citizen Participation: Local Participatory Policy-making and Democracy’. Local Government Studies 36 (4), 477-491.

Laurens de Graaf is a lecturer at Tilburg School of Politics and Public Administration, Tilburg University, The Netherlands. In the last ten years he conducted theoretical and empirical research with regard to citizens participation and in a broader sense: the functioning of local democracy. He is often in the field moderating workshops and trainings for councillors, mayors, active citizens and (neighbourhood) professionals about their role and their potential added value to local democracy.

Reflecting on the doctoral take-over

Stephen Jeffares

Over the last ten days the INLOGOV blog has reflected some of the great talent we currently have within our PhD cohort. INLOGOV has 28 students, a quarter of which are part time and working in public service. We like to think this gives a distinct flavour to our programme.

The blog over recent days profiled some of the great work among current students and reflections from previous INLOGOV graduates of our doctoral programme.

Becky outlined her current work exploring the role of evidence in decisions around High Speed 2, Abena’s blog compared how western and non-western states have differing approaches to public management, George described the relationship between financial crisis and citizen participation in Greece, and Thai student Pobsook offered her reflections on her first year of PhD study. Abena, George and Pobsook’s posts reflect the international dimension to our programme, with students from the USA, south east Asia, the middle east and continental Europe.

They also reflect our culture of sharing work in progress among peers. This culture is reflected in our monthly PhD showcase sessions, where researchers share their work with the department. We deliberatively hold the session in our open corridor space rather than a seminar room – although sometimes cramped and with a limited view of the projector screen, this offers an opportunity to get together to discuss ideas and eat cake.

Back to the blog. Tom’s post explored a growing theme in local government, the advance of digital technology and the preparedness of public services to make the most of these opportunities. INLOGOV’s blog and Twitter followership continues to grow, something we think is a reflection of the increased use of social media platforms among today’s policy actors. Pete reflected on blogging itself as a means for doctoral researchers to develop an academic profile, develop networks and refine their arguments.

Our former students have reflected a diverse range of post-PhD experiences in their blog posts. Four of the posts, from Tatum (now a Research Fellow in the University of Birmingham’s Business School), Katie (a Research Fellow here at INLOGOV), Thom (now at Oxford Brookes) and Mark Roberts (at De Montfort) are examples of how many of our doctoral researchers secure academic positions after leaving us. Mark Ewbank’s blog also demonstrated how many of our students carve out successful careers in public service after graduation.

For more information on applying to our doctoral programme please see our website or contact Stephen Jeffares.


Stephen Jeffares is a Roberts Fellow in the College of Social Sciences based in INLOGOV, and is also INLOGOV’s Director of Doctoral Research.  His fellowship focuses on the role of ideas in the policy process and implications for methods.  He is a specialist in Q methodology and other innovative methods to inform policy analysis.

Finding an academic home

Mark Roberts

I graduated from Birmingham in 2008 and initially found it difficult to leave INLOGOV, where I enjoyed my time completing my PhD. However I live near Leicester and, by chance, shortly after I left INLOGOV several friends and colleagues moved to the Local Government Research Unit at De Montfort University. Initially I took on a number of short term contracts as a visiting research fellow, mainly helping with field research on public participation and neighbourhood working. The turning point came when, encouraged by colleagues at DMU, I apply for a Research Fellowship with the Arts and Humanities Research Council which was focused on a Citizen Power project in Peterborough.

While I wasn’t successful in gaining the fellowship itself, I was offered an award from the AHRC for a research project on Understanding the Impacts of Citizen Participation in Peterborough. This award funded my work on a more stable basis for two years and, in addition to reports and journal papers, one of the outputs from that research is an RSA animation aimed at disseminating the main findings to a non-academic audience.

When I was not interviewing and observing in Peterborough, I was teaching and working on a book with Vivien Lowndes. We had a common interest in applying institutional theory to political analysis and she had a contract with Palgrave Macmillan to write a book on the subject. We put together some draft chapters pretty quickly and tested them out by going up to the Politics Department at the University of Sheffield, where Vivien gave some masterclasses to postgraduate students. The feedback we received from that and other reviewers was very positive and we ploughed on.

I should have learned from the PhD process, I guess, that the actual writing is only half the battle, and getting the thing ready for publication (references, indices, blurb, text for the cover) can take a long time. Anyway, the book was published on 31st May of this year in the Palgrave Macmillan Political Analysis series under the title ‘Why Institutions Matter: the New Institutionalism in Political Science‘, and has been a source of both surprise and amusement to my friends and family.

Perhaps the best advice I was given after graduating was from my PhD supervisor and was about finding ‘an academic home’. That can be quite difficult for many graduates who may have spent four, five or more years in the same academic environment. I have been lucky that DMU was on my doorstep and a number of colleagues moved there at the same time. From here I aim to keep the same balance going between teaching, research and writing, making sure I get out into the field as often as possible, and keep in touch with friends and colleagues from INLOGOV and elsewhere.


Mark Roberts is a Research Fellow in the Department of Politics and Public Policy at De Montfort University, Leicester. His research interests include citizen participation, neighbourhood working, new institutional theory, interpretive analysis and the influence of religion and race in urban politics. Before completing his PhD at INLOGOV, Dr Roberts worked in local authority social work for twenty nine years, with his last post being Deputy Director of Social Services in Sandwell MBC in the West Midlands.