From literally his first day as leader, part of David Cameron’s project of ridding the Conservatives of their ‘nasty party’ image has involved, to quote his acceptance speech, changing “the scandalous under-representation of women in the Conservative Party” and generally diversifying the profile of its MPs and councillors. Stirring stuff, were such a change within his power. However, the “scandalous under-representation” – 17 women out of 198 Tory MPs in 2005 – was created not by his predecessors, but by autonomous local constituency associations who jealously guard their right to select their own candidates – with minimal interference from the national party, thanks very much.

Neither the party nor Cameron will accept anything seriously effective, like all-women shortlists or legally enforceable gender quotas. So they’re left with what they call “equality rhetoric and promotion measures” to increase the selection of minority groups – like the notorious Conservative Central Office ‘A-list’ of some 150 favoured women, black/minority ethnic (BAME) and disabled candidates, from which it was ‘expected’ that selectors in Conservative-held target seats would make their choice, at the expense, if necessary, of their own colleagues or preferred local candidates.

Perhaps surprisingly, some actually did, and in 2010 the party’s women MPs rose to 43, or one in seven of the parliamentary party, and its BAME MPs to 11. The A-list had served its immediate purpose, and the plan was for the diversification project to be driven in the new Parliament by another scheme favoured by both Cameron and the Lib Dems: open primaries – shorthand for the nominating primary elections in which US parties select their Presidential and legislative candidates.

Our recent candidate selections have been mostly through what could be termed membership or closed primary elections. Applicants are shortlisted by local party officials, but the final selection is open to all registered constituency party members – either at a meeting, or through a membership ballot – but closed to non-members. Then gradually during the last Parliament the Conservatives started opening up their selection process, eventually allowing non-party members to participate in the actual selection. We had the unrestricted open primary, in which all registered voters in a constituency, including members of other parties or no party, could participate.

Boris Johnson gave a massive kick start both to the idea and to his own successful 2008 London Mayoral campaign when he was chosen, overwhelmingly, as the Conservatives’ candidate in an open primary in which over 20,000 electors invested around £1.50 to call a premium rate phone line and register their votes.

Over a hundred open primary meetings were held between 2006 and 2010, but Cameron’s initial vision was to do the thing properly, Boris-scale, with postal ballots mailed to all registered voters in a constituency. The first in 2009 was in the Devon constituency of Totnes, and Sarah Wollaston, the winning candidate with nearly 8,000 votes, ticked just about all Cameron’s boxes. A Plymouth-based GP and mother of three, who’d only joined the party in 2006 to oppose the closure of the local community hospital, Wollaston would replace an embarrassing veteran MP, forced to stand down following the revelation of his abuse of parliamentary expenses for maintaining his country home and ‘rabbit protection’. A few months later a second postal primary in Gosport, Hampshire, was a virtual carbon copy. This time the expenses fiddler was the MP whose £30,000 gardening expenses included a £1,645 ‘floating duck island’, and his replacement was Caroline Dinenage, a local councillor, businesswoman, and mother of two.

To the incoming Coalition leaders, these postal primaries must have seemed like a magic potion: the key to cleaning up and transforming the membership of Parliament, opening up candidate selection in safe seats, and extending public participation all in one go. Certainly it went to their heads, for, at the height of an international financial crisis, they included in the Coalition Agreement the pledge to “fund 200 all-postal primaries over this Parliament, targeted at seats which have not changed hands for many years” – the funds to be allocated to all Parliamentary parties in proportion to their 2010 election vote.

With Totnes and Gosport having cost roughly £40,000 each, that would be £8 million-plus, and unsurprisingly it’s become one of the less contentious Coalition pledges to have fallen off the cart. What has continued, though, is that local Conservative Associations have been deciding for themselves to forgo the exclusive selection privileges to which their £25 membership fees entitle them, and to turn their final selection – from a shortlist of usually three or four – into an open primary meeting.

It’s a far less radical development than all-postal primaries. Even so, it has more than symbolically opened up a previously private process, has got Ed Miliband wondering if Labour should be doing something similar, and will surely strengthen the successful candidates’ mandate, if they’re eventually elected. As for the profile of those successful primary-selected candidates, my guess is that David Cameron is at least moderately encouraged. By my count, there have been fewer women than men, though a higher proportion than the overall one-third so far selected in the party’s target seats. Our interest, though, is in one in particular: Lucy Frazer QC, a commercial law barrister who, as re-confirmed candidate for South East Cambridgeshire, has already acquired a small, unwanted footnote in British electoral history.

Let me summarise. Safe Conservative seat; 100+ applicants, reduced by the local party executive to a shortlist of three women, one man – none, controversially, with serious local connections. In the first two rounds of voting, no candidate gets 50%, and the bottom candidate in each round is eliminated – leaving Frazer and Heidi Allen, a businesswoman and St Albans councillor. No voting numbers are revealed, even to candidates, who are not permitted scrutineers. Third round: Frazer is declared winner, by 84 votes to 48 – somewhat surprisingly, Allen having been ante-post favourite.

The presiding officer then, quite improperly, takes the ballot papers home, decides to recount them, and discovers a serious case of incompetence and/or malpractice. In the final vote, a pile of 25 ballot papers was marked as being for Frazer, although apparently only the top two actually were, the rest being for Allen.  The true result, therefore, should have been 84 less 23 = 61 for Frazer; 48 plus 23 = 71 and a majority of 10 for Allen.

Thus far, real life mirrors almost precisely a key plot device in Jeffrey Archer’s Best Kept Secret. In Archer’s version, Sir Giles Barrington narrowly retains his Bristol Docklands seat for Labour at the 1955 General Election, thanks to his sharp-eyed young nephew, Sebastian Clifton, having spotted that one of the piles of 100 ballot papers allocated to his Conservative opponent, Major Alexander Fisher, “has a Fisher ballot paper on top, and the 99 underneath are for Uncle Giles” (p.256).

At this point, however, fiction and fact diverge. Archer arranges for the mis-allocation to be discovered and confirmed during the recount already requested by Sir Giles [forget all the improbabilities – it’s a story, for goodness’ sake!]. The correct result can therefore be officially declared: a victory by 4 votes for Sir Giles, instead of the previous announced 184 for Major Fisher.

Heidi Allen was less fortunate. Not having been told her first and second round votes, she was unaware when the result was announced that she had apparently, and inexplicably, ‘lost’ more than a dozen of her supporters between the second and third counts. Others in the room, however, must have known, which seems to suggest an element of malpractice amidst all the incompetence – reinforced in my mind anyway by the fact that, as the world knows, the Archers’ home at The Old Vicarage, Grantchester is in – yes, South East Cambridgeshire.

Conservative Campaign HQ, confronted with this total car crash, reacted pleasingly predictably – first claiming that the ballot papers should have been shredded (which they shouldn’t – not for three months), then walking away from the whole thing. The South East Cambridgeshire Association then compounded their self-inflicted fiasco by refusing to contemplate a recount, calling instead an emergency closed meeting to which they invited Frazer but not Allen, and voted to reaffirm Frazer’s election in the interests of ‘party unity’ (some hope!).

As an outsider, I’m both amused and, because of the Archer coincidence, mildly intrigued. There is, though, a serious point. Open primaries are one of the more interesting electoral initiatives in recent years, and one hopes that other local parties aren’t deterred by the disaster of this single case.

Chris is a Visiting Lecturer at INLOGOV interested in the politics of local government; local elections, electoral reform and other electoral behaviour; party politics; political  leadership and management; member-officer relations; central-local relations; use of consumer and opinion research in local government; the modernisation agenda and the implementation of executive local government.