The Guardian, July 2003

David Blunkett has not been getting on too well with his chief constables. Last autumn, for example, the Home Secretary unveiled his brand new National Policing Plan, which is to guide the 43 constabularies of England and Wales in all their efforts to deal with crime and disorder.

For many months, in private, the most senior officers in the land told Mr Blunkett their opinion of this central manifesto of the government's attack on crime. One chief constable captured for us in a single vivid sentence the message which his colleagues conveyed: "Frankly, sir, with respect, this is crap."

This friction is rather more than an argument among friends. It is a profound clash of ideas about how to generate success from the structural failure of the criminal justice system, which we described yesterday - a system which succeeds in bringing to justice only 3% of the offences that are committed around it, which then frequently fails to enforce its sentences against those whom it does catch and which routinely fails to prevent them offending again.

Faced with the limits of conventional law enforcement, this Home Secretary has inherited his predecessor, Jack Straw's, attempt to work outside the system with some radical alternatives, which we will examine tomorrow. But Mr Blunkett also has applied the most intense pressure to the existing system in an effort to force it to deliver better results. And it is here - in what Mr Blunkett likes to call 'narrowing the justice gap' - that he has found himself at odds with many chief officers who complain bitterly that his law-enforcement agenda, now captured in the new National Policing Plan, is narrow, simplistic and essentially ineffective.

This conflict of ideas is, in turn, a conflict of power. Chief officers around the country worry out loud that they are faced with a Home Secretary who is dealing with their doubts about his policy by trespassing on their operations, crashing through constitutional law in a way which, for the first time in this country, involves government in the political control of the police.

According to the rules, the Home Secretary is allowed to establish the level of funding and the strategic priorities for police, but that is all; it is for the local police authorities to decide how the funds are spent; and it is for the chief constables alone to decide how officers are deployed. This awkward balance is rooted in historic fear - that none of these three could be trusted with the power which police have to take our liberty and to invade our privacy. Now the balance is tipped towards the centre. As one chief officer put it: "We have always been answerable to the law - and only to the law. Suddenly, we are answerable to a politician."

This fretwork of tension has provoked several unseen crises. One of the earliest and most important involved the chief constable of Avon and Somerset. Steve Pilkington is one of a new generation of highly-educated chief officers who are willing to tackle crime and disorder with a far more sophisticated approach than their predecessors. Believing that there is more to policing than simply arresting and convicting offenders, Pilkington has championed 'geographic policing', using beat managers to build links with communities so that finally and genuinely, his officers can claim to be policing by consent; and he has trained all his officers in problem-solving so that they can try to cut crime before it reaches the criminal justice system, by working with other agencies to unravel the problems that lie behind the offence.

Pilkington's philosophy was tested to the limit late one night last summer when two men opened fire on a couple of his detectives, who were unarmed. The men with the guns ran, the detectives ran after them, radioing for help. An armed response vehicle caught up with them. A minute or two later, one of the gunmen was dead on the pavement. A police bullet had hit him in the back.

This was nightmare time for Avon and Somerset police. All this had happened in St Paul's, an impoverished mixed-race community in the centre of Bristol which is beset with crime problems. The man with the bullet in his back was black. St Paul's notoriously rioted in the early 1980s. As a young inspector in London, Pilkington himself had seen Brixton burn at the same time (he'd been caught in a van which was petrol-bombed). Those past riots had been triggered by heavy-handed policing. Indeed, one of the Brixton riots was caused directly by a police shooting.

Using the tools of conventional law enforcement, a chief constable would have reacted to the threat by flooding the area with officers, including riot squads, trying to use force and the threat of arrest to hold down the lid. Pilkington's officers did something very different. They had spent years trying to persuade this community to trust them, so now they put their trust back in them and simply gave them the facts. They set up a team of local leaders and a telephone tree to link up with their contacts in St Paul's and, through their beat managers, to people on the street. Each day, as they found out more about the shooting, they fed out more information: the two men were not local but Jamaican; they were carrying a loaded gun; they had fired on unarmed officers; witnesses had heard the armed response unit shout a warning; the men had ignored it and fired back. They held public meetings. Some officers left their private mobile phone numbers so that people could call them to check out any rogue rumour. Pilkington himself went down to St Paul's to talk to people. And... there was no riot, no disorder. Nothing.

In old-fashioned terms of arrest and conviction, this was a non-event, and yet it was a policing triumph. It was not simply that the facts were on the side of the police action, but that this community, which could so easily have rejected those facts, accepted them and accepted that the police were not strong-arm enemies - because Avon and Somerset had spent years building a relationship with the people they were policing. And the important point is that while Pilkington's philosophy was rescuing Bristol from riot, just about every chief constable in the country was watching the Home Secretary apparently attempting to sack him.

Within days of taking office, David Blunkett had made his mark by elbowing the chief constable of Sussex, Paul Whitehouse, out of his job, sending a clear signal to every chief officer in the country that, regardless of the legal nicety that it was up to the local police authority to hire and fire the chief constable, they were vulnerable to his will. Behind the scenes, he had taken two steps further to curb their power by creating a new regime of centralised control.

Clearly borrowing from his previous job as Education Secretary, Mr Blunkett transferred the role of the former Ofsted chief, Chris Woodhead, to a new role for HM Inspectors of Constabulary. Instead of reviewing the work of forces and giving advice to the Home Secretary, as they had done for years, the inspectors would now inspect individual Basic Command Units (the new jargon for police divisions). This was a slap in the face for chief constables who were by-passed by the process, but the punch on the nose came from Mr Blunkett's second new step - the creation in the Home Office of a Police Standards Unit which could be called in to any BCU which was deemed to be failing. This was the most direct threat to chief constables, usurping their authority over their own officers, cutting straight through their own policing plans and effectively requiring their resignation.

In July 2001, a team from HM Inspectorate of Constabulary, working to this new agenda, arrived in Bristol to review the work of the Central District BCU, which is responsible for the commercial heart of the city as well as the university, the pubs and clubs in the new '24-hour city' and a cluster of deeply deprived wards, including St Paul's. Before they even set foot in Bristol, the inspectors reached for the weapon which is now at the heart of the struggle between the Home Secretary and his chief constables: they asked for the numbers.

Chief constables, like the chief executives of every other branch of the public sector, are now surrounded by numerical performance indicators. In negotiations with the Home Office, they have expressed their deep and often bitter opposition to this. They have warned ministers that the numbers do not measure the reality of their work (the non-riot in Bristol is the exactly the sort of example they quote). Beyond that, they have complained that the numbers actively distort their operations, pushing them away from real local needs in order to hit national targets and giving them a perverse incentive to ignore serious criminals in favour of "hunting down the young and stupid, the impulsive, the mentally frail and the absolutely desperate", as one chief constable put it to us.

One of the most troubling examples has been the government's recent decision to require all police forces from April of this year (2003) to deliver an annual increase of at least 5%in the number of persistent offenders who are brought to justice. Apart from some strong reservations about the impact of this kind of conventional law-enforcement, the chief constables were prepared to go along with the idea: their intelligence networks identify the people who commit most of the crime on their patches and they want to focus on them. But the Home Office, attempting to satisfy the Treasury's demand for numerically measurable results, converted the initiative into a bureaucratic obstacle. They insisted that persistent offenders must be defined as those who had been convicted of at least six offences in the previous 12 months.

The chief constables angrily complained that that was a definition not of a persistent offender but of a stupid one; that it diverted their efforts from the clever offender who had evaded conviction and also from the imprisoned offender who was about to be released without any recent convictions simply because he had been locked up; that it invited them to go after the school girl who had been caught shoplifting six times but not the habitual burglar who had been caught only twice. Even worse, we have picked up reports from around the country of forces who early this year deliberately gave a holiday to their persistent offenders, choosing not to arrest them until April 1 when the counting of the new target officially started.

When HM inspectors descended on the central Bristol BCU, the National Police Plan was still in the pipeline, but most of its numbers were already pouring down from Whitehall, which had ignored almost all of the chief officers' warnings. These numbers dominate police strategy and permeate every corner of their decision making, but the numbers which matter most - the ones which Mr Blunkett was already waving in front of the press, the ones which Home Office ministers inserted into the new role of HM inspectors despite the opposition of chief constables - are the levels of burglary, car crime and robbery. Those were the numbers which the inspectors asked for before they even arrived in Bristol. This was bad news for Steve Pilkington and the central Bristol officers.

For several years, their own approach had been cutting the local crime numbers but in early 2001, they had suddenly leaped upwards - by 27.3% in only three months. In part, this was just numbers, nothing to do with reality: Avon and Somerset had adopted new rules for counting crime, throwing out years of dubious practice and simply recording every allegation from a member of the public as an offence which needed to be cleared up. Pilkington had opted to do this before most of the country and he had made it very clear to all his officers that he wanted this done ethically. In a deliberate break from the national police history of fiddling figures, he insisted that Avon and Somerset crime statistics must tell the truth. That alone had sent the numbers jumping upwards by some 10%.

In addition, central Bristol had been invaded by Yardie gangsters who had arrived at the end of 2000 and embarked on a campaign of casual violence which had seen opponents abducted, stabbed or shot (sometimes as a reprisal for some gang dispute back in Kingston) and who had also succeeded in boosting the market in crack cocaine, whose heavily-addicted buyers were now frantically burgling and robbing and breaking into cars to pay their Yardie suppliers. Pilkington's officers were fighting back, but they had a real struggle on their hands.

That year of the Yardies' arrival, 2000/01, saw a 72% increase in serious crime in central Bristol - a 52% increase in grievous bodily harm, a 100% increase in kidnaps, a 128% increase in abductions, a 75% increase in rape. The government's most favoured targets were doing just as badly - robbery, burglary, theft of cars and from cars were all substantially higher. The impact had spread across the whole of Avon and Somerset.

The chief constables had warned the Home Office that random local events - often much smaller than a Yardie invasion - could have a dramatic effect on crime figures. One problem family could move into an estate and send BCU burglary statistics through the roof. One football riot could distort the whole BCU picture on crimes of violence. Random events could cut the numbers too: robbery statistics in the BCU in Somerset East, for example, plummetted when the Glastonbury festival was cancelled by the foot and mouth epidemic.

The Association of Chief Police Officers had been particularly scathing about the Home Office plan to group BCUs together in 'families' which were supposed to be similar in terms of miles of motorway or rates of unemployment, but which were often quite different in terms of crime. ACPO even commissioned research from the Police Foundation think tank, who ran a binary logistic regression and found that fewer than 5% of the 318 BCUs in England and Wales had low clear-up rates for burglary or robbery which could not be explained by local factors.

All these underlying difficutlies now came to the surface in Bristol. Months before HM inspectors arrived, Steve Pilkington and his senior officers agreed that the only way to tackle the sudden crime boom in central Bristol was to tackle its underlying cause - the Yardies. If they could do that, they could kill off the motive that was driving something like 70% of property crime in the city. That meant more than just arresting and processing Yardies. They would work with the immigration department to stop the Yardies wandering through the airports where they were sometimes spotted and stopped only to be released a few hours later because there were no holding cells. They would negotiate with customs too to work with them as the Yardies' mules arrived in the country: there had been examples of police in various parts of the country tipping off customs with specific details of mules flying in only to find that customs took no action because their own performance targets diverted them into the interception of much larger drug consignments. They would also carry on working with the community, so that they accepted their work without seeing it as a threat.

But this was Avon and Somerset's approach - not the Home Office's. With this strategy, there would be no spare officers to run any kind of campaign for Mr Blunkett's car crime target. If the strategy worked, it would work slowly, waiting for immigration and customs to play their parts, potentially failing to yield the burglary figures as quickly as the Home Secretary wanted. Fundamentally, the operation against the Yardies was part of a wider effort to work outside the frame of conventional law enforcement by taking a 'problem-solving approach', recognising that most of their persistent offenders were drug addicts and/or illiterate and/or homeless and/or mentally ill and/or jobless and that there was simply no benefit to the community in arresting them yet again and sending them off on a tragic roundabout of ineffective consequences.

As a related example of problem-solving, the local commander at the Central Bristol BCU, Chief Superintendent Mike Roe, wanted to extend the network of beat managers into local schools, not to arrest children but to spot those who were getting into trouble and to work with them and their families before they slipped all the way into crime. But Roe could not do this on the scale he wanted while his officers were being diverted by Mr Blunkett's targets. And, even if he found the officers, the impact would not show up in the crime numbers for years. Mr Blunkett wanted his results before the next election.

When HM inspectors produced their draft report, in September 2001, they scarcely noticed the force's problem solving. The inspectors' report, which we have seen, made no allowance for the new counting rules or the Yardies. And they made a fetish of the numbers. They said Central Bristol had the worst numbers per thousand of population in its family of 17 BCUs - even though they had failed to count more than 100,000 commuters, nightlifers and students who came into the area, all of them potential crime victims and offenders, but none of them on the electoral roll. The inspectors went on to say that Central Bristol was underperforming compared to the other 17 BCUs in its family, even though none of the others contained a city centre (15 of them were suburbs, 2 of them provincial towns) with all that that involved for late-night drinking and crime; and none of them had been invaded by several hundred armed Jamaican gangsters with a ready supply of crack cocaine.

As part of their attempt to police in the way that their community wanted, the BCU had set up special units to deal with domestic violence, sex offences and racist crime. Officers were enormously proud of their success, but in the world of New Labour numbers, their success looked like failure, because the victims of these crimes were now trusting the police to deal with them so they were coming forward and reporting more offences. Women who are battered by their partners, for example, typically suffer 35 assaults before they approach the police. Now, Bristol women were coming forward much quicker. It was the same with race crime and sex offences. But HM inspectors complained that their numbers for crimes of sex and violence were increasing and stood well above the other 17 BCUs.

Faced with the impact of the Yardies, central Bristol officers were working long hours, but instead of reflecting on the difficulty of dealing with a huge increase in crime without an increase in resources, the inspectors simply asserted that manpower should be better managed. The inspectors produced eleven coloured graphs and two statistical tables, they worried about the absence of 'robust audit trails linking targets and objectives through to operations and their outputs' and they went on to make seven recommendations, dealing with internal structures, the cycle of meetings and the monitoring of performance.

For police officers on the ground and for residents of central Bristol, it was a report which had no link to the real world. What good were meetings and monitoring in the face of a Yardie crime wave? (And, indeed, how could the Home Office attack the police when it was their immigration service which was failing to stop the Yardies and the mules pouring through the airports at will, carrying an estimated 4.5 tonnes of cocaine a year?) How could they possibly enjoy the consent of the people they were policing if they started dancing to a political tune? Avon and Somerset officers wanted the report re-written, but HM inspectors stuck to their new agenda and insisted on publishing it without any significant change.

Now, chief constables around the country were watching closely. Their concern was not just that Pilkington was in danger but that all of them were in danger from a Home Secretary who was apparently willing to upset the delicate balance of power over their work. They had seen the danger dawning before Blunkett took over the Home Office. Some of them had been deeply opposed to Jack Straw's idea of a new national police radio system, Airwave - the biggest capital investment ever made by the police in this country. They said it was unnecessary and grossly overpriced but when they took their complaints to the supposedly independent Police Information and Technology Organisation, who had to make the decision, they found that Straw's officials had gatecrashed and were insisting that the scheme must be adopted - because it was the biggest PFI scheme in the government's gift and, regardless of benefit, regardless of cost, it must go ahead. And it did, in 2000. (The deal has subsequently been mauled by the Public Accounts Committee as 'disastrous', by the National Audit Office for its 'lack of competition', and by some experts in pulsed frequencies as a health risk comparable to asbestos.)

In the same way, as David Blunkett was closing in on Steve Pilkington, his officials were intervening in the supposedly independent Police Negotiating Board, some of whose members report that, while they were attempting to strike a new deal for police pay in the normal way, the Home Secretary was dictating terms and using leaks to the press to undermine the board's position. "It was outrageous," according to one board member. "There was one Sunday morning when we were pursuing our position with the staff organisations - and there in the Sunday papers was the Home Secretary putting completely different points. We are supposed to be independent." One result was that the Home Secretary forced a national cut in police overtime, even though that was the fuel which police were using to hit his targets.

But what worried them most was that the Home Secretary was top-slicing their money: instead of handing cash down to the 43 forces, he was holding more and more of it in the centre, inviting bids on which he and his ministers would make decisions. This stole power from the police authorities as well as the chief constables. And some of these central decisions were unwelcome. For example, David Blunkett wanted forces to hire Community Safety Officers as a cost-effective way of putting uniformed patrols on the streets. This was popular with some of the big city forces who could provide rapid back-up to the CSOs, but it was despised by most chief constables who believed uniformed patrols were a politically popular waste of time and who feared that untrained CSOs could easily cause friction. And yet they needed the money, so they bid for the Home Office's pot of CSO cash. Kent, for example, bid for 18 CSOs even though their chief constable, Sir David Phillips, as president of ACPO, had said publicly that he thought they had 'very little merit'.

It was not just that specific decisions about deployment were being stolen by the Home Office. The top-sliced funds tipped the whole balance of power. Crucially, David Blunkett had made a great public splash about increasing the number of police officers, but all of the money for these new recruits was held centrally and since every chief constable was desperate for more officers, all of them were vulnerable to Home Office pressure. (Not one chief constable would talk to us on the record about the Home Office for fear that their force would suffer. Even retired chief constables were worried. Almost all of them used the same word to describe the Home Office's attitude to them - 'spiteful'.) It was in this context of centralised power that the numerical targets became such a powerful tool.

There was now a real risk that Avon and Somerset were about to have one of their BCUs taken over by the new Police Standards Unit, that Steve Pilkington would be forced out of his job, that the whole community-based, problem-solving approach to the policing of Avon and Somerset would be crushed. Avon and Somerset decided to play the Home Office at their own game: they would accept all the recommendations of HM inspectors; they would rearrange their meetings and their monitoring and their audit trails; and they would let events prove the point for them. They warned Home Office ministers directly that unless they admitted that there was a national crime problem with Yardies and crack cocaine, not only would HM inspectors fail to help central Bristol but the government nationally would fail to reach its own crime targets.

The following summer, July 2002, HM inspectors published a draft follow-up report on the Central Bristol BCU, in which they patted themselves on the back, crediting their first report as 'a catalyst for change', and noted that the BCU had done all that had been asked. This time, they acknowledged that the BCU's similarities to the others in its family were 'somewhat limited' but neverthless produced eight different graphs to make comparisons. This time, they acknowledged that new counting rules had had a 'serious impact' on their figures but nevertheless used the figures to conclude that this was 'what can only be described as a comparatively poor crime performance'. This time, they noted 'the influx of drugs into the Bristol area, mainly through couriers from Jamaica' and recorded the BCU's strong view that they could not do much unless the government did something to stop the Yardies entering the country. But nevertheless they went ahead and delivered the judgement which most worried chief constables around the country: the inspectors suggested that the Police Standards Unit should be called in to the BCU.

At this point, David Blunkett was finally confronted by one of the many significant differences between education and criminal justice: the police are organised. The teacher unions were divided and swept aside by Blunkett. The police groups are much tougher. The Association of Chief Police Officers, in particular, runs a 100% closed shop, it has powerful political allies in Fleet Street and Westminster, and more than any other group of organised labour in this country, it has succeeded in infiltrating its employers by inserting its various sub-committees as national policy-making bodies. ACPO covers 43 forces, each of which has its own structure and philosophy, some of which have no time at all for problem-solving and geographic policing, but all of whom were threatened by what was happening to Steve Pilkington.

Within days of the inspectors' follow-up report, the Police Standards Unit were in touch with the local police authority in Avon and Somerset to inform them that they would be making 'a formal intervention'. Members of the police authority found themselves being phoned by MPs who were hostile to Pilkington (several chief constables around the country report being 'bullied into line' by local Labour MPs, apparently acting at the behest of the Home Office). Some members of the police authority were upset that they were being over-ridden: if the chief constable really was at fault, then it was for them to act, not the Home Office.

Their chairman, John Christensen, insisted on meeting the head of the standards unit, former police superintendent and businessman Kevin Bond. Christensen, knowing that this meeting could set a national precedent, took with him the head of the national Association of Police Authorities, Melanie Leech. Bond listened to them and undertook to write to Christensen to spell out what was happening.

In the meantime, the then president of ACPO, Sir David Phillips, travelled from Kent to Bristol to find out what was happening. According to ACPO sources, he left the city appalled at the naivety of HM inspectors and determined to stop the standards unit moving in. The ACPO machine started to mobilise. According to one source, they confronted senior Home Office officials with a passionate complaint about the Home Office's theft of power from chief constables and police authorities and with the specific prospect that if Pilkington was ousted, he would be entitled to sue them, thus making a public spectacle of the power struggle.

Behind the scenes, Sir David Phillips is believed to have visited HM inspectors at the Home Office, where he warned that they were on the brink of a move which could be deeply counter productive. Some in HMI were not keen to encourage the standards unit, which was trespassing on its patch. Sir David is believed to have pointed out that the standards unit could not intervene unless HM inspectors formally recommended it. Although they had written their report, the inspectors had not yet made the formal move. And they never did. At the standards unit, Kevin Bond never wrote his promised letter to the Avon and Somerset police authority. HMI's follow-up report was stamped 'confidential' and, unlike the orginal report - and in breach of HMI's stated policy - it was never published. Steve Pilkington, who had been hanging over the edge of a precipice, was hauled back to safety.

As a compromise, the standards unit agreed to 'collaborate' with the central Bristol BCU, offering advice and even helping them to access new money, but not formally intervening in their strategy. Since then, the Home Office's own researchers have confimed what central Bristol was trying to tell them, that there is a national surge in the black market for crack cocaine, which is jeopardising the whole of the government's crime strategy. The Home Office, in December last year, belatedly produced a national crack plan.

In central Bristol now, the crime figures are falling again. Sticking to their problem-solving approach, Avon and Somerset sent officers over to Jamaica and brought Jamaican police to Bristol to spot known offenders. They intercepted £10 million of Yardie cash leaving the country; closed down the bogus college which was giving cover to Yardie mules entering the country as students; forged links with immigration and customs; and saw the Home Office belatedly bring in visas which may make travel more difficult for the Yardies. They also worked so closely with the community that they were able to put armed officers onto the streets without losing their support. The result of their work is reflected now in a newsletter version of their annual report showing sharp falls in the number of crime victims in the BCU, under the headline "Partnerships - the key to cracking crime."

Nobody at Avon and Somerset wants to dig up the past. The head of central Bristol BCU, Chief Supt Mike Roe, stresses the good side of what happened: "It is true that we were disappointed with the HMI report, that they did not see our crime in its context and did not make the link to the Jamaican organised crime groups. But at the end of it all, you could argue that HMI did help us to do our job better and the standards unit also had a positive role to play. They did support us, they also got us extra funds. "

In the meantime, the performance regime has been tightened still further. It has been recast into a new Policing Performance Assessment Framework which sets yet more targets under five new domaine headings (it was six, but they cancelled 'Helping The Public'), all still grouped into 'families' of forces so that their average performance can be compared. The government has just created a new national network of criminal justice boards, pulling together the chief officers of every criminal justice agency in every region in order to ensure that all of them meet their numerical targets.

And this year, at the behest of the Treasury, Activity Based Costing is to be introduced across the country, requiring that each year, for a three-week period, every police officer should pause every 15 minutes in order to fill in a card to summarise the work they have been doing so that Whitehall can monitor efficiency. In diplomatic words, HM Inspector of Constabulary, Sir Keith Povey, noted in a recent report: "This can be regarded as absurd".

And this whole regime is enforced with the same engulfing bureaucracy which we described earlier in this series with the Drug Action Teams. Police officers meet as a senior management team and work their way through service level agreements and tendering protocols, producing business plans with audit trails and budget codes and business score cards to monitor the result. They bid for funds under dozens of different headings, wait months for approval and then scramble to spend the cash before the financial year ends.

There are very few police officers now who do not expect to be held to account for their work. And they understand the importance of numbers - as a clue to crime patterns, as an intelligence tool to steer their work, not as a concrete overcoat. Earlier this year, the chief constable of Thames Valley, Peter Neyroud, had the courage to announce publicly that he was setting aside Mr Blunkett's targets for burglary and robbery. He said they "fly in the face of reality" and threatened his ability to protect his community from other crimes. Other chief officers in mute rebellion have quietly adopted their own targets.

Down on the ground there are people who know a lot about crime but who are running a criminal justice system which does not deliver results. In the centre there is a government which is desperate for results and therefore takes over from the people on the ground even though - and here's the point - they know just about nothing at all about crime and even less about the different needs of different communities. Pounding a system to deliver results which are beyond its reach, the horrible outcome is that the Home Office are actively making the fight against crime more difficult. The symbol of David Blunkett's police service is a blue light wrapped in red tape. "Frankly, sir, with respect, this is crap."

Additional research by Tamsen Courtenay