What's wrong with the war against drugs

The Guardian, February 2001

On April 3 1924, a group of American congressmen held an official hearing to consider the future of heroin. They took sworn evidence from experts, including the US Surgeon General, Rupert Blue, who appeared in person to tell their committee that heroin was poisonous and caused insanity and that it was particularly likely to kill since its toxic dose was only slightly greater than its therapeutic dose.

They heard, too, from specialist doctors, like Alexander Lambert of New York's Bellevue Hospital, who explained that "the herd instinct is obliterated by heroin, and the herd instincts are the ones which control the moral sense... Heroin makes much quicker the muscular reaction and therefore is used by criminals to inflate them, because they are not only more daring, but their muscular reflexes are quicker." Senior police, a prison governor and health officials all added their voices. Dr S Dana Hubbard, of the New York City health department, captured the heart of the evidence: "Heroin addicts spring from sin and crime... Society in general must protect itself from the influence of evil, and there is no greater peril than heroin."

The congressmen had heard much of this before and now they acted decisively. They resolved to stop the manufacture and use of heroin for any purpose in the United States and to launch a worldwide campaign of prohibition to try to prevent its manufacture or use anywhere on the planet. Within two months, their proposal had been passed into law with the unanimous backing of both houses of the US Congress. The War Against Drugs was born.

To understand this war and to understand the problems of heroin in particular, you need to grasp one core fact. In the words of Professor Arnold Trebach, the veteran specialist in the study of illicit drugs: "Virtually every 'fact' testified to under oath by the medical and criminological experts in 1924... was unsupported by any sound evidence." Indeed, nearly all of it is now directly and entirely contradicted by plentiful research from all over the world. The first casualty of this war was truth and yet, 77 years later, it still goes on, more vigorous than ever, arguably the longest-running conflict on the planet.

Drugs and fear go hand in hand. The war against drugs is frightening - but not, in reality, for the reasons which are claimed by its generals. The untold truth about this war, which has now sucked in every country in the developed world, is that it creates the very problem which it claims to solve. The entire strategy is a hoax with the same effect as an air force which bombs its own cities instead of its enemy's. You have to go back to the trenches of Flanders to find generals who have been so incompetent, so dishonest, so awesomely destructive towards those for whom they claim to care.

The core point is that the death and sickness and moral collapse which are associated with Class A drugs are, in truth, generally the result not of the drugs themselves but of the blackmarket on which they are sold as a result of our strategy of prohibition. In comparison, the drugs themselves are safe, and we could turn around the epidemic of illness and death and crime if only we legalised them. However, it is a contemporary heresy to say this, and so the overwhelming evidence of this war's self-destructive futility is exiled from almost all public debate, now just as it was when those congressmen met.

Take heroin as a single example. And it's a tough example. In medical terms, it is simply an opiate, technically known as diamorphine, which metabolises into morphine once it enters its user's body. But, in terms of the war against drugs, it is the most frightening of all enemies. Remember all that those congressmen were told about 'the great peril'. Remember the Thatcher government's multi-million pound campaign under the slogan 'Heroin Screws You Up'. Think of Tony Blair at the 1999 Labour Party fulminating about the 'drug menace' or of William Hague last year calling for 'a stronger, firmer, harder attack on drugs than we have ever seen before'. And now look at the evidence.

Start with the allegation that heroin damages the minds and bodies of those who use it, and consider the biggest study of opiate use ever conducted, on 861 patients at Philadelphia General Hospital in the 1920s. It concluded that they suffered no physical harm of any kind. Their weight, skin condition and dental health were all unaffected. 'There is no evidence of change in the circulatory, hepatic, renal or endocrine functions. When it is considered that some of these subjects had been addicted for at least five years, some of them for as long as twenty years, these negative observations are highly significant.'

Check with Martindale, the standard medical reference book, which records that heroin is used for the control of severe pain in children and adults, including the frail, the elderly and women in labour. It is even injected into premature babies who are recovering from operations. Martindale records no sign of these patients being damaged or morally degraded or becoming criminally deviant or simply insane. It records instead that, so far as harm is concerned, there can be problems with nausea and constipation.

Or go back to the history of 'therapeutic addicts' who became addicted to morphine after operations and who were given a clean supply for as long as their addiction lasted. Enid Bagnold, for example, who wrote the delightful children's novel, National Velvet, was what our politicians now would call 'a junkie', who was prescribed morphine after a hip operation and then spent twelve years injecting up to 350 mgs a day. Enid never - as far as history records - mugged a single person or lost her 'herd instinct', but died quietly in bed at the age of 91. Opiate addiction was once so common among soldiers in Europe and the United States who had undergone battlefield surgery that it was known as 'the soldiers' disease'. They spent years on a legal supply of the drug - and it did them no damage.

We cannot find any medical research from any source which will support the international governmental contention that heroin harms the body or mind of its users. Nor can we find any trace of our government or the American government or any other ever presenting or referring to any credible version of any such research. On the contrary, all of the available research agrees that, so far as harm is concerned, heroin is likely to cause some nausea and possibly severe constipation and that is all. In the words of a 1965 New York study by Dr Richard Brotman: "Medical knowledge has long since laid to rest the myth that opiates observably harm the body." Peanut butter, cream and sugar, for example, are all far more likely to damage the health of their users.

Now, move on to the allegation that heroin kills its users. The evidence is clear:
you can fatally overdose on heroin. But the evidence is equally clear, that - contrary to the claims of politicians - it is not particularly easy to do so. Opiates tend to suppress breathing, and doctors who prescribe them for pain relief take advantage of this to help patients with lung problems. But the surprising truth is that, in order to use opiates to suppress breathing to the point of death, you have to exceed the normal dose to an extreme degree. Heroin is ununusally safe, because - contrary to what those US congressmen were told in 1924 - the gap between a therapeutic dose and a fatal dose is unusually wide.

Listen, for example, to Dr Teresa Tate, who has prescribed heroin and morphine for 25 years, first as a cancer doctor and now as medical adviser to Marie Curie Cancer Care. We asked her to compare heroin with paracetomol, legally available without prescription. She told us: "I think that most doctors would tell you that paracetamol is actually quite a dangerous drug when used in overdose, it has a fixed upper limit for its total dose in 24 hours and if you exceed that, perhaps doubling it, you can certainly put yourself at great risk of liver failure and of death, whereas with diamorphine, should you double the dose that you normally were taking, I think the consequence would be to be sleepy for a while and quite possibly not much more than that and certainly no permanent damage as a result." Contrary to the loudly expressed view of so many politicians, this specialist of 25 years experience told us that when heroin is properly used by doctors, it is "a very safe drug".

Until the American prohibitionists closed him down in the 1920s, Dr Willis Butler ran a famous clinic in Shreveport, Louisiana for old soldiers and others who had become addicted to morphine after operations. Among his patients, he included four doctors, two church ministers, two retired judges, an attorney, an architect, a newspaper editor, a musician from the symphony orchestra, a printer, two glass blowers and the mother of the commissioner of police. None of them showed any ill effect from the years which they spent on Dr Butler's morphine. None of them died as a result of his prescriptions. And, as Dr Butler later recalled: "I never found one we could give an overdose to, even if we had wanted to. I saw one man take 12 grains intravenously at one time. He stood up and said, 'There, that's just fine,' and went on about his business."

Heroin can be highly addictive - which is a very good reason not to start taking it. In extreme doses, it can kill. But the truth which has been trampled under the cavalry of the drug warriors is that, properly prescribed, pure heroin is a benign drug. The late Professor Norman Zinberg, who for years led the study of drug addiction at Harvard Medical School, saw the lies beneath the rhetoric: "To buttress our current program, official agencies, led originally by the old Federal Bureau of Narcotics, have constructed myth after myth. When pushers in schoolyards, 'drug progression', drugs turning brains to jelly, and other tales of horror are not supported by facts, they postulate and publicize others: 'drugs affect chromosomes'; 'drugs are a contagious disease'. Officials go on manufacturing myths such as the chromosome scare long after they are disproved on the self-righteous assumption that 'if they have scared one kid off using drugs, it was worth the lie.'"

Take away the lies and the real danger becomes clear - not the drugs, but the blackmarket which has been created directly by the policy of prohibition. If ever there is a war-crimes trial to punish the generals who have gloried in this slaughter of the innocent, the culprits should be made to carve out in stone: "There is no drug known to man which becomes safer when its production and distribution are handed over to criminals."

Heroin, so benign in the hands of doctors, becomes highly dangerous when it is cut by blackmarket dealers - with paracetomol, drain cleaner, sand, sugar, starch, powdered milk, talcum powder, coffee, brick dust, cement dust, gravy powder, face powder or curry powder. None of these adulterants was ever intended to be injected into human veins. Some of them, like drain cleaner, are simply toxic and poison their users. Others - like sand or brick dust - are carried into tiny capillaries and digital blood vessels where they form clots, cutting off the supply of blood to fingers or toes. Very rapidly, venous gangrene sets in, the tissue starts to die, the fingers or toes go black and then have only one destiny - amputation. Needless suffering - inflicted not by heroin, but by its blackmarket adulterants.

Street buyers cannot afford to waste any heroin - and for that reason, they start to inject it, because smoking or snorting it is inefficient. The Oxford Handbook of Clinical Medicine records that a large proportion of the illness experienced by blackmarket heroin addicts is caused by wound infection, septicaemia, and infective endocarditis, all due to unhygienic injection technique. Street users invariably suffer abscesses, some of them of quite terrifying size, from injecting with infected needles or drugs. Those who inject repeatedly into the same veins or arteries will suffer aneurysms - the walls of the artery will weaken and bulge; sometimes they will start to leak blood under the skin; sometimes, these weakened arteries will become infected by a dirty needle and rupture the skin, leaving the user to bleed to death.

In the mid 1990s, the World Health Organisation estimated that 40% of recent AIDS cases internationally had been caused by drug users sharing injecting equipment. The British record on AIDS is better because in the late 1980s, the government quietly broke with its prohibition philosophy and started to provide clean needles. Nevertheless, by June last year, one thousand blackmarket drug users in this country had died of AIDS which was believed to have been contracted from dirty needles. More needless misery and death.

Far worse, however, is the spread of Hepatitis C, which can kill by causing cirrhosis and sometimes cancer in the liver. The official estimate is that 300,000 people in this country are now infected. Dr Tom Waller, who chairs Action on Hepatitis C says the truth is likely to be much worse. And almost all of these victims are blackmarket drug users who contracted the disease by sharing dirty injecting equipment. Dr Waller says there is now a 'major epidemic', threatening the lives of 'a great many people'. Needlessly.

Street buyers buy blind and so they will overdose accidentally: they have no way of telling how much heroin there is in their deal. Dr Russell Newcombe, senior lecturer in addiction studies at John Moores University in Liverpool, has found the purity of street heroin varying from 20% to 90%. "Users can accidentally take three or four times as much as they are planning to," he says. It is peculiarly ironic that governments set out to protect their people from a drug which they claim is dangerous by denying them any of the safeguards and information which they insist must apply to the consumption of drugs which they know to be harmless. (Compare, for example, the mandatory information on the side of a bottle of Vitamin C tablets with the information available to a blackmarket heroin user.)

Street buyers often run short of supplies - and so they mix their drug with anything else they can get their hands on, particularly alcohol. Heroin may be benign, but if you mix it with a bottle of vodka or a handful of sedatives, your breathing is likely to become extremely depressed. Or it may just stop. In any event, whether it is poisonous adulterants or injected infection; whether it is death by accidental overdose or death by polydrug use: it is the blackmarket which lies at the root of the danger. The healthiest route, >>> Of course, is not to take the drug at all: but for those who are addicted, prohibition inflicts danger and death. Needlessly. Water would become dangerous if it were banned and handed over to a criminal blackmarket.

The same logic applies to drugs which, unlike heroin, are inherently harmful - like alcohol, which is implicated in organic damage (liver) and social problems (violence, dangerous driving). American bootleggers brewed their moonshine with adulterants like methylated spirits, which can cause blindness. (Hence the proliferation of blind blues singers.) And there are documented cases of drinkers during prohibition injecting alcohol, with all of the attendant dangers. (It is instructive to look back on the prohibitionists' efforts to justify their war against alcohol with hugely inflated statements of its danger. In his history of drugs, Emperors of Dreams, Mike Jay records the claims that alcohol was an 'environmental poison' which generated cretinism and several otherwise unrecognised syndromes including 'blastopthoric degeneration' and ‘alcoholic diathesis'.)

The risks of consuming LSD and Ecstacy are increased enormously by their illegal and unsupervised manufacture. Nobody knows what they are swallowing. Yet, when a Brighton company developed a test to check the purity of Ecstacy, the government's drugs advisor, Keith Hellawell, condemned it and warned that the company risked prosecution. It is the same with blackmarket amphetamines: speed alone may not kill, but speed with a blindfold is highly likely to finish you off.

In the same way, the classic signs of social exclusion among addicts are the product not of their drug but of the illegality of the drug. If addicts fail to work, it is not because heroin has made them workshy, but because they spend every waking minute of the day hustling. If addicts break the law, it is not because the drug has corrupted their morality, but because they are forced to steal to pay black market prices. If addicts are thin, it is not because the drug has stripped away their flesh, but because they spend every last cent on their habit and have nothing left for food. Over and over again, it is the blackmarket, which has been created by the politicians, which does the damage.

The man to whom the government turns for advice on drugs, Keith Hellawell, appears to know none of this. When we interveiwed him for Channel Four, he insisted that heroin itself was dangerous and then repeatedly dodged requests to come up with any evidence at all to justify his claim. Subsequently, when we offered his department as much time as they would like to find any evidence, they failed to come up with anything at all and passed the question to the department of health, who also failed. It is fair to conclude that the government's drugs adviser literally does not know the first thing about heroin.

The confusion between the effect of the drug and the effect of the blackmarket is riddled not only through government policy but also through government statistics which completely ignore the distinction with the result that teams of researchers study drug policy, use compromised statistics and simply recycle the confusion, thus providing politicians with yet more false fuel for their fire. Home Office figures on drug deaths, for example, are hopelessly compromised. Eighteen months ago, the department of health, which might have been expected to know better, produced new guidelines for doctors dealing with drug users and recorded the following: "Generally there is a greater prevalence of certain illnesses amongst the drug misusing population, including viral hepatitits, bacterial endocarditis, HIV, tuberculosis, septicaemia, pneumonia, deep vein thrombosis, pulmonary emboli, abscesses and dental disease." All of it true of the blackmarket. None of it true of the drug. No attempt to make the distinction.

The blackmarket damages not only drug users but the whole community. Britain looks back at the American prohibition of alcohol in the 1920s and shudders at the stupidity of a policy which generated such a catastrophic crime wave. Yet, in this country now, the prohibition of drugs has generated a crime boom of staggering proportions. Research suggests that in England and Wales, a hard core of blackmarket users is responsible for some £1.5 billion worth of burglary, theft and shoplifting each year - they are stealing £3.5 million worth of property a day. As a single example, Brighton police told us they estimate that 75% of their property crime is committed by blackmarket drug users trying to fund their habit. And yet goverments refuse to be tough on the cause of this crime - their own prohibition policy.

The global version of this damage was put succinctly by Senator Gomez Hurtado, former Colombian ambassador to France and a high court judge, who told a 1993 conference: "Forget about drug deaths and acquisitive crime, about addiction and AIDS. All this pales into insignificance before the prospect facing the liberal societies of the West, like a rabbit in the headlights of an oncoming car. The income of the drug barons is an annual five hundred thousand million dollars, greater than the American defence budget. With this financial muscle they can suborn all the institutions of the state and, if the state resists, with this fortune they can purchase the firepower to outgun it. We are threatened with a return to the Dark Ages of rule by the gang. If the west relishes the yoke of the tyrant and the bully, current drug policies promote that end."

Having attacked and maimed and killed the very people they claimed to be protecting; having inflicted a crime wave on the same communities which they said they were defending; having run up a bill which now costs us some £1.7 billion a year in this country alone: this war's generals might yet have some claim to respect if they were able to show that they had succeeded in their original objective of stopping or, at least, of cutting the supply of prohibited drugs. They cannot.

In December 1999, the chief constable of Cleveland police, Barry Shaw, produced a progress report on the 1971 Misuse of Drugs Act, which marked the final arrival of US drugs prohibition in this country: "There is overwhelming evidence to show that the prohibition-based policy in this country since 1971 has not been effective in controlling the availability or use of proscribed drugs. If there is indeed a war against drugs, it is not being won... Illegal drugs are freely available, their price is dropping and their use is growing. It seems fair to say that violation of the law is endemic, and the problem seems to be getting worse despite our best efforts."

Mr Shaw was able to point to a cascade of evidence to support his view: between 1987 and 1997, there had been a tenfold increase in the seizure of illicit drugs, and yet the supply on the streets was so strong that the price of these drugs had kept dropping; in 1970, only 15% of people had used an illegal drug, but by 1995, 45% had; in 1970, 9,000 people were convicted of a drugs offence but in 1995 94,000 were. The Home Office responded to the chief constable's report with complete silence: they refused even to acknowledge receiving it. Internal reports from the American Drugs Enforcement Agency confirm the chief constable's conclusion. (They say Britain now produces so much cannabis that we actually export it to Holland.)

Prohibition has not merely failed to cut the supply of illicit drugs: it has actively spread drug use. The easiest way for new users to fund their habit is to sell drugs and consume the profit; so they go out and find more new users to sell to; so it is that when one child in the classroom starts using, others soon join in; one user in the street and neighbours soon follow. Blackmarket drug use spreads geometrically. The Health Education Authority in 1995 found that 70% of people aged between eleven and thirty five had been offered drugs at some time. Pushers push. When Britain began to impose prohibition of heroin, in1968, there were fewer then 500 heroin addicts in Britain - a few jazz musicians, some poets, some Soho Chinese. Now, the Home Office says there may be as many as five hundred thousand. This is pyramid selling at its most brilliantly effective.

In private, the Home Office's best defence is that they are so short of reliable intelligence on drugs that nobody can finally prove that the war is lost: they simply don't know how much heroin or cocaine is imported, or many peope are using it. At the Cabinet Office, Keith Hellawell argues that the 30 years since the Misuse of Drugs Act do not really count, because, until he took over, British governments did not have a real strategy. He told us he was supporting new international tactics (which he could not divulge) and was now seeing figures (which he could not give us) to suggest finally they were going to succeed. This recalls earlier declarations that "we have turned the corner on drug addiction" (President Nixon, 1973) or "Heroin availability continues to shrink" (DEA,1978). In the meantime, world heroin production has tripled in the last decade, cocaine production has doubled and, in the Home Secretary's Blackburn constituency, police say drug use in the Asian community has soared by 300% in four years.

But the underlying point is even more worrying: once you understand that the real danger comes from the blackmarket and not from the drug, you can see that even if, with some magic formula, the generals started to cut the supply of these drugs, the result would be disastrous. The price of heroin, for example, would start to rise and, since there is no evidence at all that heroin addicts cut their consumption to fit their wallets, they would have to commit more crime to fund their habits. And if the dealers also responded like good entrepreneurs, they would try to keep their prices down by adding even more pollutants to the heroin, thus increasing the health risks to users.

This government has not begun to consider legalisation. No matter the truth about the danger and the death, no matter the truth about the cause of crime, the position is, as Jack Straw put it to the 1997 Labour conference: "We will not decriminalise, legalise or legitimise the use of drugs". Why? The obvious answer was offered to us by Paul Flynn, Labour backbencher and staunch opponent of prohibition: "It is being fuelled by politicians who are vote gluttons, who believe that there is popularity and votes to be gained by appearing to be tough on drugs."

While Keith Hellawell and other prohibitionists are embarrassed by their screaming lack of success, those who want to legalise can point to clear evidence that providing a clean supply of drugs will help with the physical and mental health of users, will cut crime in the community and drain the life out of the blackmarket.

The Swiss, for example, in 1997 reported on a three-year experiment in which they had prescribed heroin to1,146 addicts in 18 locations. They found: "Individual health and social circumstances improved drastically... The improvements in physical health which occurred during treatment with heroin proved to be stable over the course of one and a half years and in some cases continued to increase (in physical terms, this relates especially to general and nutritional status and injection-related skin diseases)... In the psychiatric area, depressive states in particular continued to regress, as well as anxiety states and delusional disorders... The mortality of untreated patients is markedly higher." They also reported dramatic improvements in the social stability of the addicts, including a steep fall in crime.

There are equally impressive results from similar projects in Holland and Luxemburg and Naples and, also, in Britain. In Liverpool, during the early 1990s, Dr John Marks used a special Home Office licence to prescribe heroin to addicts. Police reported a 96% reduction in acquisitive crime among a group of addict patients. Deaths from locally acquired HIV infection and drug-related overdoses fell to zero. But, under intense pressure from the government, the project was closed down. In its ten years' work, not one of its patients had died. In the first two years after it was closed, forty one died.

There is room for debate about detail. Should we supply legalised drugs through GPs or specialist clinics or pharmacists? Should we continue to supply opiate substitutes, like methadone, as well as heroin? Should the supply be entirely free of charge to guarantee the extinction of the blackmarket? How would we use the hundreds of millions of pounds which would be released by the 'peace dividend'? But, if we have any compassion for our drug users, if we have any intention of tackling the causes of crime, if we have any honesty left in our body politic, there is no longer any room for debate about the principle. Continue the war against drugs? Just say No.

Additional research by Jane Cassidy

See below for quotes on prohibition of drugs and alcohol

"All penalties for drug users should be dropped ... Making drug abuse a crime is useless and even dangerous ... Every year we seize more and more drugs and arrest more and more dealers but at the same time the quantity available in our countries still increases... Police are losing the drug battle worldwide." Raymond Kendall, secretary general of Interpol, January 1994

"The prestige of government has undoubtedly been lowered considerably by the prohibition law. For nothing is more destructive of respect for the government and the law of the land than passing laws which cannot be enforced. It is an open secret that the dangerous increase of crime in this country is closely connected with this."
Albert Einstein "My First Impression of the U.S.A.", 1921

"The current policies are not working. We seize more drugs, we arrest more people, but when you look at the availability of drugs, the use of drugs, the crime committed because of and through people who use drugs, the violence associated with drugs, it's on the increase. It can't be working." Keith Hellawell, Guardian 23 May 1994, three years before he was appointed drugs adviser to the government.

"Our emphasis here is based not only on the growing seriousness of drug-related crimes, but also on the belief that relieving our police and our courts from having to fight losing battles against drugs will enable their energies and facilities to be devoted more fully to combatting other forms of crime. We would thus strike a double blow: reduce crime activity directly, and at the same time increase the efficacy of law enforcement and crime prevention." Milton Friedman "Tyranny of the Status Quo"

When the tyrant has disposed of foreign enemies by conquest or treaty, and there is nothing to fear from them, then he is always stirring up some war or other in order that the people may require a leader. -- Plato

Prohibition is an awful flop.
We like it.
It can't stop what it's meant to stop.
We like it.
It's left a trail of graft and slime
It don't prohibit worth a dime
It's filled our land with vice and crime,
Nevertheless, we're for it.
-- newspaperman Franklin P. Adams, 1931, in the New York World, on the release of the Wickersham Commission report

I am against Prohibition because it has set the cause of temperence back twenty years; because it has substituted an ineffective campaign of force for an effective campaign of education; because it has replaced comparatively uninjurious light wines and beers with the worst kind of hard liquor and bad liquor; because it has increased drinking not only among men but has extended drinking to women and even children. -- William Randolph Hearst, initially a supporter of Prohibition, explaining his change of mind in 1929. From "Drink: A Social History of America" by Andrew Barr (1999), p. 239.

"There is thus general agreement throughout the medical and psychiatric literature that the overall effects of opium, morphine, and heroin on the addict's mind and body under conditions of low price and ready availability are on the whole amazingly bland." Edward M. Brecher, 1972

"The available evidence indicates that heroin, when provided in pure form, is a relatively safe drug. Hence it is primarily the illegal nature of the drug, rather than its pharmacological properties, which leads to the health and social problems associated with its use." Ostini, Bammer, Dance and Goodwin. 'The Ethics of Experimental Heroin Maintenance.' Journal of Medical Ethics, 1993.

"When heroin-dependent persons have been provided with daily maintenance doses under medical supervision, marked physiological deterioration or significant psychological impairment has not been observed. In fact, most of the serious adverse consequences of chronic heroin use are generally related to lifestayle and factors involving needle administration." Cox et al, Toronto Addiction Research Foundation

"Heroin is very addictive but does not in itself cause any serious illnesses, nor does it harm any organs or tissues." Dr Ben Goldacre Dr Ben Goldacre, 'Methadone and Heroin: An Exercise in Medical Scepticism'

"To our surprise we have not been able to locate even one scientific study on the proved harmful effects of addiction. Earlier investigators had apparently assumed that the ill effects were so obvious as not to need scientific verification. " Dr. George H. Stevenson, British Columbia, 1956.

Dr Van den Brink, in charge of Duthc research into prescription of heroin for drug users, preparing report for Dutch Health Minister Borst, press interview: "We can only do what is within our reach. But if we thought that treating heroin addicts with heroin was nonsensical and dangerous, we would not make these recommendations."

"The addict when not deprived of his opium showed no abnormal behavior which distinguished him from a nonaddict." yielded similar findings. Dr. George B. Wallace on two studies at Bellevue Hospital in New York City

"It has not been possible to maintain that addiction to morphine causes marked physical deterioration per se." Dr Harris Isbell, director of the Public Health Service's Addiction Research Center in Lexington, 1958,

"The addict under his normal tolerance of morphine is medically a well man." Dr Walter G. Karr, University of Pennsylvania biochemist,1932

"Given an addict who is receiving (adequate) morphine ... the deviations from normal physiological behavior are minor (and) for the most part within the range of normal variations." Dr. Nathan B. Eddy, after reviewing the world literature on morphine, 1940

"Medical knowledge has long since laid to rest the myth that opiates inevitably and observably harm the body." Drs Richard Brotman, Alan S. Meyer, and Alfred M. Freedman, 1965:

"The incidence of insanity among addicts is the same as in the general population." Dr Marie Nyswander, 1956.

"As to possible damage to the brain, the result of lengthy use of heroin, we can only say that neurologic and psychiatric examinations have not revealed evidence of brain damage.... This is in marked contrast to the prolonged and heavy use of alcohol, which in combination with other factors can cause pathologic changes in brains, and reflects such damage in intellectual and emotional deterioration, as well as convulsions, neuritis, and even psychosis." Dr. George H. Stevenson, British Columbia 1956.

"Morphine does not cause any permanent reduction in intelligence." Drs Harris Isbell and H. F. Fraser, Public Health Service addiction center, Lexington, Kentucky, 1950.

"In spite of a very long tradition to the contrary, clinical experience and statistical studies clearly prove that psychosis is not one of 'the pains of addiction.' Organic deterioration is regularly produced by alcohol in sufficient amount but is unknown with opiates." Deputy Commissioner Henry Brill, New York State Department of Mental Hygiene, chairman of the American Medical Association's narcotics committee, after a survey of 35,000 mental hospital patients. 1963.

"That individuals may take morphine or some other opiate for twenty years or more without showing intellectual or moral deteriorationis a common experience of every physician who has studied the subject." Dr Lawrence Kolb, US assistant surgeon general, 1925.

The politics of the drug war

The Guardian, February 2001

It is a strange but revealing fact that hundreds of thousands of people in this country are currently afflicted by a dangerous and highly infectious disease and that, even though the government has been warned repeatedly that many thousands of these people will die, the current position of the Department of Health is that they are reviewing the report of an advisory group to decide whether they might then set up a special working group which might then develop a strategy to deal with it.

The disease is Hepatitis C, which attacks the liver. Even though there are probably at least 300,000 sufferers in the UK; even though specialist doctors say that 100,000 of them will suffer cirrhosis or cancer of the liver in the next five or ten years; even though the infection is still spreading: the position remains that the Department of Health has set up no system to monitor the epidemic, has failed to fund any kind of public information campaign, refuses to offer systematic screening or testing for potential carriers, has established no prevention strategy at all and refuses even to treat many sufferers.

The explanation for this extraordinary lack of action appears to be that almost all of the victims of Hepatitis C belong to one of the least popular political minorities in Britain - drug users, who contracted the illness by using dirty injection equipment. Dr Tom Waller, who chairs the medical pressure group Action on Hepatitis C, says it is 'a distinct possibility' that this is the cause of the problem: "This is life threatening to a large number of people. You'd think the Department of Health would want to stand on its head if necessary to prevent it." As it is, many health authorities simply refuse to fund the best available treatment, which involves a combination of interferon and ribavirin.

There is no arm of British health care which has been so perverted by politics as the treatment of drug users. In the early days of American prohibition, this was the politics of racism - spics and niggers smoked marijuana, chinks smoked opium, and they would all get what was coming to them. In the 1960s, it was the politics of reaction - hippies smoked everything and attacked the establishment, so the establishment attacked them back. Now, it is simply the pure politics of power: you win votes by waging war on druggies.

You can see the politics perverting the health care with particular clarity in New Labour's adoption of Drug Treatment and Testing Orders. The problem here is not just the moral one of whether it is acceptable to compel drug users to undergo treatment under threat of punishment. Nor is it simply the practical problem of allowing those who have broken the law to jump the queue for treatment in front of those who have not. The real problem with DTTOs is that they are a political project built on a foundation of falsehood.

It was the Home Office minister Paul Boateng who last September announced that courts all over England and Wales would now be allowed to impose DTTOs to compel offenders to undergo treatment for their drug problems. Mr Boateng explained that his decision followed three pilot schemes, in Croydon, Liverpool and Gloucester which had proved to be successful. Among those who took part in the pilots, he said, there had been a 'dramatic' fall in the number of offences they committed and in the amount of money which they spent on drugs. The reality, however, was rather different.

One of the key questions for these pilots was whether drug users would co-operate with treatment which was being forced upon them. The researchers who were hired to study the three pilots found that, even though the 210 offenders had been handpicked, nearly half of them (46%) vanished or were thrown out of the scheme long before it finished its trial run; numerous others were warned for breaching its conditions; and the researchers found that "failure to meet conditions of the order was common in all three sites". Mr Boateng simply did not mention any of this.

One of the 'dramatic' results to which Mr Boateng referred was that within a month of being put on the order, offenders had cut their weekly spending on drugs from £400 to only £25. This was, indeed, a dramatic fall, which sat oddly with the conclusion of the researchers that "quite clearly, many offenders in all three pilot sites were continuing to use illegal drugs". It turns out that this supposedly dramatic result was based entirely on untested claims made by those offenders who had not already been thrown off the scheme and who knew that if they were caught taking drugs, they were liable to be sent back to court for a harsher punishment. Furthermore, these offenders who were claiming to have cut their spending on drugs by 94% had been failing urine tests throughout the scheme: they had failed 42% of their heroin tests, 45% of cocaine tests and 58% of methadone tests. In some cases, they were failing more urine tests at the end of the 18-month pilot than they had been at the half-way point. Indeed, their consumption of drugs remained so high that, by the end of the trial, all three schemes had stopped even requiring them to be drug free, asking only that they "make progress in addressing" their drug problems. Mr Boateng did not mention any of this either.

The other 'dramatic' result on which Mr Boateng relied for his success story was that, within a month, offenders were committing far less crime - only 34 offences a month compared to 137. But this, too, was based on nothing more than asking the offenders who stayed in the scheme whether they had been out thieving. Mr Boateng failed to mention that some of these law-abiding guinea pigs were actually arrested for committing new offences during the pilots. At the end of the 18-month scheme, the researchers could find only 27 of the 210 offenders who "seemed to emerge drug free" - and they were able to come to that conclusion only by a) overlooking the fact that only 13 offenders passed the final urine tests and b)ignoring their use of cannabis. The best that the researchers could say was that the scheme was "promising but not proven."

However, none of this troubled Mr Boateng. Even though these pilots had been set up explicitly "to enable the Home Office to decide whether or not to extend the order across the country" and even though the results were so equivocal, Mr Boateng went ahead and declared them 'successful' and invested £60 million of tax payers money in rolling them out nationally. He managed to square this with the results of the pilot studies with one brilliantly effective tactic: in a move which left his researchers 'flabbergasted', he simply did not wait to be told the bad news and made his decision months before the results of the research were known. And this really did not matter at all because even if the scheme does fail, its no-nonsense toughness on druggies has been a great success from the political point of view.

The real problem, however, lies deeper - in the profound and alarming ignorance of the power elite. There are vocal politicians and senior officials who make policy on drugs and there are leader writers and pundits who support them, and yet they genuinely do not know the first thing about them. Specifically, the politicians' love of prohibition identifies the drugs themselves as the source of danger to their users. As the Guardian showed yesterday, the truth is that the real dangers come from the blackmarket which has been created by prohibition. By refusing to acknowledge this medically verifiable fact, the politicians have created a treatment strategy which consistently pushes highly vulnerable drug users into extreme danger. Take heroin as an example.

Until the early 1970s, Britain was a haven of enlightenment: every doctor in the country had the right to prescribe heroin for the welfare of patients. This reflected the idea, powerfully proposed by the Rolleston Committee in 1926, that drug use should be seen as a problem which needed help, not as a sin which needed punishment. There were fewer than 500 addicts in the country, most of them musicians or Chinese. With a clean, legal supply of their drug, they remained healthy and were able to live normal lives. Then three London doctors were caught selling inflated prescriptions; there was a moral panic; and Britain's resistance to prohibition started to crumble under political pressure, some of it from the United States which was already committed to imposing a global policy of prohibition.

The result was that doctors generally were forbidden to prescribe heroin to addicts, who were thus forced to buy their supplies illegally: the blackmarket started to grow, inflicting illness and infection on addicts and embroiling them in theft and prostitution to find funds. A small detachment of common-sense realism slipped under the fence, but was soon pinned down by hostile political fire: the Home Office agreed to license specialist psychiatrists to continue to prescribe for heroin users. This might have saved addicts from disaster, but, as the babble of the prohibitionists drowned the voice of reason, the Home Office - apparently under more pressure from the United States - undermined the system by insisting that these licensed doctors should prescribe heroin substitutes, such as physeptone and methadone, instead of heroin. Furthermore, the Home Office insisted, these substitutes should be prescribed only in rationed and rapidly diminishing quantities.

This sealed the catastrophe: most heroin users did not like physeptone and methadone and sold their supplies; those who did like them found their supplies were rapidly cut off. In either event, to satisfy their addiction, they were pushed back onto the blackmarket, back to the dangers. The British System of support for addicts, which had been admired around the world, was dead.

Since then, it has emerged that the government's favourite heroin substitute, methadone, is more addictive than heroin and also more likely to cause fatal overdose. In a detailed study, 'Methadone and Heroin, an exercise in medical scepticism', Dr Ben Goldacre found that: "Methadone is a more dangerous drug than heroin, and causes more deaths than even adulterated street heroin". A study by Dr Russell Newcombe, senior lecturer at John Moores University, Liverpool found that methadone was four times more likely than heroin to cause fatal overdose. And yet - for entirely political reasons - this is the drug which the government insists be prescribed to heroin addicts.

The bottom line now is that after thirty years of prohibition, the number of heroin addicts has rocketed from less than 500 to as many as 500,000. Around 20,000 of them are being given the arguable benefit of a limited prescription for methadone. And the number of heroin addicts who are allowed a limited prescription for a safe supply of the drug to which they are addicted is less than 500. The hundreds of thousands of others are are thrown out onto the blackmarket, condemning them to precisely the dangers from which which the politicians claim to be saving them.

New Labour's strategy for the treatment of heroin users compounds all of these errors - consistently increasing the risk to addicts. So, for example, ignoring more than 15 years of medical warning on the relative danger of methadone, the department of health's new 1999 prescribing guidelines, known as the Orange Book, continue to advise doctors who care for heroin addicts to prescribe methadone instead of heroin. And, repeating the policy which for 30 years has pushed addicts into the dangers of the black market, the Orange Book continues to urge that doctors should generally prescribe only in rationed and rapidly diminishing quantities.

The Orange Book makes matters even worse by giving GPs an explicit responsibility not just to prescribe the approved quantity of methadone but then to ensure that "the drug is used appropriately and not diverted onto the illegal market". GPs have no such power. The result is that, spurred on by the government's ferocious rhetoric, police have moved in on doctors whose patients have sold their methadone or overdosed: GPs in Carlisle, Essex, London, Luton, Plymouth, Portsmouth, Suffolk and Surrey have found themselves in serious trouble. This, in turn, has had a chilling effect on other GPs who might have considered prescribing methadone to local users as a temporary refuge from the blackmarket. Professor Gerry Stimson, of Imperial College London, who has studied illicit drugs for 30 years, told us: "We're seeing court cases against doctors and other drugs workers or police attention to prescribing doctors which is actually scaring many doctors away."

Across the field, the government's professed desire to offer more treatment to drug users is being undermined by its hardline politics. Chemists who try to supply prescriptions of methadone or diamorphine report hostile visits from police. The All Party Parliamentary Drugs Misuse Group last year took evidence of a psychiatric ward where drug users seeking treatment had been confronted by police with sniffer dogs. A north London priest, Father Peter Anderson, found himself denounced by the local coroner for supposedly condoning criminal activity because he had allowed homeless drug users to sleep in the grounds of his church. A Release conference last year heard that, if a drug user overdoses, other users are often scared to call an ambulance for fear of being arrested.

Professor Stimson says the root of the problem is the government's ferocious rhetoric: "It sets the wrong tone. You are dealing with people who are already quite marginalised and stigmatised and, if you are having that sort of rhetoric, then you are pointing the finger, scapegoating people. But also politicians get carried away with that rhetoric and they become tougher, they dream up new legislation, they dream up tougher ways of doing things which can backfire and can have adverse effects."

So, for example, the government wants police to be able to deny bail to anyone they suspect of being involved with drugs (so users will be discouraged from carrying their own clean needles or drugs-advice leaflets); to introduce new licences to limit the number of doctors who can prescribe injectible methadone as opposed to the oral linctus (so users who like to inject will end up using blackmarket needles in dirty conditions); to remove the passports of anyone who has a drug offence (so no past user will ever be able to enjoy a normal life).

The drugs war is a political war. It was political when, as Edward Jay Epstein recorded, President Nixon gave a shot in the arm to his election campaign by fiddling the figures to create a non-existent heroin epidemic, from which he could then promise to rescue the electorate. It was political when Tony Blair announced his plans to appoint a 'drugs czar' - in a secondary school full of sweet, vulnerable children in the middle of the 1997 election campaign.

The masters of the war have always been American politicians. When the Swiss held a referendum on limited heroin legalisation in 1997/8, the US congressional subcommittee on national security, international affairs and criminal justice openly intervened. "We wholeheartedly oppose this sort of government gambit", the committee declared, as though it had some sort of jurisdiction in Switzerland. When Dr John Marks was forced to close Britain's most successful recent project to provide clean heroin for addicts, on Merseyside, the International Herald Tribune carried a report that American drugs agencies had been infuriated when they saw the project on CBS television: "Dr Marks was warned by friends in the Home Office that the US Embassy was exerting tremendous pressure to shut him down and, in the end, it was successful."

New Labour drugs policy has been shaped by political in-fighting. The Home Office tried to stop Keith Hellawell setting targets for reduction in drug use. Hellawell went off to Downing Street and got the Prime Minister on side. The Home Office then complained that the targets were too high. Hellawell persisted and then found reporters were being briefed that his targets were nonsense since nobody knew how many people were using drugs now, so there was no baseline to set a future target. In the meantime, the Prime Minister announced that anyone who is arrested will be urine-tested for drugs; Hellawell's people had a fit because they hadn't approved the plan and , very soon, reporters were being briefed that the PM's plan would be "kicked into the long grass". And in the background, Hellawell was falling out with the department of health who produced their Orange Book guidelines for doctors without consulting him. "I am the line," he announced.

In this politically-charged atmosphere, it is a heresy to question the value of prohibition. Transform, the only pressure group campaigning for legalisation, wants to commission an opinion poll which may show politicians that public opinion has moved ahead of them, but, at the moment, nobody with any power dares to break ranks. The interesting thing is that the group who in private are now most keen on legalisation are chief constables. We spoke to four of them who were passionately opposed to the war against drugs. None of them would speak publicly. What they can see, however, is that there is a way out. The war against drugs is unique in all conflict: we can win it, simply by ceasing to fight it.

The faith of well-meaning liberals in New Labour's plan to offer more treatment for drug users has all the moral force of well-meaning Christian folk in the nineteenth century who considered the use of child labour in Victorian coal mines, saw that it was wrong, lacked the political or intellectual courage to say that it must stop and suggested instead that their hours of work might be limited. No treatment strategy will succeed for as long as it is based on the medically false but politically popular idea that the nature of these drugs is such that they must be banned.

Future historians will look back on our treatment of drug users in the same way as we now look back on the Victorian treatment of those in Bedlam - beaten for their pain. Every victim of the war against drugs is a lesson in the futility of the war, a screaming message of contradiction to the politicians' errors. They may have become drug users for all kinds of reasons - the pursuit of pleasure, or obsessive flight from pain - but most of those who have lost their jobs or homes; most of those who have been driven into prostitution or thieving; most of those who have become ill or who have died, have been sacrificed to the ambition of politicians who never did have any reason to attack them but who continue to do so now only because they are too stupid or too ignorant or too callous or too plain scared to admit the truth, that, with their policy of prohibition, they are themselves the architects of this disaster.

Additional Research by Max Houghton

System failure: How to lose the fight for law and order

The Guardian June 2003

Right there. That's where they got the Yardie guy. He was in that pub, the Jolly Roger, over on the corner of All Hallows Road and, although it's dark now and our van is racing, we can still catch a glimpse of the lamplit pavement where he lay with his blood pooling over the kerb and onto the tarmac street.

There were people standing all around, but it was one of those times when nobody saw anything. It seems like the men with the knives sent a message into the pub - "Tell Chrissie to come outside" - and for some reason (maybe he was stupid, maybe he was just too cool to be scared) he went out to them, and they slashed him 24 different times, tore open his belly and stabbed him through the skull on the neatly swept pavement of a quiet street in a city in south west England.

The inspector in the van says they never caught the people who did it. They tried. They arrested something like 19 different men including a former police officer - most of them Yardies from East Kingston, which is the original home of the Hype Crew and the Mountain View Posse and the Back to Back Gang and of most of the other Jamaican gangsters who have moved into this part of Bristol. But they couldn't prove anything against anybody. They just didn't have the evidence.

And that's how it is. The inspector knows it. Any police officer of any rank knows it. What see you on The Bill is not what you get in real life. Most of the time, most criminals get away with most of their crimes. The Home Office know it too. They crunched together their best statistics and analysed a hundred typical offences and then they worked out how many of them were actually brought to justice (which means a conviction, a caution, or being 'taking into consideration' for sentencing). And the answer they found was..... three. To put it the other way around: as far as we know, from the best research available, 97% of offences are never brought to book.

And that's the strange thing about this criminal justice system. It has power and, ever since Sir Robert Peel first picked up a truncheon, it has been gathering more. It has money. It is the one and only limb of the public sector which escaped the cuts of the Tory years. Even defence took a dip with the 'peace dividend' after the fall of the Soviet Union. But all through the years of the cuts, the funding of the enforcement of the law grew, from less than £5 billion in the early 1980s until, by the end of the last Major government, we were spending £16.2 billion a year on chasing and punishing criminals, most of whom were not Yardie gangsters but pimply adolescents armed with nothing more sophisticated than the sawn-off top of a Pepsi bottle (they use it to prise open Yale locks). And they were still getting away with 97% of their crime. The criminal justice system bristles with questions. There is only one that really matters. Who's winning?

An hour ago, the inspector addressed the troops. They were in the old police gym, 40 men and women in white shirts and black stab-proof vests, sitting on crooked rows of plastic chairs in amongst the battered punch bags and weight-lifting machines. A rusty bucket collected rainwater at the bottom of the concrete steps. It was very quiet while the inspector spoke, and they sat with their arms folded and their chins on their chests. "This is a really serious situation," he told them.

He described how the Yardies were being attacked by Bristol-born dealers who call themselves the Aggi Crew. The Aggis used to run the St Paul's area in the city centre, which is a magnet for drug deals, but a few years ago, the police busted them, and they all went to prison. Now they're out again and they want their turf back. A few weeks ago, five of them put on balaclavas and picked up guns and made a personal tour of the local bars - the Black Swan, the Malcolm X Centre, Lebeqs and the Caribbean Club - and announced that they were taking over. They offered the Yardies a deal: they could stay and work in St Paul's, but they'd have to pay a tax for the privilege. £50 a day for each Yardie and £100 a day for any business they were running.

The Yardies didn't like that idea. They met in one of the local cafes that has an illegal gambling room in the back, carrying their Mag 9 automatic pistols and their Brocock Magnum air pistols converted to fire live rounds, and they agreed that, first, they would not be paying the tax, and second, they would take on the Aggis. But one of the Yardies changed sides and went over to the Aggis (for a fee). The next night, in Badminton Road, he caught up with one of his former mates and shot him through the back of the knee with a 9mm handgun. The main man in the Aggi Crew repeated the message in the Caribbean Club, where he started jeering at a bunch of Yardies; he attacked one of them, a guy called Dufus, and hit him across the face with his pistol; Dufus went home and got his gun and came back and shot the Aggi (didn't kill him.)

So, naturally, the Aggis attacked the Yardies' street-dealing operation, staged some rip-off robberies, stole their drugs at gun point, and kidnapped one of them - they hauled this guy out of his home, locked him up in the boot of their car and drove him off somewhere quiet where they pistol-whipped him. And just to make the point that they were in charge now, two Aggis walked into the Black and White Cafe on Grosvenor Road and robbed every single Yardie in there at gun point.

If tonight's the night that the Yardies hit back, there is going to be blood on the streets. The inspector tells his officers: "The danger is that there are people out there who have killed before, in other countries." He tells them their priorities: above all, community reassurance; apprehend drug dealers; take firearms off the street; public safety; flexible response to whatever goes off. In one corner of the gym, a group of the officers are carrying handguns. For the first time in the history of Bristol (and for almost the first time in the history of any British city) there will be armed police on the streets - not covertly in plain cars, but overtly where the people of St Paul's can see the protection and where the gangsters can see the threat.

But the rest of them will be unarmed. The inspector tells them how to contact the special intelligence cell, to check car numbers and IDs; how to link up with the 24-hour charging unit who will handle all the paperwork on arrests. He shows them mugshots of the Aggis and warns them to look out for the mixed-race male with the burgundy scarf who shot and pistol-whipped a man on Saturday night. He has a silver revolver, his mate has a seven-inch knife. He tells them not to use their radios - there are scanners out there, so stick to mobile phones. "And don't self-deploy," he tells them. "This is about your safety as well as public safety."

There has never been a human community without crime. There are thousands of specialist academics around the world who have tried to put their finger on why some communities generate so much more than others, why some individuals are so criminal, why some victims are so vulnerable. Some thought that crime was linked to consumption: more goods, so less need to steal. Then they discovered the opposite: more goods, so more opportunity to steal. Then they discovered the opposite again: more goods, so lower prices, so less that is worth stealing. No theory fits. Not poverty, or inequality, or maternal deprivation or paternal absence. They all have some impact - big sources and small sources, all constantly shifting in power and in relationship to each other. It's like trying to map the wind - infinitely complex.

And crime control is just as complicated. Councils put up street lights: the night crime falls; but the day crime falls too. Nobody knows why. New York police clamped down on their first generation of crack dealers, imprisoned masses of them, but gun crime soared: the second generation were younger, more impulsive and they had seen what had happened to their older brothers. People carry credit cards instead of cash; mugging falls. They get mobile phones; mugging rises. The police chase around behind them. Governments hire more police officers to cut crime; the extra officers discover more offences; so recorded crime rises. Crime has risen almost constantly for many decades, and yet the criminal justice system now delivers fewer detections and fewer convictions than it did only 15 years ago.

An hour after the briefing, the inspector in the van prowls St Paul's, his ear-piece crackling with updates from the men on the women on the street. This is the police at maximum force - manpower, fire power, intelligence back-up. Who's winning? At the moment, it's quiet on the streets - on Foster Street where a man had his car stolen from him at gun point, on Denbigh St where they stabbed a man in the street before they robbed him, outside the St Nicholas pub where somebody absent-mindedly left a fistful of shotgun cartridges on the bar. The inspector cruises past the Black and White Cafe and automatically glances up at the roof of the council flats across the road. The Aggis were up there the other day, firing giant fireworks like mortar rounds at the cafe door, to oust the Yardies. They've fired them direct at his van too, from close range.

Tonight is just the latest crisis. This has been building up for two years. The night they murdered Chrissie Hewitt outside the Jolly Roger was more or less the beginning. That was in June 2001 and the police knew then that something strange and terrible was happening in St Paul's. There was crime bubbling out of the ground like swamp water. Some of it was hideous - vicious assaults and kidnappings - and, when they looked back at the end of the year, they found their 'major and serious' crime had shot up by 72%. And the ordinary everyday crime had shot up with it - robberies nearly doubled that year. People breaking into cars, breaking into houses, snatching bags in the street.

That summer, the St Paul's Carnival turned into a shooting match. The Burga Crew came down from Birmingham; one of the Bristol Yardies shot one of them with a nail gun; the Burga Crew pulled out their guns; the Bristol guys got theirs; somebody had a machete. That night, there were at least five separate gun attacks. Nobody even really knew what it was all about - might have been a squabble about a drugs deal, might have been a ripple from some turf war over in Kingston, where the Yardie gangs are locked into alliances with local politicians, controlling whole neighbourhoods with corruption and fear.

The police were ahead of the game. As early as February 2001, they had spotted the Yardies coming in and set up a special team to gather intelligence. They found there were Yardies crawling all over St Paul's, maybe 200 of them, some of them well-known in East Kingston as killers. Within a week of that first Yardie murder, they had pulled in 19 more officers to tackle the open drugs market which was now booming around Grosvenor Road and to target the Yardies who were behind it - who had been giving away crack cocaine and selling 'snowballs' of heroin and crack combined, expanding their client base. On Grosvenor Road, the dealers had started wearing bullet-proof vests.

That autumn, 2001, the police were all over them. They found a local guy was running a phony college on Lower Ashley Road, selling enrolment to non-existent courses so that gangsters and their smugglers could slide through immigration controls - 300 of them in less than a year. They set up a new unit to track their money, busted a greengrocer and a travel agent for money-laundering and followed streams of cash from Bristol back to Kingston - just under £10 million in their first year, some of it buying property in Jamaica, some of it funding new crack consignments to the UK. And they arrested hundreds of street dealers, something like 700 of them in a year.

By now they had set up a special unit, Operation Atrium, which pulled in officers from all over Avon and Somerset. All the divisional commanders lost men and women. They pleaded for replacements and eventually the local police authority agreed to backfill the lost manpower. It would take 18 months to fund, hire and train the new officers. The street dealers they were arresting were being backfilled in less than 24 hours.

This was a police force recognising a threat to one of its communities and defending it with all of the weapons of the criminal justice system, and... it made no difference. It was like trying to walk on water. Each day began with dozens of reported crimes; each day ended with most of them undetected. The police would arrest people; the courts would give them bail. They would deport people; the airports would let them straight back in again. One of the Yardies they arrested for the murder outside the Jolly Roger was a guy known as Mr C. He was here illegally, so they deported him. He came back; they caught him a few months later and deported him again. In October 2002, they found him a third time and deported him yet again. This time, he was slower to come back: he got shot by another gangster in Kingston. And all the time the Yardies were dealing, bringing in more users, who committed more crime.

Even when the courts did lock someone up, it was nothing more than a breathing space. The ones who were worth locking up - the 'prolific offenders' - came straight back out and carried on offending. Down the road, for example, the city centre car parks have just seen a surge in break-ins on cars, simply because one local lad has been released from prison and gone straight back to the life he knows best. (There's another lad down there who's also a specialist in car crime, but he lives on the job: he sleeps in a multi-storey car park, wakes up, robs a few cars, goes out for the day and drifts back to base when he's tired or needs some more cash. He's been arrested more times than they can count, only to pop out of one of the multiple exits in the system and carry on thieving.)

The inspector turns out of St Paul's and stops in a street full of shops on the edge of the City Centre. He wants to check the cells in the Bridewell. At the desk, he ticks charts and signs forms. Down in the cells, he tells the girl who's been done for shoplifting that he'll get her a light for her cigarette, tells the man who is sitting in total darkness in the next cell that he'll be seen by the doctor very soon. This is the 3% in real life, the offenders who have finally been brought into the system. This is where you begin to see that the machine whose neck is so narrow that it misses 97% of the offences it is aiming at, then lacks the equipment to change the behaviour of most of those it does manage to catch.

Sometimes, it makes no difference simply because it is too weak to enforce its will. Most of the offenders who are convicted will end up in magistrates court where the bench may solemnly fine them. And 41% of those fines will never be collected. We know that, because the Lord Chancellor's department, which is responsible for the courts, checked the numbers. They reckon it would take four years just to collect the backlog even if there were not something like 320,000 new fines being doled out each year. In the financial year to April 2001, the courts wrote off £74 million of fines which they had failed to collect. The offenders simply shrugged and walked away.

It's the same with community sentences. The courts handed out 169,000 of them last year (and they are much fiercer than they used to be), but tens of thousands of offenders simply don't turn up to be punished. With the most severe - the Community Punishment and Rehabilitation Order - the Home Office's records show that 47% of offenders who were supposed to serve community punishments simply didn't bother. >>> Of course, their breach was eventually reported to the court which duly issued a warrant for their arrest, but we know from the Audit Commission that 56% of those breach warrants were never executed. They just piled up on a shelf somewhere.

And then there's prison. At least with prison, the sentences are usually enforced; very few inmates now escape. But here is the radical contemporary critique of its impact: "Prison sentences are not succeeding in turning the majority of offenders away from crime. Of those prisoners released in 1997, 58% were convicted of another crime within two years... The system struggles particularly to reform younger offenders. Eighteen-to-twenty-year-old male prisoners were reconvicted at a rate of 72% over the same period.... At a conservative estimate, released prisoners are responsible for at least one million crimes per year... In terms of the cost to the criminal justice system of dealing with the consequences of crime, recorded crime committed by ex-prisoners comes to at least £11 billion a year." That is the verdict not of some maverick commentator but of the Prime Minister's Social Exclusion Unit.

The inspector ticks the last form in the Bridewell and drives back to St Paul's. And we drift through the dark together, the inspector and the reporter, surrounded not so much by the life of crime as by the near-death of the criminal justice system. In the dark shadows of Brunswick Square, a pimp sprawls across the front seat of a Mercedes, smoking and eye-ballling the inspector. Not five yards from him, there is a bright yellow sign, posted by the inspector, warning punters that their car numbers may be recorded by CCTV if they cruise the square. The pimp's girls are working just around the corner, standing there as thin as string and using the old graveyard behind the unitarian meeting house to deal with their customers. The inspector could arrest dozens of working girls every night: he'd tie up multiple hours of manpower to send them to court where they'd just be fined, and then they'd be back again, working even longer hours to pay the magistrates as well as the pimp.

The failure of the system is mirrored by the complacency of many of those who talk about it. The government will tell you it is hitting its key targets, right-wing pundits still cheer the idea that prison works, liberals twist themselves around statistics like beans running up a pole, to prove that all is well. The truth is that the evidence that crime is falling is about as solid as mist (see sidebar). Even if, behind the veil of numbers, the truth is that nationally crime is coming down, communities like St Paul's - always the most impoverished - are nevertheless pockmarked with streets which tell a different story.

There's a family who live not far from here with five adolescent boys. All of them are thieves. They all steal almost every day. One of them was at it on Christmas Day. Another brother recently was arrested and released on bail - he was caught at it again within an hour. There's another lad whose routine is to go out screwing houses each night - at least three of them. Every night. And they steal from here. They don't go off into the middle class areas - that's foreign territory, they'd stand out on the street, and they don't know the houses or the routines of the people. Thieves steal from their neighbours.

Those who raise their claret to the success of law enforcement should talk to the primary school teacher here who has to tell her ten-year-olds that it is not cool to carry guns, that the flash lads who parade on the pavement with the rolls of cash and the chunky gold jewellery are not role-models; to the parents of those children, trying to get to school in the morning without stepping on bloody syringes; to the man who says he can't reach the corner shop to buy a paper without being hassled by some toerag with a hood over his face; to the old woman clutching her handbag like a baby as she takes her pension home from the post office. The council street sweeper here the other day was stopped in his work by a guy who jammed a gun up against his head: it turned out that the dealers had been stashing rocks of crack cocaine in old milk cartons on the pavement, so that they could sell their drugs without actually holding them, and the sweeper was demolishing their business.

Most of all they should talk to the police, who really know who's winning out there. Those on the right might learn that a lot of these officers have zero faith in their zero tolerance gospel. The left might learn that something important has changed. Twenty years ago, this country's police were led by a generation of chief officers who had left school at 15 and learned their lessons in life from the army. Frequently, they were crude and unsophisticated. A lot of them didn't like hippies or lefties or queers. Some of them were racists. But a great deal has changed. Partly, this is about one of the great success stories of state education. In 1976, the Edmund Davis report gave the police better pay and far better pensions than anybody else in the public sector, and so - slowly at first - the police service began to attract working class men and women who were the cream of comprehensive education. They joined a force which was then battered by a series of traumatic scandals - ruthless corruption in London and Birmingham, the miscarriages of justice around IRA bombing trials, urban riots spawned by clumsy and sometime vicious street policing, the Yorkshire Ripper inquiry, Stephen Lawrence. They had to change.

>>> Of course, there are still police officers who abuse their power, who are idle and incompetent and dripping with prejudice, but far more than ever before, there are officers who are neither the lapdogs of the right nor the demons of the left. And they are losing. It is not for want of trying. They work in a system that bristles with power, that is rich in individual effort and (relatively) rich in resources. Every day, they win battles and yet overwhelmingly they are still defeated in the war.

That's not because they are all idle or corrupt or racist. It's not even primarily because of the practical difficulties of their work - that most victims (55% of them) do not even tell the police that they have suffered a crime; that it is simply very difficult to detect an offender whose crime was not witnessed; that more witnesses now are intimidated and unwilling to come to court; that courts which demand proof beyond reasonable doubt necessarily release some of the guilty. The real problem is that these officers are running a system which has a design fault on the scale of the Maginot Line: it confronts a problem of infinite complexity with a response of gross simplicity. It approaches all crime with the same rigid routines - patrolling, detecting, imprisoning. They were born in the distant past; they are strikingly ineffective (see sidebar). Those antique rituals are themselves based on one rigid underlying idea which is equally weak: that all criminals will behave in the same rational and predictable fashion; that they will all be deflected by the prospect of arrest and the threat of punishment.

Conventional law enforcement does not fail completely. Its great achievement is that it regulates the behaviour of law-abiding people, who do pick up the signals it sends and react to them. Its great failure is in dealing with criminals. Think about those Yardies, about the five brothers burgling each day away. Conventional law enforcement rarely catches them and scarcely deters them. It controls them to the precise and limited extent that it can lock them up and incapacitate them - about a tenth of the 3%. But that's more or less it.

The government relentlessly ratchets up public expectation for law enforcement. And yet: "Despite all the changes of recent years, the signals sent out are ones of weakness in critical areas: too few criminals brought to justice; too many defendants who offend on bail; too slow to bring them to trial; too many guilty go unconvicted; too many without the sentence they and society need..... Far too many offenders escape justice, creating the ‘justice gap' between the number of crimes recorded by the police and the number where an offender is brought to justice." That is from the same government's preamble to its most recent criminal justice white paper. They put more money into more officers following the same rigid rituals. And yet: "There is very little correlation between levels or changes in funding and crime rates, or between changes in police numbers and crime rates." That is from the Audit Commission.

We spoke to a civil servant who has spent most of his working life in the criminal justice sytem. He put it this way: "The point is that the old Benthamite theory of a rational system producing results has collapsed. We are left with a different set of objectives which are really to do with pleasing people - the electorate, the civil servants, the prison officers.... The public and the press sing a contant mantra about putting more police back on the streets. It doesn't work. Same with long prison sentences in unpleasant conditions - they don't work. But they do work as ways of making people happy. So that's what we do." And we spoke to people at one level or another of every law-enforcement agency in the country and, over and over again, they came up with a version of the same wry line: "The criminal justice system doesn't catch criminals, it doesn't dispense justice, and it's certainly not a system."

Any government which confronts this reality is faced with a fork in the road: go down one route and increase the power and funding of the existing system in the hope that it may do a little better and in the knowledge that every new power for the police is a potential threat to the public co-operation on which they rely; or take a new route, recognising that a complex problem requires subtle solutions, and break out of the boundaries of conventional law enforcement in search of intelligent alternatives. Since April 1997, generally without public recognition, this government has done both.

It has made far more noise about its journey on law enforcement. It has taken the criminal justice system and shaken it like a Victorian nanny used to shake an insolent child. It has injected the whole system with an almost dizzyingly energetic programme of reform and used criminal justice bills for the hazardous purpose of giving it new muscle. But, behind the almost ceaseless flow of law-enforcement rhetoric, they have also looked outside the system. They have taken the advice of some of the most radical criminologists in the country and embarked simultaneously on a completely new approach, trying to link up all the agencies of social welfare to draw the criminal poison out of impoverished communities. The effect of this twin strategy has been electrifying. The great question is whether, as a result, the police have started winning.

In a small way, the inspector won tonight. The streets of St Paul's are quiet, and they stay that way - until two in the morning, when he has to pull back his officers, and the dealers and the gangsters will come out to work. They are working even now, around the corner on the pavements in St Jude's, and in their cars using mobile phones to find their buyers. The drug-buyers who have been robbing and burgling in St Paul's to find the cash for the dealers, have moved up the hill temporarily to St Lawrence's where they have been mugging the prostitutes and unleashing a mini epidemic of burglary.

Success is hard to find. This wave of crime, which has now become a shooting war, was almost certainly triggered by what looked like a law-enforcement victory: it was only because the police, through diligence and skill, arrested the Aggis a few years ago, that the drugs market in St Paul's had no owner, and the power vacuum sucked in the Yardies with their crack and their guns. The best available tactic for stopping the war now - quicker than arrest and prosecution - is for the police to link up with probation, using minor offences and intelligence to revoke the parole licences of the leading Aggis. That will put them back behind bars. For a little while.

The inspector has stopped the van for a moment in a dark side street. The radio plug in his left ear is babbling with feedback about Yardies and Aggis. There's another radio hooked into his right ear feeding him with reports of ordinary life on his sector. Somebody can't find the charging unit to handle an arrest. There's a 15-year-old girl wandering through the middle of the operation. The mental hospital has lost a patient, and they think she may be headed for the river. There's a battered woman and child at Trinity Road police station, looking for a refuge. The inspector needs his mobile to get the helicopter to look for the hospital woman. Another mobile in his pocket starts to ring; he's eating a sandwich; he can't read his notes; he reaches for the light, accidentally hits the siren and for a moment or two, we just sit there, wailing in the darkness.

Who's winning?

Additional research by Tamsen Courtenay

The futility of punishment

The Guardian, April 2004

There is order in the court. There is chaos on the streets. And they meet in the main hall of Thames magistrates court in the East End of London. It is busting with people - this guy made of muscle yelling at his tiny female lawyer "This is MY case, this is MY life"; the elegant Somali man with the beautiful black suit cruising quite lost through the crowd without a word of English to find his way; the young Bengali lad who just blew a spliff in the toilets; the prosecutor reading "God Knows" by Joseph Heller; the little knot of regular defence lawyers, Charlie and Teresa and Denis and Keith, swapping the gossip and wondering how long it will be before somebody tells them the new security code for the door to their room so they can finally start work for the day.

The courtrooms open. Here come the overnights - the people who've been arrested by Hackney police and held in the cells for this morning's hearing.

Case One: male, white, aged 26, charged with theft of eight packets of bacon from Londis supermarket - tested postive for heroin and cocaine. Case Two: male, Bengali, aged 19, charged with theft of £59.63 of food from Tesco - tested positive for heroin and cocaine. Case Three: male, white, aged 32, charged with harrassing his mother and stepfather by kicking down their door, breaking their window, threatening them in persistent search for money - tested positive for heroin and cocaine.

The cases blur. Amost all male, almost all young, almost all of them trailing a string of other cases that have just been dealt with or are about to be dealt with, almost all of them already serving one or more community punishments, many of them with unpaid fines. And over and over again, the magistrates' mantra, "tested positive for heroin and cocaine". (Finally, an overnight defendant appeared who had not tested positive for drugs. He was an alcoholic.) Most of them plead guilty and are adjourned for reports.

This is the story of a week in court - the story of what happens when the criminal justice system finally deals with the mere 3% of offences which it manages to capture. The courtroom is the crossroads for all the players in the system. Here is where the police send those they have arrested. Here are the prosecutors with the evidence they have collected. Here are the probation officers and the prison escort officers waiting to dispose of the guilty. Here are the offenders "brought to justice". And here is the question: what does it achieve?

It is a question of acute importance in a system which now more than ever relies on court-sanctioned punishment as its mechanism for controlling crime. As this series has already described, the Home Office has quietly compromised its innovative crime-reduction programme, including its support for problem-solving policing; and the National Treatment Agency has taken the promising initiative to give treatment to drug users and smothered it in misconception and mismanagement. In the background, the probation service, which was once the world leader in tackling the causes of crime, has been sliding into a state of demoralised chaos - restructured, mismanaged, starved of funds, overburdened and now on the verge of being restructured yet again. With its best efforts at rehabilitation struggling, this government now presides over a system which is overwhelmingly devoted to punishment, handing out more jail sentences and more community punishments than ever before in the history of this country. What does it achieve?

Look at this man here, standing in the dock, fiddling with his fingers. There is something Victorian about him. It might be the respectable best suit he is wearing - white shirt, black tie and a black jacket that's so big he must have borrowed it. It might be his supplicant posture, his hands neatly folded in front of him, his head hung down on his chest. It might just be the sheer frailty of the man. He is as thin as a whippet, his face is as pale as paper, and he is quivering slightly as the district judge and the clerk of the court assemble the facts about him.

At first, the case of Mathew King sounds simple enough. On January 17 last year, in this same court, he admitted driving whilst disqualified and trying to syphon petrol out of somebody else's car. He tested positive for heroin and cocaine. He was given a 12-month community rehabilitation order which meant he had to keep regular appointments with a probation officer, but in September, he failed to turn up twice, and now he is charged with breaching the order. He admits it, and the judge adjourns the case for three weeks so that probation can prepare a pre-sentence report on him.

Mathew King turns and slips quickly out of the court and, just as the door swings shut behind him, there is a fleeting glimpse of his life - a small woman with long blond hair and a face like a snapshot from another world. It is slightly frightening and immediately haunting and somehow familiar, the way her skin is stretched so tight across her cheeks that it has made her eyes grow large: this woman has the face of a starving child. Again, there is something Victorian here. And she is pulling at Mathew's black-jacket sleeve - no time to lose. Why the urgency? Where are they going with their pale, skinny bodies? What goes on in the private world of Mathew King?

Tracing him was easy enough: he had given his address at the start of the hearing. It turned out to be a tower block in the backwash of a motorway in the heart of what was once the teeming bustle of the London docks. The lift was busted. The stone stairs smelled of piss and garbage. On the fifth floor, just outside his flat, somebody had smeared something brown and nasty on the wall. And when the door finally cracked open on the security chain, there was Mathew and his world, shaped by chaos.

He was just back from Bethnal Green Road. He had gone there with the woman with the hungry face, who turned out to be his girlfriend, Lisa, and with John, who turned out to be the father of Lisa's two oldest children. The three of them live here together and every day they go to Bethnal Green Road to buy their gear. Everybody does. But today it went wrong. John gave £70 to this black lad on a bike. The black lad's mate kept Mathew and Lisa talking round the corner. The black lad rode off and just didn't come back, but Mathew knows him from a year ago and he's got a mate of his coming round - he's well over six foot and built out of bricks - and they're going to go back down there and sort him out. But just now they've got no gear.

Lisa sweeps the carpet. She is 29, although she looks nearly twice that. John fiddles with his fingers, which are mangled and maimed by bad injections. Mathew says he's had a letter from probation about preparing his pre-sentence report, and John gets it for him and - just look at this - it says "Please come and see me at this office. See address below." But there is no address below! And no phone number. So Mathew's going to miss another appointment. At least this letter reached him. The last lot got sent to his mum's house, and he can't go round there because he doesn't get on with his step dad. He's had all this with probation before. The last time he went, he said "What time have I got to be here?" and they're "You haven't got to be here." Then he tells the story of how he missed the appointments in September.

It's all about him and Lisa and the terrible business with the baby. But then again, it's not - it's all about their whole life. Mathew is 29, the oldest of two brothers, born around the corner in Mile End. His father was a drunk, used to smash the house up, and, when Mathew was five, he left. His mum gave him a chance to come back, but he just turned up and stole the stereo. Soon afterwards, his dad's mate came out of prison and came round to see him and found he wasn't there, so he moved in with Mathew's mum, and he's still there. His dad is still in the East End somewhere, still drinking. As far as Mathew is concerned, that man doesn't exist.

He went to Hackney Down school and he was no good at it, he still can't read or write except for his name, and the teachers picked on him, so he swore at them, and they sent him home, so he carried on swearing and didn't do no more school. The first time he was nicked was for breaking into a car when he was 13; he got a caution. He is good at breaking into cars. His crime was just silly stuff, though, until he started using gear when he was about 18. Then he got into nicking cars - he's sold a lot of taxis that way - and he was doing burglaries too (shops, not houses).

Lisa and John both came up the same broken ladder. Lisa's dad left, and she doesn't get on with her stepdad. John's dad died of cancer when he was young and then he got teased for wearing crap clothes because they were poor. Both of them were no good at school. John beat up a teacher and left early. Lisa ran away from home and by the time she was 12, she was selling herself down Aldgate to businessmen on their way home from the City, then she ended up in foster homes and secure units. She was 15 when she met John and got pregnant with their first child, a boy. The social workers didn't like the look of them, but John thought it would be OK: he was 20 then, working as a site supervisor down at Canary Wharf, and they had a room at a bed and breakfast on the Romford Road. But the social workers took the baby away, so that same day Lisa went and found her mate Nicola (she's dead now) and she smoked £50 of heroin in one afternoon. She'd never done it before. She's been doing it ever since. John was soon at it too - he just wanted to know where all his money was going.

Mathew didn't meet them until five years later. By that time, Lisa and John had just had a second son. Lisa was still selling herself, and John was on three kinds of medication for a paranoid obsessive disorder. He was committing crimes every day of the week to fund their heroin habit, so the social workers took this little boy away as well, although in the end Lisa's mum went to court and got custody of both Lisa's kids. Lisa and John sort of drifted apart. Mathew was arguing with his step dad and so he moved out to live with Lisa, and John stayed around.

Now, the three of them spend their days ducking and diving and trying to find gear. Most days, they'll spend £100 down on Bethnal Green Road for the three of them. They can get some of that in benefits, but most of it, they have to hustle, so they have to break the law. Most of the time, nobody knows about it. Then, once in a while, they get caught. Mathew has only been done half a dozen times. The worst one was when he and a mate broke into a car in the West End without noticing there was a police car parked right there in the road, so the cops chased them, and Mathew started driving like a maniac, going through red lights, heading the wrong way up a one-way street until finally the police cornered him on the Embankment. He got 30 days in Wandsworth for that and a driving ban, so he lost his job driving for a haulage firm (Mind you, he never had a real licence in the first place.)

Lisa and John also get away with most of what they do, although Lisa's criminal record fills six pages. She's had fines, community orders, five or six jail sentences. John's been done for theft, burglary, fraud, bits of violence including battering a cop, driving whilst disqualified. He's done 19 prison sentences. Nothing changes. It might have been different if there was real treatment around, if they could get a decent prescription for methadone or diamorphine, instead of the rations and rudeness they get from NHS clincis. Most of the time, no court even tried to help them. So, they still need £100 a day - and, by the time you allow for the prices the fences will pay, that means laying their hands on more like £500 of property. John says it's simple: going without gear is not an option; you do what you have to do; the law doesn't come into it.

It all bent out of shape last summer. Lisa was pregnant. She and Mathew really wanted the baby, but they were afraid the social workers would move in again, so they kept it a bit quiet. Then one day in June, Lisa fainted in the street - she can't afford money for food so she'll often go two or three days at a time without eating - and the hospital told the midwife, and the midwife told the social workers, and so Lisa and Mathew cut a deal with them: they would give up gear, they would sign on at the local NHS drugs clinic for some methadone, and so the social workers would let them keep the baby. They had to wait two weeks to get the methadone. When they got it, they were only allowed 40 mill a day, which wasn't enough. And they were refused injectible methadone, which was bad for Lisa, who is a fixer. But they stuck to the deal more or less. For the sake of their baby. Because they already loved the baby.

He was born in August. They named him after Mathew. They were proud and happy, and the baby was healthy - he wasn't addicted to anything - and they spent just about all day every day in the hospital, like real parents. And after two weeks, the social workers took the baby away from them. It did their heads in. Lisa says it really did Mathew's head in. He thought they had a deal. He just couldn't see the fucking point of doing anything any more. Then, Lisa and John got into a row with the staff at the drugs clinic, and they got banned - except that John was wearing Mathew's hat, so they banned Mathew instead of him. Mathew wasn't that bothered. They just went back down Bethnal Green Road.

And that's how Mathew ended up back in court. Not that they got nicked down there, just that Mathew couldn't see the point of keeping his probation appointments, so he ended up with a letter telling him to turn up at Thames magistrates court. That's a hassle. He's got to find this probation officer with the secret address. But before he does that, he's got to go back down Bethnal Green Road and get their £70 back off the little toe rag that rode off with it this morning.

And then what will this court hearing achieve? At what point in a typical day do Mathew and Lisa and John stop and quietly consider the impact of their behaviour on anybody else or on themselves? At what point do they consider anything at all beyond finding some way to get hold of £100 a day? Born and raised in chaos, they have been punished before by the system, often for good reason, but never changing their behaviour, occasionally becoming much worse. Surviving now in chaos, at what point does the threat of punishment stop them committing a crime?

There are signs of the government changing its track. Following January's Carter report, it is setting up the new National Offender Management System, which is supposed to cut the rise in the prison population. It wants to use new day fines - geared to the income of offenders - and also its Sentencing Guidelines Council to encourage courts to use more fines for petty offenders. The Crown Prosecution Service has plans to extend the use of cautions by adding conditions to them. But these moves are designed to take the pressure off the prisons and the courts. None of them challenges the underlying assumption that punishment is the primary tool of crime reduction.

If they had been born 40 years earlier, Mathew and Lisa and John would have been allowed to get clean drugs from their GP. John would not have lost several fingers and half a lung from blood clots. Lisa would not be suffering from malnutrition in the heart of one of the richest cities in the world. None of them would be involved in stealing something like £500 of property a day. As it is, they can turn to the NHS which will offer them long delays, lots of paperwork, tight rations and lectures about abstinence, or they can come to court to be punished.

The courthouse is busy again when Mathew and Lisa turn up there a week or so later. An ulcerated old man whose boot laces are tied up round the ankles of his trousers, totters around the hall looking for a lawyer. A fog of nicotine rolls out along the ceiling from the smoking area. There are two televisions mounted on the walls: the studio audience are clapping, the caption says "Leave my lover alone". Outside court number three, a baby in a buggy is crying its lungs out.

Mathew has got his act together and found the probation officer who needed to see him, even without an address. And the pre-sentence report is OK. It explains that Mathew missed the two appointments because he was so upset about losing the baby and it recommends another community sentence. Mathew's lawyer is good at his job. Lisa and John sit in the public gallery, while Mathew takes his place in the dock.

The district judge has a dark suit, gold-framed glasses on the bridge of her nose and pearl ear-rings. For two or three minutes, she reads the report from probation. The court is quiet. "Yes," she says finally, in a voice just like the Queen's.

Mathew's lawyer starts: "I think it's worth noting - "

But the judge cuts across him: "He was fortunate, wasn't he, to be given a probation order in the first place?" She seems irritated. The lawyer continues, reciting all the appointments which Mathew did keep before he missed just two. She sits with her eyes shut and then cuts in again: "He was given the opportunity but he doesn't seem to have taken it." The lawyer quotes from the probation report and explains that Mathew has had his baby taken from him. "Because he is a heroin addict," says the judge as though that explained everything. The lawyer keeps making his points, the judge keeps interrupting, Mathew is looking paler than ever. Then suddenly, the judge has heard enough.

"Mr King, will you stand please? You were given the opportunity of doing a community penalty and you chose to lose contact with -"

Now Mathew interupts: "I been in contact with them. I give them my new address. I give them my mobile number. I been down there before this court business started."

"You have had your opportunity. Now you will go to prison." Mathew stares blank at her, as she hits him with it: "Three months on each count."

He turns towards the public gallery, breathless. Lisa is on her feet. She knows in an instant that the prison is not the problem - it's having no gear that is going to hurt him. And she knows he hasn't even got a pack of cigarettes in his pocket to help him get through it: he was never expecting to go to jail. Two security men are closing in on the dock. Lisa leans out of the public gallery, her cigarette pack in her hand, Mathew reaches out to grab them, the security men close in, Mathew stretches quickly, Lisa strains towards him, can't reach, the cigarettes fall on the floor. Lisa goes berserk, barges into the court, shouts that the judge is a scumbag, but the judge has gone. And so has Mathew. Mathew has been brought to justice.

Minutes later, Lisa and John slip away into the streets. They are in a hurry, just like they were when Lisa tugged Mathew away from court a few weeks ago. They have to get down to Bethnal Green Road. On the way, they have to find £100 from somewhere. Justice has been done today. Chaos has encountered order. And what did we achieve?

* At his request, Mathew King's name has been changed.

See below for background statistics

How the courts increased their sentences:
During 2002, all courts gave immediate prison sentences to 111,600 offenders - a record and 53,500 more than in 1992.
During 2002, all courts gave community sentences to 186,500 offenders - a record and 84,100 more than in 1992.

How the magistrates made community sentences more punishing:
In 1992, rehabilitation orders (then known as probation orders) accounted for 41.5% of all community sentences. In 2002, they accounted for only 31% of community sentences.
In 2002, rehabilitation orders accounted for only 2.2% of all magistrates sentences.

How magistrates moved from fines to prisons:
In 1992, magistrates fined 1,074,800 offenders. In 2002, they fined only 894,300.
In 1992, magistrates jailed 10,300 offenders. In 2002, they jailed 26,500.

Outcomes in London magistrates courts:
81.7% of defendants plead guilty;
8.9% are found guilty in their absence;
7% are convicted after summary trial;
2.1% are acquitted after trial;
0.3% are dismissed with no case to answer.

Note: A community rehabilitation order (formerly known as a probation order) requires the offender to stay in regular contact with a probation officer for up to three years. In some cases, the court may also require attendance for treatment or other courses.
A community punishment order (formerly known as a community service order) requires the offender to perform up to 240 hours of unpaid work.

Madness behind bars

The Guardian, December 2004

On the evening of Thursday August 19 this year, a prisoner was locked into his cell in the segregation unit of Wormwood Scrubs prison in west London. This man had been sent to the seg to be held in solitary confinement as a punishment because he had threatened a cellmate. But that was three weeks earlier and, since then, he had settled down and been quite easy to manage. Everything was normal.

On the following morning, August 20, an officer called Dickie Hampson unlocked the door of the the cell, and, without warning, the prisoner pounced on him and stabbed him in the back of the shoulder with a toothbrush which he had sharpened into a rigid blade. Hampson was rushed to hospital where doctors found that the epaulette on his shirt had saved him from the worst of the wound.

Nobody could understand why this prisoner had suddenly turned violent. Immediately after the attack, a group of officers with a riot shield restrained him, handcuffed him and removed him to the 'safe cell', designed to prevent self-harm, while they searched his possessions to make sure he had no other weapon concealed in there. That afternoon, the prisoner was calm again, remorseful and worried about Hampson's condition, and soon he was taken back to his normal cell.

On the next morning, Saturday August 21, a second officer, John Leadley, unlocked the same prisoner to take him to the governor for an adjudication for the assault on Dickie Hampson. As Leadley approached him, the prisoner slipped another sharpened toothbrush out of his shirt sleeve - nobody ever found out where he had got it from - and clawed at Leadley's face: the spike cut down through his eye brow and into his cheek bone, missing his eye ball by less than an inch.

With blood flowing from his face, Leadley too was taken to hospital. The prisoner was moved again to the safe cell. It was several days before the prisoner spoke to the senior officer on the seg, whom he trusted, and explained what had happened: soon after he was locked into his cell on that first night, he said, a black cat had slipped through his window and sat on the bed next to him. This cat had been following him for some time. Now, it handed him a card, the ace of spades, and then dropped down to the floor where it danced for him. There was music. The dance went on for hours, the prisoner watched, and then the cat turned to him to warn him that the next person who came into the cell would be his enemy: the prisoner must kill him. The cat had promised to stay with him to make sure he was all right. The prisoner had started sharpening the toothbrubush.

This man was psychotic, he had already been diagnosed as paranoid schizophrenic. None of the officers who work at Wormwood Scrubs is trained in psychiatry. Yet routinely, they deal with mentally-disordered men. When John Leadley was stabbed in the face, it was the fifth time he had been attacked in three months; every one of his attackers was suffering from a severe mental illness.

There is nothing unusual about Wormwood Scrubs. Every prison in the country now warehouses the mentally disordered: the numbers have been spiralling upwards since the closure of the old asylums. This has reached a point which beggars the imagination: figures from the Office of National Statistics show that, if we diverted to treatment all those prisoners who are mentally disordered and/or addicted to alcohol or drugs, 90% of inmates would no longer be held in jail.

There are now some 75,000 men and women behind bars in this country. The findings of the Office of National Statistics suggest that nearly 50,200 of them have personality disorders; 6,175 are psychotic; and more than 35,000 of them have neurotic disorders. Several tens of thousands of them suffer a combination of disorders. More than 75% of them are intellectually impaired, with IQs below the national average. And these are not figures that the government denies. The prisons minister, Paul Goggins, himself describes these ONS statistics as "our principal source of knowledge about the extent of mental ill health in the prison population." Most mentally disordered people do not end up in prison; but overwhelmingly most prisoners are mentally disordered.

And every one of our prisons suffers from the 'treatment gap' - the gulf between the care that is needed for the mass of mentally disordered men or women within its walls and the care that is actually provided. After years of neglect, when there was no effective mental health care in prisons, the government is finally tackling the crisis, but resources are short, there are real limits to what can be achieved in a prison regime and, worst of all, as the health minister, Dr Stephen Ladyman, told the parliamentary mental health group this year: "It is generally accepted that mental health will deteriorate in prison."

The results are deeply worrying. It is not simply that mentally-disordered prisoners may be violent to staff or other inmates. Indeed, they are far more likely to be violent to themselves. On average: every single week in prisons in England and Wales, two prisoners take their own lives and two or three others are resuscitated after trying; and every day, some forty prisoners cut or gouge themselves in their cells. Beyond this, the fact is that we are herding disordered men, women and children into our prisons, storing them there for months and years, and then processing them back into the community with every prospect that their disorder has not been addressed and so they will offend again. They suffer. Those around them in prison suffer. Everybody suffers.

Every afternoon at Wormwood Scrubs, the white prison vans queue up outside the gate, ferrying men back from the courts - the young and surly, the old and confused, the crackhead who was caught shoplifting yet again, the homeless man who set fire to an empty building, the man who thumped a stranger because he looked at him in a funny way, the old Rasta who screams at himself in the park. The reception officers process them and give them a bedroll, a nurse takes them through a health questionnaire: "Have you ever received treatment from a psychiatrist outside prison? Have you ever received medication for any mental health problems? Have you tried to harm yourself?"

Once, several tens of thousands of mentally-disordered men and women were held behind the high walls of the old asylums, but they failed and, in the late 1980s, they were all closed down. Now, those who would have been their patients rely on a network of community care which is notoriously overstretched and, if they fall through its gaps, they are scooped up by police and sent back behind high walls. Those walls protect a crooked structure.

We now have139 prisons. We are already committed to building five more. By the end of the decade, the Home Office plans to incarcerate at leaste 80,000 men and women. If the Office of National Statistics is right, some 56,000 of them will suffer from at least two types of mental disorder. Yet we have only just over 4,000 secure psychiatric beds, which are already full, and no plans to increase them. The whole structure is crooked, because it is built on a crooked foundation - the emotionally satisfying but deeply ineffective idea that if we hurt these people enough, they will obey the law. Hospitals are built with bricks of care, prisons with the plans of politicians.

The old Rasta who screams in the park was arrested for trying to steal a handbag and for exposing himself in a public place. He is soon processed and taken down to A wing. Nobody realises he has lied about his psychiatric history, nor that he has been arrested, charged and now remanded in custody under a false name. Lots of new prisoners conceal their psychiatric history: they don't want the stigma, they are afraid it will count against them, they have forgotten. On the wing, the old Rasta is soon marking himself out as an odd ball, muttering to himself, eating just about nothing, refusing to wash himself or to clean his cell. It gets so bad that one day the officers force him to take a shower: nobody can stand the smell any more.

By chance, a visiting nurse recognises him from the hospital where she works and remembers his real name. The prison doctors contact the hospital's community mental health team for his notes and persuade the old man to take some medication while they wait for them to arrive. Three weeks later, the notes finally reach Wormwood Scrubs, detailing his long history of schizophrenia, but by that time, he has been taken back to court where he is given bail or a non-custodial sentence - nobody bothers to tell the prison - and he is back out on the streets again.

The number of mentally-disordered men and women who are held in our prisons has increased seven-fold since the old asylums were closed. We have talked to staff at every level of the prison service who are alarmed and depressed to find themselves warehousing the sick. The former director general, Martin Narey, now in charge of probation as well as prisons, has publicly described the strain as 'overwhelming'. A senior manager told us it was 'a bloody awful problem'.

The cutting edge of the government's response has been to create 'inreach teams', to deliver care-in-the-community on prison wings. In Wormwood Scrubs, the team consists of one consultant psychiatrist, one social worker and one community psychiatric nurse. They care for a prison holding some 1,167 men at any one time, with some 3,900 coming and going over an average year - 90% of whom they reckon to be mentally disordered. The simple reality is that the Scrubs team spend so much time assessing new patients that they rarely have time to deliver a care plan.

There is also a new Day Care Centre which borrows money from the education budget to run courses in relaxation, art and acupuncture, but staff are so short that sessions sometimes have to be cancelled. An unpaid counsellor comes in once a week, but there is no psychotherapy at all. There are signs that the centre has helped some of the low-end patients, but the service is limited and helpless to deal with the most severely ill who need beds in outside hospitals, which are extremely hard to find. In a special report on nursing in prisons, the Department of Health itself acknowledged that, while staff may do their best, there is a level of care which "prison health care does not and can not provide".

Even at the lowest end of the scale of mental disorder, this leaves the neurotics with their phobias and anxieties and panic attacks hiding quietly in their cells, not eating and/or not sleeping and/or being punished for low-level disobedience. In the outside world, they might end up on the general ward of a local hospital; here, their disorders frequently pass unnoticed in the muddle of daily life. Sometimes, they are prescribed something to help them; sometimes it is stolen by other prisoners. The parliamentary mental health group has taken evidence on the victimisation of mentally-disordered prisoners who report being robbed, bullied and indecently assaulted.

Others with neurotic disorders are screamingly obvious. A man was shipped into the Scrubs from Highbury magistrates in north London a few months ago. Suffering from depression after the break-up of a relationship, he had slapped a police officer. Within four hours of arriving, he had cut his throat from ear to ear including his jugular and slit both wrists. All the prison could do was to rush him into Hammersmith Hospital to tend his wounds for four days and then beg the Home Office to allow them to transfer him to a local psychiatric ward to treat his anxiety. They refused: the courts had ordered that the man be held in custody awaiting trial and they deemed the security in the local hospital was inadequate.

Some of the psychotics too can be withdrawn - "quietly mad" in the language of doctors - and likely to remain undiagnosed and untreated. To untrained prison officers, they may seem irritating, asking the same question over and over again, or simply weird, like the man who was frightened of water. He would sit on one of the wings at the Scrubs staring at a splash of water on the floor, worrying that it was evaporating too quickly. It was two weeks before he clumped a member of staff, triggering a process that diagnosed his paranoid schizophrenia and eventually transferred him to an outside hospital. (His worry about water was that it was poison and he would die if he drank it.)

Once they are spotted, the psychotics can often be stabilised. There is a young schizophrenic in the Scrubs at the moment: his father lives abroad; his mother has been evicted from her home and is sleeping rough somewhere; he ended up living on the streets, cold, hungry, hallucinating and finally trying to steal a woman's handbag. He is on remand awaiting trial and he is now stable and relatively secure in a single cell with a television. His main worry is that the court may take pity on him and put him out. The big problem with psychotics is that, if they refuse medication, the prison - unlike a hospital - has no right to treat them against their will.

One of the staff at the Scrubs told us about a psychotic man who refused to be treated. The prison wanted to transfer him to an outside hospital, but there was no bed. Without medication, the man's condition started to deteriorate. Soon, he took to standing on the sink in his cell, holding his arms out sideways and swallow-diving head first on to the concrete floor. His face was soon broken and bleeding, and staff were then able to use their limited power under common law to tranquilise him by force for long enough to stitch his wounds. Then the law required them to stand back and let him carry on swallow-diving. At one point, they got special permission from the Home Office to hold him in a padded cell in a strait-jacket (something which officially is no longer done in UK prisons). Finally, an outside hospital bed was found for him.

The most disruptive are those with personality disorders. Technically, they are not suffering from a mental illness, but their behaviour is distorted by traumatic experience, usually in childhood: one out of every three men who is remanded into custody by our courts has been in care as a child; one in three women has been sexually abused; one in ten men has been sexually abused. They may be withdrawn, mistrustful, aggressive, anti-social. At the top end of the scale, they may be grossly callous and cruel. In prison cells, they cut themselves, they bang their heads against the wall, they manipulate, they protest, they swallow pills like sweets. But, from the doctors' point of view, although they may be mentally disordered, they are not 'ill' and are often dismissed as untreatable. The daily strain of the treatment gap, dealing with so much mental disorder without the resources or skills to match the challenge, produces some real tensions in the prison.

On Saturday August 21 - the same day that John Leadley was stabbed in the face - there was another prisoner on the segregation unit who was causing trouble. He had been moved there from B wing, where he had been cutting himself, and now he was standing on the water pipes of his cell with a ligature around his neck, threatening to hang himself. Just a fortnight earlier, a prisoner had succeeded in committing suicide in the seg, so the senior officer there was in no doubt about what he wanted: this man had to be transferred immediately to the prison's health care centre, where there are seventeen cells for acutely ill prisoners. He called the centre, and one of the doctors came down.

To the senior officer's horror, the doctor refused to take the prisoner. He said the man was suffering from a personality disorder, which was not treatable, and that this behaviour was simply a self-harming gesture. The senior officer became agitated, reminding the doctor that two weeks earlier, they had cut down a body which had then laid in the cell all day while the investigators did their work: "I'm not going to engage in semantics with you. Suppose it happens again. You just can't take the chance." The doctor stood his ground. The senior officer reluctantly backed down but told the doctor: "If this goes wrong, I guarantee you I will stand up in coroners court and point the finger at you". Then he recorded all that had happened in the seg's observations book.

In the event, the man did not hang himself. The friction passed. But the problem remains. Just like the officers in the seg, the staff in the health care centre see themselves struggling to deal with an impossible problem. Apart from the shortage of specialist psychiatric help, the prison has12 unfilled vacancies for general nurses, leaving only 28 to cover the whole prison, day and night, for all forms of illness. Sometimes, they try to fill some of the gaps with agency nurses, but they are expensive and can wait up to a month for security clearance. They have only 17 in-patient beds for the whole prison. And, running through all this, they are medical staff working in a prison which has different priorities and understandings and even language.

Are these men prisoners or patients? Who should win the argument if prison officers want to search a man's cell but health staff say it will aggravate his mental disorder? What should happen when health staff want half-a-dozen prisoners out of their cells for some group activity, and security staff say they cannot do that without three officers - who are not available? How can prison officers be expected to understand the mental disorders of the men they are looking after, if generally they have no psychiatric training and are not even allowed to read their medical notes? How can health staff provide regular care when their patients are suddenly snatched from their surgeries and sessions because there has been an incident and prison officers are locking down the wing?

The diagnosis of mental disorder is difficult - and made more difficult by the possibility of prisoners trying to manipulate the system. Certainly, there is some manipulation: a prisoner at the Scrubs this summer repeatedly evaded his court appearances, because he knew that if he got into the prison van and cut himself - even superficially - the security company would refuse to transport him. It was a prison nurse - not an officer - with years of experience who told us that some prisoners manipulate the regime for preventing suicides: "Let them hurt themselves, let them cut themselves, it's their responsibility. I have stood outside a cell and watched an inmate tear up his clothes and make a noose and put it round his neck; And I've said 'Yeah? Go on then, do it.'"

The bottom line is that, with difficult patients, poor resources, and the culture clash between security and care, prison staff can be stretched to breaking point by some of the people in their care. Look at what happened in September with Terry Moreton. He has a history of violent crime, including a serious attack on a teenaged girl at a railway station, but he was in the Scrubs awaiting trial for a fairly minor offence. The Inreach team had spotted him and diagnosed him as suffering from bipolar illness (manic depression) as well as personality disorder and they had given him some medication. But he had stopped taking it. For several days, he had been thumping his head against his cell wall in B wing, and officers had opened an F2052SH form, which is kept for any prisoner who is deemed to be at risk of self-harm. That Friday night, Moreton ran out of tobacco, threatened to kill himself if he was not given some, assaulted his cell mate and barricaded the cell.

During the night, officers managed to get in and took him down to the seg. On Saturday morning, the senior officer came in and said Moreton did not belong there: only 48 hours earlier, the doctors had said his illness was so bad that he was not fit for segregation, and there had been no doctor on duty the previous night to 'fit' him. He called health care and, before long, a doctor came along with the duty governor. By now, Moreton had sharpened the end of a metal flask, which is used to give the prisoners hot water, and was threatening to slash anybody who came into his cell. The duty governor wanted him restrained. The seg officers said they could not do that: if he was not deemed fit for the seg, they would be guilty of assault if they restrained him. The doctor suggested he could now fit him, and so the officers went in.

Moreton was taken to the seg's safe cell and strip-searched. The doctor then suggested that he might not be fit for seg after all, which was bad news for the seg officers if it meant that they had committed an assault, but good news if it meant they got rid of him. Late that Saturday, Moreton was finally taken to the in-patient health care cells where he proceeded to tear the plastic lid off the toilet, break it into shards, wrap one of them in a torn shirt and threaten to slash anybody who came near him. He then smashed his sink, broke the water pipes and flooded the whole health care unit in two inches of water. The health staff now said he was, in fact, fit for seg and sent him back.

The following morning, an officer outside the seg saw him sharpening his breakfast spoon on the window. The senior officer persuaded him to hand it over. That night, Moreton made a dummy out of his bedding, hung it from the window and hid himself under the bed, apparently hoping to ambush any officer who came in. He spent the next day literally gibbering in his cell, while those in neighbouring cells threatened to kill him, because he had now kept them awake for three nights.

Seg officers were fed up with the health care staff. They felt their unit was being used as a dumping ground, that a solitary cell was the worst possible place for somebody who was mentally ill, that it was absurd that the man could be unfit for seg one moment and fit the next. Health staff were just as fed up. They, too, felt that their unit was being used as a dumping ground and that, although Moreton had a history of bipolar illness, he also suffered from a personality disorder - which was not treatable - and that it was the disorder, not the illness, which lay behind his behaviour. They explained that he lurched between being fit and unfit for seg simply according to whether or not he was taking his medication. And both groups felt they suffered from decisions made by night staff and weekend staff who were not equipped to make these decisions. The truth, >>> Of course, is that they are both right: they are both being used as a dumping ground; the whole prison system is.

For years nobody really bothered about the mental health of prisoners. Only four years ago, research published by the British Medical Journal found: no doctors who were in charge of prison inpatients had psychiatric training; only 24% of prison nurses had mental health training; patients were locked up for between 13 and 20 hours per day; services for the mentally ill in prisons fell far below standards in the NHS; patients lives were restricted and access to therapy limited.

In the last few years, as the problem has grown into a crisis, the prison service has started moving the mountain. Three years ago, they published a joint report with the department of health, Changing the Outlook, which frankly admitted: "There are too many prisoners in too many prisons who, despite the best efforts of committed prison health care and NHS staff, receive no treatment, or inappropriate treatment for their mental illness, from staff with the wrong mix of skills and in the wrong kind of setting." The prison service adopted the 'principle of equivalence', that prisoners should have the same health care as any other NHS patient, and took the historic step of handing over the commissioning to local Primary Care Trusts.

Since then, they have funded 300 community psychiatric nurses to work on Inreach teams, like the one in Wormwood Scrubs, providing a care-in-the-community service for prisoners; adopted the Care Programme Approach which should link their work with community teams before and after a prison sentence; and set up new screening for prisoners on their first night to do a better job of identifying mental disorder (the old system was found to be missing 75% of cases.) But can a prison pretend to be a hospital?

Some of the culture clash between care and custody can be dealt with. There is a new programme to teach prison officers how to spot the symptoms of mental disorder. In principle, the rules may be changed to allow officers to read some of the medical notes of their inmates, which are currently hidden from them on grounds of confidentiality. There is some hope that the new NHS computer system will cut the number of prisoners who turn up with no medical notes.

But there are other problems which are harder to reduce.The Mental Health Commission last year warned of the limits of community care in a prison: "The prison 'community' cannot offer any real equivalent to the support and care available outside prison, and any assumed equivalence between prison and the community outside greatly under-estimates the isolation and bullying of the mentally ill in prison and the stigma of mental illness in such a situation."

There are limits also to what can be done with medication in prison, because prisons cannot treat inmates against their will, which means that those who reject their medication can deteriorate rapidly. But, with few exceptions, we have found prison health staff are deeply opposed to allowing compulsory treatment in an institution which is not a genuine hospital: the risk of accident and the temptation for disciplinary prescribing are too great. The Mental Health Commission last year said it was "deeply concerned" about the "extremely serious risk" of unrecognised or unchallenged coercive treatment in prisons.

Beyond that, it is hard to see how prisons will find the cash to reverse years of neglect, and there is a real fear among governors that the Treasury have stored up trouble by failing to ring-fence future funding for prison health, leaving Primary Care Trusts free to divert the money to patients who are more popular than offenders. The treatment gap will not be closed, and yet the problem will persist.

Prisons are the wastepipe down which other institutions send their rejects. The flow of mentally-disordered men and women into custody is being pumped not simply by the failure of the Department of Health to organise effective care in the community. Within the criminal justice system, the police and the courts have special systems to divert the mentally-disordered. They don't work.

Police can remove them to a place of safety for 72 hours for assessment - but only if they are in a public place and not if they are being charged in the police station, and only if their illness is recognised by the officers and not if it involves drug abuse, and only if they are in immediate need of care and control and not if they are merely ill, and only if the police have time to go through the procedure rather than simply processing them for court.

The courts, too, have powers to send people to prison or hospital to be assessed - but only if they are told that a bed is available. In 1990/1, all the courts in England and Wales between them used this power only 412 times. Since then, despite the massive increase in the charging of mentally-disordered people, they have used it even less: in 2000/1, they sent only 168 defendants to be checked for mental illness. Two years ago, the Home Office studied its court diversion schemes and found that "many schemes are currently ineffective" - patchy in geographical availability, peripheral to local psychiatric services, poorly designed, "inadequately supported" by local hospitals and "unpopular with local psychiatrists." Juliet Lyon, director of the Prison Reform Trust, earlier this year said the network of 136 schemes had "virtually fallen apart".

So, at a prison like Wormwood Scrubs, they can find themselves dealing with a man like Manuel York who was agitated, unpredictable and hallucinating sounds, visions and physical sensations. A visiting psychiatrist at the prison immediately recognised that he was profoundly disturbed and diagnosed hebephrenic schizophrenia, a particularly disorganised version of the illness. And yet that man, who had been arrested for making unwanted phone calls to a woman, had passed straight through the police and the courts without anybody stopping to ask if he was ill, let alone diverting him for treatment.

And the failure is repeated when it comes to their release. There was a man at the Scrubs recently who tried to hang himself three times in a week; he also bit, punched and spat at staff. Then he left the prison - and nobody knows where he has gone. Research two years ago found that 96% of mentally-disordered prisoners were put back into the community without supported housing, including 80% of those who had committed the most serious offences; more than 75% had been given no appointment with outside carers. And yet Department of Health policy requires all mentally-disordered prisoners to be given a care plan on release.

When the Home Office researched mentally-disordered defendants in court, they found that: "The majority were not career criminals who had become mentally ill. Most appear to have offended in the context of mental illness and social exclusion, having fallen through gaps in community care." When the Home Office recently reviewed the care of those men and women in prison, the researchers concluded by quoting an earlier survey: "It seems that mentally ill offenders will be as much at risk from society, as they will be a risk to society." The white prison vans still roll up outside the prison gates.

* To protect medical confidentiality, names and some identifying details of prisoners have been changed

Additional research by Roxanne Escobales

See below statistics of mental disorder in prison

Mental disorder in prisons -

Source: Office of National Statistics

Suicide and self-harm among prisoners -

Source: Office of National Statistics

Mental disorder includes
* mental illness such as psychosis or severe depression, often occurring as an episode in an otherwise healthy person and liable to respond to treatment;
* personality disorder, such as antisocial or paranoid, occuring as a continuing pattern of abnormal behaviour, sometimes the result of childhood experiences, generally difficult to reverse;
* neurotic disorder, such as anxiety and phobias, occurring at a level likely to interfere with normal activity, generally amenable to treatment;
* learning disabilities, usually involving significantly impaired intellectual functioning.

The most dangerous prisoner in Britain

The Guardian, December 2004

In the second part of his investigation into mentally-disordered prisoners, Nick Davies tells the story of a prisoner who has become a living nightmare, one of the several thousand severely ill held behind bars and denied a hospital bed.

We do not need to know exactly what happened to Glenn Wright when he was a child. It is enough to say that he suffered most painfully at the hands of an adult who forced him to live through nightmares. Before that, he was just an ordinary boy, growing up in Northampton in the 1970s, his father a supervisor at the Carlsberg brewery, his mother looking after the five children, of whom he was the youngest. Afterwards, he was himself a nightmare in the making.

Glenn Wright is not at all well-known to the outside world, but inside the prison system, he is notorious, and he is sometimes described by governors as the most difficult prisoner in the country. Even from a maximum security cell, he exerts enormous power over those around him. He does this most often by attacking his own body as a form of blackmail. He swallows razor blades and screws. He eats batteries. For years, he has kept open a savage wound in his leg; sometimes he puts his own shit into it. He pushes taps and broken porcelain into his rectum. He went through a phase of keeping a cup of blood in his cell to throw at anybody who tried to come too close.

Occasionally, in his campaign to control those he comes into contact with, he embroils others in his theatre of horror. He told a prison pyschiatrist that he would like to kill her, slice her, fry her and eat her. He has talked about murdering people and feeding them to dogs or to their own families. He has killed other prisoners without touching them: he talked three of them into hanging themselves; one recovered, and two of them died, and he was given life for murder and aiding suicide.

For the psychiatrists who have examined him, Glenn Paul Wright, now aged 32, is suffering from a severe personality disorder, the legacy of the relentless torment which he suffered as a child. The disorder pitched him into a life of offending, he ended up in prison, where he remains trapped in a regime which cannot treat him effectively. He is locked up alone for 23 hours a day. If he leaves his cell, he has both hands ratchet-cuffed to a prison officer with four others in escort; he has the option of exercise in a fenced cage. There have been times when, as a deliberate policy, he has been denied all access to psychiatric care. Routinely, he has been shifted from one high-security prison to another, even though prison psychiatrists have warned that this aggravates his condition.

The one solution to the problem - both for Wright and for the prison service - is that he might be tranferred to a high-security hospital for treatment. Five different forensic psychiatrists have assessed Wright and recommended that he be moved to a secure psychiatric bed. And yet, he remains in prison.

It is government policy that all patients be given "timely access to an appropriate bed". And that specifically includes prisoners. That policy is wrecked. There are thousands of severely mentally-disordered prisoners who are being denied transfer to outside hospitals. When the chief inspector of prisons' medical expert studied a sample of inmates in local prison hospital wings two years ago, he found 40% of them were not just mentally ill, but so ill that they should have been sectioned under the Mental Health Act.

The problem is two-fold: first, that the Department of Health has failed consistently to provide anything like enough secure psychiatric beds; and second, that many NHS psychiatrists simply do not want disruptive patients on their wards. Even the new units which the government has created for the most dangerous prisoners with severe personality disorders have refused to take Glenn Wright.

Dr Adrian Grounds, a forensic psychiatrist who works in prisons and who is also a senior lecturer at the Institute of Criminology at Cambridge University, estimates that there may be up to 3,700 prisoners who are so severely mentally disordered that they should be in a psychiatric hospital but for whom there is no bed. And where beds are available, delays are commonplace, and prisons are replete with stories of hospital psychiatrists ducking and weaving to avoid taking the most difficult prisoners.

One of the most senior forensic psychiatrists in the country told us: "Psychiatric teams come into prisons and make the most amazing judgments. I had a man in custody who was grossly manic, naked, occasionally smearing himself in his faeces and spending 22 hours a day in seclusion. He needed to be in a safe environment. But the visiting psychiatric team said: 'No, no, you're doing very well with him, he's fine here.'"

Glenn Wright drifted easily from a boyhood of pain to an adolescence of crime. By the time he was eleven, he was regularly truanting from primary school. By the time he was 13, he was expelled from secondary school and appearing in court for burglary and theft. He was recommended for a special school; it never happened. By the time he was 15, he was sniffing glue, smoking dope, swallowing blackmarket pills and serving his first sentence at Glen Parva Young Offenders Institute. He told psychiatrists he was hearing voices; nothing was done. When he was 19, he served his third sentence at Glen Parva. Since then - a period of 13 years - he has spent all but seven months in jail and has little prospect of release.

At the beginning of those 13 years, he was just another mentally-disordered offender. In Glen Parva, he was beaten up by other prisoners and swallowed some razor blades. He was transferred to Feltham, where he cut himself and was put on the hospital wing. The doctors said he had an anti-social personality disorder with impulsive acts of deliberate self-harm in the context of abnormal personality traits. He took to slashing his arms, cutting his wrists and swallowing more razor blades.

A couple of times, he was released, utterly unchanged in his behaviour. Rapidly, each time, he went back inside. The last time was in June 1996 when, aged 24, he was given three years for burglary. At the police station, he slashed his wrists. A few months later, he reported being assaulted and seriously harmed by other prisoners at Woodhill. Now, his self-harm started to take off: he swallowed screws, smashed up his cell, strangled himself, removed the tap from his cell sink and buried it in his rectum, tore a deep wound in his leg, ripped bits of fixtures and fittings from the cell walls and inserted them into his leg wound. Then his behaviour spilled over onto those around him.

On November 16 1996, a few weeks after he reported being seriously assaulted, he persuaded his cell mate at Woodhill, William Scott, to hang himself. On February 8 1997, still in Woodhill, he persuaded another cell mate, Peter Karelius Smith, to do the same. On January 27 1998, in Pentonville Prison, he persuaded Kenneth Cross to swallow an overdose of tablets and hang himself with a torn sheet; the noose broke, and Cross survived. As a result of what Cross told the authorities, Wright was prosecuted for the incidents with all three cell mates.

Thus far, there was no sign at any time of the prison service providing any effective treatment for his disorder, which was clearly becoming worse. That is not unusual: as we described yesterday, the mental disorders of prisoners were ignored for years; and even now, when the prison service has started to tackle the problem, treatment in prison remains deeply inadequate. And even those with the most serious disorders - those who would be sectioned and sent to secure hospitals immediately if they were in the community - are trapped in a system that cannot help them.

The chief inspector of prisons, Anne Owers, personally came across a man who was strapped into a body belt with his wrists clamped to his waist. When she asked about him, she was horrified to be told that he was kept like that permanently: it was the only thing the prison could do to stop him castrating himelf. One of her inspectors witnessed a prisoner gouging out his own eye ball - he had started on the other eye before officers managed to restrain him.

With their move to an outside hospital either blocked or delayed, the condition of many severely disordered prisoners deteriorates. The Prison Service is currently being sued by the family of a man who was stored on a hospital wing, awaiting a long slow transfer, and who tried to kill himself by hanging. He failed and, a week later, he tried again. Once more he failed. Two weeks later, he tried a third time and, by the time he was discovered and saved, he had been hanging long enough to suffer apparently permanent brain damage.

When he was charged with murdering his two cell-mates and attempting to murder a third, Glenn Wright finally forced the system to confront his condition. In the run-up to his trial, he was visited in prison and assessed by two independent forensic psychiatrists. Both agreed that he now had a serious psychopathic disorder and that he met the criteria for admission to a special hospital. On April 30 1999, Wright was given a life sentence at the Old Bailey and waited to be transferred to a secure psychiatric bed.

A year later, he was still in prison, untreated, still attacking his body and now covering himself and his cell in his own dirt. He waited like that for two years before finally, on November 7 2000, he was moved to Broadmoor.

The government says it has introduced "quicker and more effective arrangements for transferring the most seriously ill prisoners to appropriate NHS facilities" and claims that, on average, at any one time there are only 40 prisoners waiting longer than the official target of three months to be transferred to a secure mental hospital. The truth is more complicated than that. First, any patient in the community who was considered so mentally disordered that they had to be sectioned would be moved to a psychiatric bed within hours. A delay of a week would be extraordinary, and so a target of three months is scarcely effective.

Second, the government is dealing in averages, which disguise the long delay experienced by some prisoners. And the disguise appears to be deliberate. The Department of Health are refusing to release the true figures even though they have them and even though they relate to a government target for which they are accountable to the public. We spent more than two months asking for them: the department finally replied by claiming that the figures are "management information, not routinely published".

Third, prison staff report that outside doctors will fiddle the system by stalling their assessment until they have a bed available so that the true length of the delay is disguised. Finally, psychiatrists who work in prisons tell us that the process of asking for a transfer is so time-consuming and so unlikely to succeed that they begin it only for the most extreme cases, leaving a long queue of severely mentally-disordered prisoners who ought to be transferred but who do not even get to the starting line (or show up in the government's statistics).

There is a man in a Midlands prison now, for example, who is below average intelligence, diagnosed psychotic and hearing voices, frequently cutting himself, his condition exacerbated by being held in prison. The consultant psychiatrist who works part-time at the prison says he should be in hospital, but he is not dangerous enough for a high-security bed, not sick enough for a medium-security bed and too likely to abscond for a bed on a local locked ward.

The Department of Health's refusal to provide enough secure pyschiatric beds is like a boulder in the doorway that should lead the very ill from prison to hospital (just as their failure to provide effective community treatment diverts so many of them to prison in the first place). This summer, Dr Adrian Grounds of the Institute of Criminology told a Cambridge conference that the shortage of secure beds was so grave that: "Overall, the picture was of a system that was under such pressure that it was almost seized up."

Dr Grounds referred back to research in the early 1990s by professors John Gunn and Tony Maden, who analysed samples of prisoners to discover how many were in clinical need of transfer to an NHS psychiatric bed. Acknowledging that this was "fraught with difficulties and uncertainties", Dr Grounds applied their findings to today's much larger prison population and concluded that there may now be up to 3,660 men and woman who are so severely ill that they should be in a psychiatric ward but who are trapped in prison for want of a bed. Most of these men and women will not even have been assesed for transfer. Of those who are, we are told that at the moment there are some 300 men and women who have actually been sectioned but are still stuck in prison cells. Dr Grounds said: "The policy aim of diverting the mentally disordered from prison to the care of health and social services has chronically failed."

The Department of Health refused to comment on Dr Grounds' findings even though we gave them three months to do so. They said it was no longer their job to provide secure beds - it was a matter for the local health trusts. If Dr Grounds is right, up to 3,700 prisoners are blocked from appropriate beds - and yet, according to the latest available figures, in 2002/3 we transferred only 219 sentenced prisoners to psychiatric hospitals - and 45% of them (99) were sent back again. Which is what happened to Glenn Wright after he finally made it to Broadmoor in November 2000.

He lasted only four months. During that time, he threatened staff, incited other patients to refuse their medication and was put into seclusion where he tore panels from the wall and covered himself in faeces. The doctors at Broadmoor did not dispute the diagnosis which had brought him there but told the Home Office that he could not be managed safely in their high-security hospital. In March 2001, he was taken back to prison.

Over the following twelve months, he was shuttled from one high-security unit to another. This is a strategy which is known among prison governors as 'sale or return': when a prisoner is difficult, the governor ships him off to another prison on the understanding that, if he acts up there too, the second prison can send him back. The inspector of prisons, Anne Owers, earlier this year told the parliamentary joint committee on human rights: "You have a whole category of prisoners who are basically on a merry-go-round. That is likely, I would have thought, to exacerbate their mental condition." She said she knew of one prisoner who had been transferred 30 times in two years and who was now dead. After he was removed from Broadmoor, Glenn Wright was moved seven times in twelve months from one maximum-security unit to another, finally bouncing from Long Lartin to Woodhill, back to Long Lartin, then back to Woodhill again.

His attacks on himself became frantic. In Full Sutton, he swallowed three batteries and was taken to general hospital to have them removed. In Whitemoor, he cut his arms and covered himself in shit. In Long Lartin, he kept a razor blade under his foreskin and put a tap up his backside. In Woodhill, he buried a pen and more shit in the open wound in his leg and put another tap up inside him. En route to one of his stays in Long Lartin, he produced a razor blade, cut up his clothes to make a noose and started strangling himself. When he finally arrived at the prison, he produced another blade, made deep cuts in his arms and legs and refused to have them treated. While this was going on, the Home Office and Rampton high-security hospital refused even to assess him and said he could have hospital treatment only if he improved his behaviour.

It is not just the shortage of beds which keeps mentally-disordered prisoners out of secure hospitals. There is clear evidence that the NHS discriminates against them. In the ten years between 1992 and 2002, the number of men and women sectioned to secure beds increased by 24% - but the number sectioned from the courts and the prisons fell by 24% (and this was while the prison population was rocketing). As the Department of Health itself admitted in a policy paper three years ago: "In some cases, it seems that the NHS does not always give a prisoner patient the same level of priority as they would have in the community."

At its worst, this sees NHS psychiatrists occasionally resorting to apparently cynical misdiagnosis to justify rejecting difficult patients. The mental health group MIND told the parliamentary joint committee on human rights in January of a mentally-ill woman who succeeded in being transferred from prison: "She got into the ward and was apparently more trouble than the ward anticipated, so she was hastily reclassified as having an untreatable personality disorder and booted out that same night. One wonders whether this was on medical or management grounds."

There is clear evidence that, under pressure of a gross shortage of places, hospital doctors may cherrypick the easiest patients. Researchers in 1998 studied 44 women from Holloway prison who had been referred to secure hospitals. Half were turned down, and the researchers found that, compared to those who were allowed to have beds, the rejected women were more likely to have harmed themselves, to have suffered childhood abuse, to have committed serious offences and to be seen as violent or dangerous. The research concluded that their rejection was the result not simply of difficulties in treating them but also of "inadequate service provision".

Further evidence of distorted judgements by NHS psychiatrists was uncovered by Home Office researchers two years ago who studied what actually happened when mentally-disordered offenders were transferred by courts to secure hospitals. They found that in reality, they were less likely than other patients to be violent on the ward or to re-offend after treatment: "The perception amongst some psychiatrists that admissions from court gain little benefit from admission, are more disruptive in hospital, fail to engage in follow-up, and rapidly offend upon discharge is almost entirely false."

NHS psychiatrists who turn their back on prisoners are aided by the fact that their hospitals have the right to refuse admission: prisons have no such power to refuse to hold an inmate, no matter how inappropriate. A senior forensic psychiatrist told us: "Putting mentally disordered offenders into prison is hugely ineffective. There is no argument against the moral point of view that they should be transferred. That is very strong. It is not just that the faciliities are not there. The whole ethos of psychiatry is against it."

Since Glenn Wright was returned to prison from Broadmoor, in March 2001, three more consultant psychiatrists have assessed him and deemed him suitable for transfer to hospital. Yet, he not only remains in prison but has, at times, been deliberately denied psychiatric care. He was held at Long Lartin for two months in early 2002, when the prison said explicitly that his regime would be one of containment and not treatment. Two years later, he was back in Long Lartin, where staff were instructed that he must be held in solitary confinement, exercised in isolation and denied access to a psychologist.

His self-harm now has become an explicit protest at the lack of treatment
for his mental disorder. Ironically, he has then lacked treatment for the physical damage he has caused himself. In March 2002, the general hospital at Milton Keynes, which serves Woodhill Prison, refused to remove jagged porcelain and batteries from his rectum, having already performed the operation twice before. In September 2002 and again in March 2004, the wound in his leg was untreated and said to be producing an unbearable smell. In some prisons, he is allowed to see a doctor only in the presence of six prison officers, leading him to refuse to have objects removed from his penis and backside. On other occasions, as part of his protest, he himself has formally refused to be treated.

The prison service reacted by continuing to shuttle him between prisons, despite the evidence that this was making his condition worse. In the second half of 2002, they moved him six times in seven months. Finally, in January 2003, he came to rest at Woodhill, where, for the first time in all his years in prison, he was given a full assessment and a care plan. He was given weekly meetings with a psychologist, and sessions in dealing with stress and self-harm. By April, he was showing signs of improvement and his meetings with the psychologist, which had originally been conducted through the hatch in his cell door, were held at a table.

Yet, despite this improvement, the prison service started to shuttle him again. In one prison, he claimed that officers in riot gear punched him and slammed his head against the floor. Another prisoner who was on a segregation unit with him, reported that officers were banging on Wright's cell door during the night to keep him awake and telling him: "As long as you are as you are, you're getting fuck all." The shuttling climaxed in a move to Long Lartin in March of this year, when the Woodhill psychiatrists complained bitterly that they had not been consulted and that the movement was causing his mental condition to deteriorate. Wright reacted in Long Lartin by putting shards of broken glass, porcelain and wood into his body, re-opening his leg wound and covering his body and his cell in his own dirt.

During this time, the Woodhill psychiatrists have tried repeatedly to transfer him to Rampton special hospital. In February 2002, Rampton agreed to assess him but kept him for only 36 hours before saying they could not help him. In November 2003, after his improvement at Woodhill, Rampton were sent three reports by senior forensic psychiatrists who agreed that he would respond to hospital treatment, but Rampton turned him down again without even seeing him.

Earlier this year, Woodhill applied to transfer him to Rampton's new unit for dangerous and severe personality disorders, one of four such units which are being set up by the government apparently to treat prisoners of exactly Wright's kind. This time, the Rampton admissions panel agreed that he had a personality disorder, noted that he had expressed motivation to be treated, acknowledged that he had been treated effectively at Woodhill but concluded that it would be "not appropriate" to give him a bed in the new unit.

Part of the difficulty here is that Wright is suffering from a personality disorder, ie behaviour distorted by earlier experience, and this is not regarded as an illness like schizophrenia. NHS psychiatrists often argue that personality disorder is not treatable and that it would be wrong to fill a scarce bed with somebody who cannot profit from it. However, we have spoken to senior forensic psychiatrists who say that this is too narrow an interpretation of treatment, that hospitals will take in a man who has been paralysed in a car crash, even though his condition cannot be cured, in order to help him manage his condition. The same logic, they say, should be applied to those who are mentally disabled and so a man like Glenn Wright should be given the systems and therapies that will help him to manage his condition, even if he cannot finally be freed from it.

Wright's disorder may be extreme, but the trap of 'untreatability' catches most of the estimated 50,200 men and women who have a personality disorder and who have ended up in prison cells. A survey of health trusts two years ago, revealed that only 17% of them were providing a dedicated service for personality-disordered patients, and there is only one secure residential unit in the whole of the NHS - Arnold Lodge in Leicester - which provides forensic beds reserved for them. Some units reject them as a matter of policy; others will admit them only in very small numbers; many also reject patients who have drug problems, which is commonly part of life for offenders with personality disorders.

Now, the government is proposing to change the law so that those who have disordered personalities can no longer be rejected as untreatable. Clearly, there is a benign argument for this, but there is also a real danger if the new law is not backed up by a regime of effective treatment. Without that, those who suffer from these disorders may be sectioned to secure hospitals and simply stored indefinitely even though they have not been convicted of any crime. The legal right to treatment makes sense only if there really is treatment on offer.

But the underlying problem for prisoners like Glenn Wright is that they are being managed by a system which insists on seeing them primarily as offenders who need punishment rather than as patients who need care. There was a poignant example of this in the case of a nineteen-year-old woman called Petra Blanksby. She was charged last year with arson with intent to endanger life and sent to Newhall Prison in Wakefield. But the life she had intended to endanger was her own: she had set fire to herself, and the flames had spread to her room. She had a long history of trying to hurt herself on more than 90 different occasions. Yet the system treated her as an offender and held her on remand in custody for 130 days. She hanged herself in her cell.

There are senior psychiatrists who are urging the government to mount a double-barrelled attack on the problem, by embarking on a major programme of investment in supported housing, employment and care in the community for mentally-disordered men and women; and by supplying the secure NHS facilities which would make sense of their claim to want to provide an appropriate bed for all mentally-disordered prisoners. These psychiatrists say the investment would yield a clear cost-benefit, particularly in crime-reduction.

In the meantime, tens of thousands of mentally-disordered men and women remain in prison, often deprived of the treatment they need. And several thousand of them, like Glenn Wright, are so severely disordered that they need hospital treatment. Once, in their struggle to contain him, officers locked Wright into a body belt in his cell. Within minutes, he had dislocated his own hip and wriggled free. The escape to hospital is much harder.

Additional research by Roxanne Escobales

See below for statistics and other case histories

NHS secure psychiatric beds -

* Institute of Criminology, Cambridge

Patrick Morley

Aged: 39.

Convicted: of rape, sentenced to life.

Diagnosis: psycopathic disorder.

In January 2000, after several years in prison, Morley was transferred to Rampton high-security hospital. There, he took part in a sex offenders programme, and, in January 2002, an independent psychiatrist reported that he was responding well and warned that he should not be returned to prison: "I have little doubt that if Patrick is returned to a purely custodial setting in which there was no understanding of his deficits and no capacity to work with him, the risk of repetition of his former anti-social behaviour on returning to the community, would be considerable."

Nevertheless, three months later, Morley was sent back to prison. A new doctor had taken over his case and told the Home Office that Morley had a bad attitude, was not responding to treatment and that he presented 'significant management problems'. Morley had been accused of sexually assaulting another patient, although no charge was brought. On the same day that they received this report, the Home Office agreed to move him.

In the background, two consultant psychologists and four trainees who had worked with Morley at Rampton objected to the move, warning that without this work, he was likely to commit more serious sexual offences. Another independent psychiatrist then reviewed the case and concluded: "I do not understand the rationale for prematurely curtailing Mr Morley's treatment at Rampton only for him to commence a similar programme in prison, but one in which the skills normally found in a hospital multidisciplinary team will be lacking." He added that evidence suggested that premature departure from sex offenders treatment programme actually increases risk of reoffending.

But the Home Office stuck to their decision. Morley's lawyers, Bindman and Co, went to court to complain, but the judges ruled that the decision of the doctor in charge was final, as long as it was fair and rational, and that the disagreement of others made no difference. Morley remains in a high-security prison unit.

Gary Frank*

Aged: 32.

Convicted: of rape, sentenced to seven years.

Diagnosis: severe mental impairment.

Gary Frank drove a car in which another man abducted and then raped a young woman. He was accused of knowingly assisting the attack. He was arrested and interviewed at length by police, prosecuted and tried, jailed and held in three different prisons without any of the police, probation officers, prison officers or prison nurses and doctors recognising that he was so mentally impaired that his IQ was in the lowest 0.1% of the population.

He could not remember his own birthdate nor count money nor tell the time. In prison, they recorded his condition: "Very distressed, not coping... Crying a lot... Not eating... Sitting on a chair in the dark... Says he wants to use a razor on his arms... Missing mum... A broken spirit, tearful." At Christmas, he asked if he could go home. The prison officers could see there was something wrong with him and took the highly unusual step of allowing him to sleep with his cell door open, and yet nobody diagnosed his severe mental impairment and nobody offered him any effective treatment for his depression, insomnia and anxiety.

His lawyers sent in a clinical psychologist who reported in some detail that he had "a very low level of intellectual functioning" and added that he was very distressed and not coping. The lawyers sent the report straight to the prison. Still he was offered no treatment, no move to the health care centre and no assessment to be transferred to an outside hospital.

Action was finally taken when a visiting consultant psychiatrist saw him, recognised his extraordinarily low IQ and severe depression and concluded that he was "one of the most ill-placed people I have seen in prison in 20 years of forensic psychiatric practice". Even then, the nearest secure hospital refused to take him, and the visiting psychiatrist had to spend nearly four months making phone calls and writing begging letters before somebody finally provided a bed in a secure unit. The court of appeal subsequently ruled that his statements to police and to his original trial were wholly unreliable, and overturned his conviction. Gary Frank said: "No wonder people commit suicide."

* Name changed at the request of the prisoner

Chris Edwards

Aged: 30

Arrested for: breach of the peace.

Diagnosis: mild psychosis

Chris Edwards was arrested for making inappropriate remarks to young women in the street. He had a history of psychiatric problems - some hallucinations, religious delusions - and he had been prescribed an anti-psychotic drug in the past. The police called a social worker to the cells where he was beating on the walls and removing his clothes and yet, despite all that, instead of using the 'place of safety' law to send him to hospital to be assessed, they charged him with breach of the peace and, for this minor offence, he was remanded to Chelmsford Prison.

That same evening, the prison was holding a man named Richard Linford, who was charged with serious assaults on a woman and her neighbour. He was being held in a single cell, because he had a history of far more severe psychosis than Chris Edwards as well as a record of violence which included an assault on a cellmate. On his previous sentence, he had been transferred to Rampton secure hospital. During the course of the evening, as more and more prisoners crowded into the jail, Linford was moved, to share a cell with Chris Edwards.

During the next three or four hours, prison officers realised that the alarm bell from the two men's cell was not working but did not investigate. At one point, they heard banging from a cell door on their landing. When an officer went to look, he found Chris Edwards had been beaten and kicked to death. One of his ears was missing. Linford had blood on his mouth and was talking about evil spirits. Chris Edwards had been in the prison for only nine hours.

An inquiry by police and prison services found that neither man should have been in prison and that Chris Edwards had been failed by "a systemic collapse of the protective mechanisms that ought to have operated". After the event, Linford was diagnosed as paranoid schizophrenic and belatedly transferred to Rampton high-security hospital.

Chris Edwards' parents subsequently sued and took their case all the way to the European Court of Human Rights. Two years ago, the court found that a series of errors had denied Chris Edwards his right to life. His parents continue to campaign for the government to make good its obligation to the court to erase such errors from the system.

The jailing of mentally disordered children

The Guardian, December 2004

Concluding his investigation into mentally-disordered prisoners, Nick Davies looks at the soaring number of children who have been locked up in prisons which cannot deal with their often alarming mental health problems.

We jail more children than almost any other country in the European Union. While politicians have been wringing their hands over the relentless increase in the number of adult prisoners, they have stayed almost silent on the fact that the number of children behind bars has been rising even faster.

During the 1990s, it just about doubled, with the sharpest increases among girls and children under 15. And now, while the adult prison population has fallen in the last six months, the number of children behind bars has risen again. We have obtained figures which show a dramatic surge in the number of children being jailed for breaching the government's Anti Social Behaviour Orders even though most of them have not been convicted of any criminal offence.

And since some 80% of children in custody suffer from at least two mental disorders - an even higher rate than among adult prisoners - the result is that young children with depression, anxiety, psychosis and severe personality disorders are now exposed to the same profound lack of effective treatment which afflicts adults behind bars. They are cut off from all psychotherapy and highly likely to be denied access to specialist child and adolescent psychiatrists.

And, just like adult prisoners, the most severely disordered children also suffer from the blockages which obstruct their transfer to secure hospitals. We have been told by psychiatrists who work in the system that there are at least a hundred children who are trapped in custody, often in punishment cells, even though their mental condition is so bad that they should be sectioned and sent to secure psychiatric beds.

These problems are particularly acute for mentally-disordered 15 and 16 year old boys. The surge of younger children into custody has pushed them out of the local authority secure homes and secure training centres, where the law allows them to be held in conditions designed for children, and into Young Offender Institutions, which are simply prisons for young adults up to 20 years old and which run a far tougher regime. There are up to 450 boys who are officially regarded as vulnerable being held in these prisons whose staff can only struggle to cope with the intensity of their problems.

Kevin Jacobs was 16 when he was sent to a Young Offenders Institution. By that time, he had been let down by the state for more than 14 years. He was less than 18 months old when he was first taken into care by Lambeth Social Services because his parents, who had learning disabilities, were failing to cope, yet he was not finally put on the register of children at risk for two-and-a-half more years; he was not presented to the fostering panel until he was eleven; even then, he was never placed with a foster family; and there is no evidence he was ever considered for adoption. Instead, he was shuttled between his family and various children's homes until, at the age of six, he was sent to a home outside London which was already being investigated for alleged abuse. Within two months of arriving, he complained that he had been sexually assaulted by an older boy and by a member of staff. Yet, he was left there for months still reporting more abuse and, when the police finally interviewed him six months later, they took no action.

When he was only four, Lambeth social workers asked for him to be assessed for special educational needs. It took nearly seven years for the education authority to do so and, in the meantime, he had been permanently excluded from three mainstream schools. Also when he was four, a paediatrician asked for him to be given a full psychiatric assessment. That was not done. When he was 15, another paediatrician made the same request - the community mental health team said they were too busy. His severe disorders and depression were never effectively treated.

From this childhood of institutional failure, abuse and confusion, Kevin stumbled into a criminal justice system which was geared to punish him but reluctant to help. He was eleven when he was first arrested, for setting fire to a barn. The police cautioned him and let him go. A Lambeth social worker who had been trying to help him and his four siblings, noted in his file: "We have wasted the best years of these children's lives already." When he was 12, the courts locked him up for two years for burglary, assault and more arson. Early in the sentence, a doctor recommended he be transferred from the secure children's home to a psychiatric unit, but they could not find him a bed. Later in the same sentence, another doctor found Kevin was a threat to himself and to others and urged that he be moved to the secure unit for adolescents at St Andrew's Hospital in Northampton, but St Andrew's said he was not ill enough and rejected him. During this two-year sentence, Kevin cut himself on some 40 occasions.

By the time he turned 16, he was locked into a pattern of crime and self-harm, often at the same time. He burgled a doctor's office to try to inject himself with insulin; stole a knife to cut himself; stole another knife to cut himself and some paracetemol for an overdose. Twice, he was taken to hospital and resuscitated. The police arrested him for trying to rob an old lady in front of his own social worker. In the cells, he cut his arms with cutlery and swallowed an overdose of ibuprofen. At court, a doctor and a worker from the local Youth Offending Team produced reports on him. Both agreed he was at serious risk of harming himself. Both said he needed a full psychiatric assessment, but it did not happen. Both said he needed secure accommodation, not prison; the magistrates agreed; but the secure accommodation was all full. The Youth Justice Board, to whom the local YOT worker reports, might have pushed and got him onto a waiting list, but vital paperwork which might have persuaded them to do so never reached them. So Kevin was sent to await trial at the Young Offenders Institution at Feltham, in Middlesex. That was when the trouble really started.

In his first two weeks in Feltham, Kevin was interviewed by a prison officer on reception, a prison GP, a prison psychiatrist and a psychiatrist preparing a report for the court. None of them was a specialist in child psychiatry, and all of them were working blind: the Youth Justice Board report on his history, which included his self-harm, was misfiled and never made it to the prison; nor did his medical records and other history. Kevin himself, like many prisoners, chose to conceal his history of disorder, claiming that he had not harmed himself recently nor ever tried to kill himself. Nevertheless, it was clear he was at risk. He was placed in the health care centre but, after three days, he was moved to a standard cell on the juvenile wing. The prison opened a F2052SH form to ensure he was checked regularly to stop him harming himself, but, after ten days, the form was closed.

When he went back to court to be sentenced a couple of weeks later, he was given a six-month detention order, three months to be served in custody, three in the community. There is no evidence that the court made any effort to ensure that he served the three months in a secure childrens home. He went back to Feltham, where he embarked on an erratic journey of self-harm and suicide attempts while the prison struggled and failed first to treat him and then to transfer him to a secure hospital bed. The sixteen-year-old boy began to slide into despair.

He was bullied, smashed the light bulb in his cell and was found lying on his back bleeding from both arms. The prison re-opened the F2052SH form on him and sent him to the health care centre. Twenty-four hours later, he was back in a standard cell where he was spotted with equipment to make a noose. So he went back to the Health Care Centre for another 24 hours.

The pattern was clear. Kevin would harm himself. Prison officers, one of whom described him as "a timebomb waiting to explode", would try to move him to health care, who would house him for a short time or turn him away because they were short of staff and/or had no suitable cell available. Sometimes, he was held in solitary confinement in the segregation unit; often he was held in a standard cell whose bars and fittings are not made safe against self-harm and suicide. The prison psychiatrist reviewed him four times and each time, still working blind without Kevin's missing paperwork, he accepted Kevin's assurances and concluded that he was at no immediate risk of suicide or self-harm: he advised that Kevin needed "no further psychiatric input" and referred him on a non-urgent basis to be seen by a trainee psychologist who was working in the prison.

Ten weeks into his sentence, two prison officers were leaving Feltham at the end of their shift on a day when Kevin had cut his arm with a broken cup and been told there was no room for him in health care. The officers were so anxious about him that they turned around and went back to tell the governor that Kevin must have a nurse to tend to his injury and special clothing which could not be turned into a noose. When they reached the wing, they asked another officer to check on him: Kevin was squatting beneath his window with a sheet tied around his neck, pulled tight against the bars above him, drifting into unconsciousness. They saved his life, walked him over to health care, insisted he be given a bed and finally left for the night. The suicide attempt was not recorded in his medical notes. Nor were his family told. The incident was not investigated.

During his first week on remand in Feltham, the psychiatrist who prepared his court report had recommended that he be transferred to a secure bed at the Bethlem Royal Hospital in Beckenham, Kent. This psychiatrist warned that, if he remained untreated, Kevin's behaviour would remain criminal and destructive. But the Bethlem wanted a consultant's opinion. The prison provided it. The Bethlem then said they would have to assess him themselves and said it could be four weeks before they did so. Four weeks later, they offered to assess him after a further four weeks - on the very day Kevin was due to be released from prison. In the background, the Youth Justice Board made no move to transfer him to a secure children's home.

As the prison struggled to protect the boy from himself, Lambeth Social Services held a series of meetings about his future and decided that it was too expensive to keep him at the children's home in Guildford, Surrey where he had been living for 20 months before his arrest. Since this was one of the few places where he had ever settled, Kevin was distressed by the news. Since Lambeth were then unable to come up with any other plan for him, and with his release date approaching, he became morose and anxious.

An outreach worker from the health centre recorded that Kevin was frightened and angry about the lack of a release plan and contacted Lambeth social services, the YOT worker and two prison governors to alert them to the crisis. The social worker and the YOT worker were both urging the prison to organise a full psychiatric assessment because of the risk that he would harm himself. A week later, Kevin was caught trying to break his cell light and tearing up sheets to make a noose. He said he had no real intention of killing himself. The psychiatrist judged he was at no immediate risk of self-harm. The health centre said their unit was too noisy for him. He was put in an ordinary cell, where officers checked on him every hour. Kevin told staff he was having flashbacks to being sexually assaulted as a child in care.

Two days later, his social worker came in to see him and told him there was still no sign of anywhere for him to live after his release, which was now just three weeks away. That afternoon and evening, Kevin sat in his cell watching television: there was a shortage of staff, and so he was not unlocked and, for five hours nobody spoke to him at all. The trainee psychologist visited him, he asked her to stay, she said she would come back the next day. By 10.45 that night he was asleep. At 2.15 in the morning, an officer looked through his hatch and found him still asleep. At 2.35, he was found hanging from his window bars. At three o'clock, the ambulance crew instructed staff to stop attempts to resuscitate him: Kevin Jacobs was dead.

It is three years now since the prison service and the department of health declared: "The government is very clear that the needs of juveniles and young offenders have to be a priority. It is therefore important to ensure that services for young offenders take account of the special and often very complex needs of this group." Those words have not been translated into reality.

An inquest into Kevin Jacobs' death found his death was "suicide to which neglect contributed". Most of those who dealt with him in prison were clearly worried about his welfare and often went beyond the call of duty in trying to protect him. The neglect to which the inquest jury referred was, in their own words, the result of "gross deficiencies within the system."

This systemic failure has a familiar shape. On the one hand, just about everybody who deals with young offenders knows that the best way to stop them committing crimes is to keep them out of custody and to attack the problems which are pitching them into crime in the first place. The government knows it too. Four years ago, they created the Youth Justice Board with a network of local Youth Offending Teams with the precise aim of pulling together different agencies to tackle the individual roots of any child's crime. And yet, the same government has sabotaged the YJB's work.

Relentlessly, this government has joined its immediate predecessors in driving more and more children into custody. Dr Barry Goldson, senior lecturer at the University of Liverpool, analysed crime figures and found that between 1992 and 2001, the number of children being jailed each year soared by 90%, from 4,000 to 7,600, while those held on remand jumped by 142%. The number of children under 15 sent to custody increased by 800%, the number of girls by 400%. The average length of their sentences also increased, by 16%, for example, for boys aged 15 to 17. And yet, he found, there had been no significant change in the number or seriousness of the offences children were committing; and no impact on re-offending. This was all about a politically-led drive to jail more children.

This government came to power in 1997 with a prominent pledge to "fast-track the punishment of persistent young offenders". Since then they have kept up the rhetorical pressure on courts and reinforced it with: the 1998 Crime and Disorder Act which specifically extended the powers of the courts to impose longer sentences on children aged 15 to 17 and made it easier to impose custody on children aged 12 to 14; and the 2001 Criminal Justice and Police Act which gave magistrates power to remand children as young as 12 for minor repeat offences whereas previously that had been done only for the most serious crimes.

They launched their new Detention and Training Orders for juveniles, which were seized on by magistrates, causing another spike in the jailing of children. In 2002, they launched their Street Crime Iniatitive. The inquest into the death of Joseph Scholes, who hanged himself, aged 16, in Stoke Heath YOI two years ago, heard evidence that the government put direct and arguably improper pressure on the Youth Justice Board to make sure that children picked up under the initiative were sent to prison.

More recently, the government have campaigned noisily to persuade police and courts to make more use of Anti-Social Behaviour Orders, which are civil measures which usually restrain their subjects from visiting named places or people but which can yield prison sentences of up to five years for those who breach them, even if they have not been convicted of a crime.

We have obtained figures which show a sharp surge in the jailing of children for breaching ASBOs: in the first four years of their use, some 255 children were sent to custody for breaching their ASBOs. The government were disappointed. Downing Street and the Home Office demanded they be used more often. The result was that in only four months this summer, 195 more children were jailed in ASBO hearings. (see panel). Some are only 12 years old. Often, these children had not even been charged with breaking the law. And 80% of them suffer from at least two mental disorders.

The result of all this incarceration is that the Youth Justice Board has been left struggling to fulfill its purpose. It now finds itself having to spend 74% of its budget buying places in custody with the prospect of spending another 1.6% just to keep pace with the new rate of ASBO jailings, all of which cuts the money which the board has left to spend on crime prevention. And these children cannot fit into the institutions which are designed to house them.

All of the local authority secure childrens' homes and all of the new private secure training centres between them hold fewer than 500 child offenders. Almost all 15 and 16 year old boys now end up in Young Offender Institutions, which have much lower staffing, much higher rates of self harm and suicide, far less education and far more chance of being locked up for 22 hours a day. They are prisons. And it is not just that these are children, but that significant numbers of them are officially regarded as 'vulnerable', like Kevin Jacobs.

When the YJB started work in 2000, it told the Home Secretary: "Young people should be placed in accommodation which most effectively meets their needs and the risk of harm that they pose to themselves and others. The accommodation should be appropriate for their age, emotional maturity and level of vulnerability." But YJB officials told the Joseph Scholes inquest in April this year that each day they can handle a hundred booking forms for children who have been sentenced to custody of whom 30 or 40 will be officially described as vulnerable and, if they are boys over 14, almost all of them will end up in a YOI. At any one time, they said, there are some 450 vulnerable children in YOIs.

Earlier this year, the board's new head, Prof Rod Morgan, opened a new building at a YOI and told his audience frankly that he hoped he could return soon with a wrecking hammer to knock the place down. But, inundated with child prisoners and short of funds, the board is being forced into reverse gear. Places in local authority secure homes are expensive, and the board has been cutting the number of beds it buys in them. As a result, two local authority security homes closed last year and two more are about to close. This means that the new private secure training centre which opened this year and two others which are due to follow in the next few years will end up largely replacing the homes which have closed instead of diverting children from prison.

For the mass of jailed children like Kevin Jacobs who suffer from mental disorder, this is a disaster. The Youth Justice Board, like the Prison Service, has genuinely tried to improve health care behind bars, but neither of them pretends that they have been able to do enough. In particular, there is a dire shortage of child and adolescent psychiatrists for juvenile prisoners. "What they get is other general psychiatrists or nothing," a senior forensic psychiatrist told us. A survey of psychiatrists working with young offenders in Scotland three years ago found that only three of them had forensic training and few even knew of a specialist child psychiatrist they could consult.

The process of finding a psychiatric bed for the most severely disordered children is even worse than that for adult prisoners, which we described yesterday. In the whole of England and Wales, there are only 44 secure psychiatric beds for children. They cannot cope with the surge of jailed children with severe mental illness. As a senior executive from the YJB told the Joseph Scholes inquest in April: "It's the view of the Board that about a hundred of these children should be sectioned and they should be kept in secure mental health accommodation, but I am afraid that does not exist." Even those who do get a bed can experience long delays - "three, six, nine months" according to a senior source - as first the specialist hospitals and then the local Primary Care Trust conduct their assessments and hold meetings.

The Joseph Scholes inquest heard from a former medical officer at the YOI where he hanged himself, who had personal experience of trying to transfer severely disordered children to secure hospital beds. Dr Jane Bright said: "My experience was that when we did have inmates who had gross psychiatric disturbance, and I'm talking about a psychosis rather than a non-psychotic patient, it was very difficult to get a forensic psychiatrist to see them, very difficult to get an appropriate bed for them. We had to wait weeks and weeks, spend hours and hours and hours making phone calls, very, very difficult. The resources just weren't there to back us up."

Kevin Jacobs' family are now suing the London borough of Lambeth, the Home Office and the Youth Justice Board. Their lawyer, Fiona Murphy of Bhatt Murphy, has told Treasury solicitors: "Imprisoning such a vulnerable and damaged boy was humiliating, debasing and degrading and had the effect of undermining his will to cope with his feelings of helplessness. His will to resist the suicidal ideation from which he suffered, was cumulatively undermined by the failure to plan for his release and address his medical needs."

Additional research by Roxanne Escobales

See below for statistics and case histories

Anti Social Behaviour Orders -

Average number of children jailed each month for breaches of ASBOs

Source: Home Office and Youth Justice Board

Children in custody -

Source: Youth Justice Board

Mental disorder among children in custody -

Source: Office of National Statistics

Case histories -

Toni aged 17 assault YOI prison

Toni used to watch her mother trying to hang herself and taking overdoses and using drugs. Her mother used to batter her with her fists or a hammer or anything else that came to hand. Eventually, she threw Toni downstairs, so Toni put herself into care. Her father was never around much. In fact, her mother told her he was not her father, and so she is not sure who he is. She remembers once he said there were dead people buried under the floor, so she and her mother pulled up all the floorboards and found a dog's head. It still had its collar on. Toni says she understands life: "My view of the world is that we are living in hell and we are all ghosts."

She has thoughts she can't control and voices in her head. She has been in trouble with the police since she was eleven - stealing, terrorising the neighbours, setting fire to things. Recently, she cut a girl: "Sometimes I feel I have to cut people up if I don't like them. Sometimes it only takes them to do one thing and, if I don't like it, then I'll have to have a go at them." So now, she is in prison, in a Young Offenders Institution. She is 17.

She wants help for her mental problems but, as far as she remembers, nobody has ever treated her. It's the same in the prison. Sometimes, doctors come to the cell. "Everyone comes to your door at once and they just stare at you." So she doesn't talk to them. When she was first in prison, they put her on self-harm watch, but then she told somebody she'd like to attack another prisoner, so the next thing she knew she was hauled out of her cell and taken down to segregation.

She spent 20 days there, locked up alone for 23 hours a day day. She could hear people screaming and smashing up their cells. One night, two people tried to hang themselves. Toni thought she could hear people crying, but she wasn't sure if she was imagining it. A 60-year-old woman in the next cell had got hold of some glass and was cutting her arm. Being on the seg made her feel low, her thoughts started to get the better of her, and so she took to banging her head against the wall. She felt paranoid, she was sure there were ghosts in the cell and then, one time, all the books fell off her shelf, so she knew the ghost was there.

Jink 17 assault and threats to kill YOI prison

A lawyer from the Howard League for Penal Reform found Jink in a prison cell - a 17-year-old boy who weighed 22 stone and who was suffering from a congenital heart disease as well as mental impairment. Almost as soon as Jink arrived in prison, the prison doctors had said he needed to be sent to a secure psychiatric unit, but seven months later, he was still in his cell. When the lawyer found him, Jink was eating faeces and urine.

When the lawyer asked why this deeply disturbed boy had not been moved immediately to a psychiatric hospital, he found that Jink had simply got lost in the system. The prison were extremely worried about him but they could not give him the care he needed. One day, Jink used his clothes bag to hang himself from the bars of his cell. They cut him down, he was still alive and one of the governors went and had a cup of tea with him, and Jink told him how his dad had started buggering him when he was six years old and used to keep him locked up in the bedroom. The governor said he seemed very low.

Straight after that, the prison called together all the agencies who had an interest in the boy - social services, youth offending team, Welsh health commissioner, prison doctors, hospital psychiatrists, defence lawyer. But they still could not sort out where to send him and how to pay for it. And all the time he waited, Jink was being bullied mercilessly. If ever he left his cell, he clung to a prison officer for protection.

Jink had assaulted his mother and threatened to kill her. From time to time, they would take him to court for a remand hearing. It was the first time he had been in trouble. He hated going to court: the cubicle in the van was too small for him; he suffers from claustrophobia; once, the door got jammed and they couldn't get him out; he had a panic attack in the van.

The lawyer from the Howard League said it was simply unacceptable that Jink had been wating seven months for a hospital bed and so he got onto his computer and Googled psychiatric units and rapidly found one which was suitable. He called the unit, who agreed to assess Jink. Two weeks later, he was moved. He is much better now.

Henry 16 arson with intent to endanger life Secure Training Centre

Henry was severely sexually abused as a young boy. He has had mental health problems since he was nine. He hears voices, has blackouts and flashbacks, sets fires, harms himself, drinks to the point of hazard. The government says no prisoner with a severe mental illness should wait more than three months to be transferred to a secure psychiatric bed. Henry waited fourteen months.

The first six months were spent simply waiting for a local community psychiatrist to assess him. Finally, the psychiatrist declared that there was no suitable placement in the Primary Care Trust's area, so he had no remit to do the job. In the background, the local youth offending team had asked another psychiatrist to asses him, but that never happened and the YOT lost interest. Henry's lawyers appealed to the local director of public health, but he was off sick.

In the seventh month, the judge who was waiting to preside over Henry's trial said he was 'extremely concerned' by the delay. The lawyers wrote to the head of the local YOT and threatened to summons him as a witness before the judge if he did not take responsibility within two weeks. The YOT replied that it was really up to the PCT. The PCT said they would consider a non-local placement, but it was up to the community psychiatrist to arrange an assessment. The community psychiatrist said he would refer Henry to the Gardener unit in Manchester. The doctor who was advising the lawyers said it was the wrong sort of unit: Henry needed to be in St Andrew's in Northampton. The community psychiatrist said St Andrew's was private and too expensive and he could not refer him there. The lawyers thought the best thing would be for both units to assess Henry. It was the ninth month before they discovered that nobody had actually referred Henry to either place.

The lawyers then got an independent psychiatrist to assess Henry, and he agreed that the Gardener unit was wrong for him but suggested the Proudhoe in Northumberland. But the Proudhoe had no beds and said they could not even assess him for two more months. The lawyers wrote to the local MP who wrote to the PCT who said they still wanted the Gardener unit because it was cheaper. The Gardener unit also had no beds. Then Henry got lucky. The Proudhoe brought forward their assessment and agreed to take him - but not until the twelfth month, when a bed was due to come free. In the meantime, St Andrew's also assessed him, agreed to take him but failed to tell the lawyers. And they also had no beds.

The MP wrote more letters. The lawyers made a formal complaint to local social services, who started a dispute about how to proceed. They also lodged a complaint with the local PCT, who failed to reply. By now, it was the twelfth month and the Proudhoe bed was due to come free. It didn't - the patient they were expecting to leave had needed to stay. Then the lawyers finally heard from St Andrew's that they had been willing to take Henry, that they had had two free beds in the past couple of months but had said nothing because they thought he was going to the Proudhoe, and now both beds were filled again. They agreed to put Henry back on their waiting list, but only if they had a new referral from the community psychiatrist, who was still saying he should go to the Gardener unit. In the fourteenth month, after four assessments involving three units and seven doctors, Henry finally was given a bed at the Proudhoe. During all that time, he received no effective treatment for his illness.

Babs 17 affray and arson with intent YOI prison

As a little girl, Babs was abused sexually, physically and emotionally and spent years going in and out of care. In one foster family, she was sexually abused again. She also saw her young brother die. As an adolescent, she tried to get help but was told she was just being manipulative. She often committed offences.

When she was 17, she set fire to her bed, was taken to court and pleaded to be sent to a psychiatric hospital. When they refused, she ran from the court, met up with two friends, who were both schizophrenics, got drunk and swallowed all their medication with them, was indecently assaulted by one of them, trashed the house, set fire to the sofa, was subdued by police using CS gas and still not sent to a hospital. Instead, she was remanded in custody to await trial.

There, a prison doctor learned about her two personalities (Andy who she liked and Andrea who she didn't) and diagnosed personality disorder, mood disorder and post traumatic stress disorder. The doctor said she must be transferred to a secure hospital. But that took eleven months, during which time her condition careered downhill and the prison alternated between desperately trying to protect her and punishing her.

She made her her most serious suicide attempt when she hanged herself with a cable from her cell window. Officers cut her down and left her in the cell under constant observation. When she recovered, she wanted to leave and tried to shove her way past the officer in the doorway. Other officers were called. She was handcuffed, stripped, put into a strip gown and placed in the punishment block where one of her friends had hanged herself.

During her eleven-month wait for a hospital bed, Babs tried to hang herself using cables, laces, torn shirts and sheets. She cut and burned her legs, hair, face, hands and tried to cut off her thumb. She also tried to throw herself off the wing landing. On several occasions, she dug a wound in her arm and buried a pen in it. The pens had to be removed at a local hospital and she was warned that if she continued to damage her arms or hands, they might have to be removed.

Margie 17 breaking and entering and assault YOI prison

Margie has had trouble with mental illness all her adolescence. She has been sectioned twice and held in hospital for psychiatric care and she has been been prescribed anti-psychotic medication. She hears voices. Given the chance, she will drink eight cans of beer a day and a bottle of vodka a week.

The reason is not hard to find: she spent the first eight years of her life being beaten up by her father and watching him do the same to her mother; when she was eight, her mother fled to a refuge taking Margie and her four siblings with her; but, one night, the father found them and broke into the room and Margie hid and watched while he stabbed her mother 17 times and then kidnapped her brothers and sisters, leaving her alone with the corpse.

At first when she was arrested - for breaking into another girl's home and forcibly shaving her head - it looked as though she would be moved from prison to hospital without much trouble. The judge said her transfer was 'extremely urgent' and, within two months of her arrest, a psychiatrist assessed her and diagnosed schizoaffective disorder with post-traumatic stress and borderline personality traits. He not only recommended that she be transferred to a secure hospital but pointed out that she wanted to kill babies and added: "Formal therapeutic interventions are vital. It would be impossible to contain her behaviours and provide such intensive interventions, if she were not to be managed in a secure psychiatric unit.... Failure to implement such provisions at this opportune time can potentially result in far more serious consequences to property and people in the future."

But the primary care trust, who were going to have to pick up the bill, were not so sure. They sent another psychiatrist to assess her, and she concluded that the first psychiatrist's diagnosis was wrong, that Margie would not respond to treatment and would not mix well with other patients in the unit and might be malingering. The PCT refused to fund the transfer.

Margie was given a long sentence and no visible prospect of a hospital place. The last time she was heard of, she was extremely anxious: she had just heard her father was due to be released from the sentence he was serving for murdering her mother. She said: "My brain thinks that if I throw a stone in a pot then I won't die, and I don't throw a stone in a pot."

* the names of all children have been changed

Dying for a break

The Guardian, May 2005

Maybe nothing really changes. Several hundred years ago when red-faced judges and pot-bellied politicians were happy to procure power by ordering men to be hanged by the neck and left dangling to rot by the wayside, there was a popular rhyme: "Little villains oft submit to fate, so great ones may enjoy the world in state."

Think of Andrew O'Connor. When Michael Howard tells you that prison works, when the Prime Minister celebrates his tough line on crime, when you hear once again that this country jails more of its men and women than Burma or China, think of Andrew O'Connor - petty thief, pain in the neck, prolific offender, habitual prisoner, dead.

There is nothing so unusual about his being dead. People who work with offenders will tell you that, as things stand, there really are only three ways that most repeat offenders finally stop breaking the law - they just grow out of it; they kill somebody and get locked up for life; or they die. Some of those who die commit suicide, some of them accidentally overdose, most of them just die from general neglect.

As offenders go, there never was anything very unusual about Andrew O'Connor. Any social worker or police officer or prison officer reading this story could stop now and, with their eyes shut, they could predict just about all of his background: his family were poor, he grew up on crap estates, he never knew his father, he was knocked about by his step father, he was a nightmare at school and a pest on the streets, he was put into care, he was sexually abused. By the time he was ten, he was being arrested. By the time he was 15, he was doing custody. By the time he was 30, he was dead. Nothing unusual. There are thousands of Andrew O'Connors shuffling through the dark tunnels of our criminal justice system.

Now, watch how the judges and the politicians play their hand. And look back, in particular, at what happened after the Prime Minister, in the autumn of 2001, decided to make good on his pledge to be tough on the causes of crime by personally commissioning the social exclusion unit in the Cabinet Office, to find out what he had to do to stop the endless reoffending of people like Andrew O'Connor.

The SEU set off on a mighty investigation, using five civil servants to work full time for the following nine months, trawling through statistics and research, visiting prisons, interviewing experts, talking to officials in government departments, all in search of an answer. And they found it. In July 2002, they published their report, "Reducing re-offending by ex-prisoners" - 218 pages of analysis, packed with facts and figures, supported by more than four hundred footnotes. Its conclusions were clear.

Essentially, they boiled down to three big themes. First: "Prison sentences are not succeeding in turning the majority of offenders away from crime." Second: "A prison sentence can - and frequently does - make things worse." Finally: the real key to reducing offending was to attack its causes (just as the Prime Minister had been saying). Homelessness, unemployment, drug and alcohol problems, mental health problems, physical health problems, educational problems - these were the seeds from which crime grew, seeds which were fertilised by the impact of imprisonment.

In many ways, there was nothing new about these conclusions. Any front-line worker in the criminal justice system would have recognised them. The importance of the SEU report was that these conclusions were backed by irresistible evidence, supported by seven different government departments who had been consulted by the SEU, and delivered direct to the most powerful figure in government, who duly welcomed them as "a significant contribution to our understanding of what works in combatting crime." This was a chance finally for the criminal justice system to start to make a difference.

It could make a difference to the communities who suffered from the crimes of prolific offenders: the SEU found that released prisoners were streaming unchanged out of custody and committing something like a million offences a year and that these offenders alone were costing at least £11 billion each year (that did not count the value of what they were stealing or damaging: £11 billion was simply the cost to tax payers of pushing them through this ineffective process of punishment.)

And it could make a difference to the offenders themselves. By the time the SEU report was published, in July 2002, Andrew O'Connor, then aged 29, had been convicted in the courts of Devon, where he lived, on 24 different occasions. Never once had any one of those court hearings done anything which was conceivably likely to stop him offending - because none of them did anything to tackle the causes of his crime. They treated him as though he really were the Daily Mail's caricature of an offender, breaking the law because he was 'a bad person', simply in need of stiff punishment to deter him. Andrew O'Connor was not this caricature: he was a young man whose life had collapsed under him like rotten floorboards.

His father left before he was born and never bothered to come back for him. His stepfather thrashed him with a belt, so the boy used to run away and sleep in the woods for several days at a time. His mother reluctantly agreed to put him into care for a while - never knowing that her much-loved nine-year-old boy would be slippered, caned, bullied and casually used as a sex object by an older boy and one or two members of staff. Then he started going out with other kids, nicking stuff from shops, setting fire to an empty building, stealing cars, taking pills, screwing houses, smoking dope.

By the time he was 12, he was the walking, talking, non-stop-offending embodiment of what the SEU report was describing. Prisoners are 13 times as likely as anybody else to have been in care. They are also 13 times as likely to be jobless, ten times as likely to have played truant at school, six times as likely to be very young parents. Eighty per cent of them cannot write with the skills of an eleven-year-old. More than 60% of them have drug problems. More than 70% of them suffer at least two mental disorders. Andrew O'Connor had all this. The courts, however, just carried on hurting him, as though they had some moral right to pile torment on a ten-year-old, as though they really thought that by doing this, they were protecting the community from his offences.

He was convicted of stealing cars in Plymouth. So they endorsed his licence. That made no difference at all. He didn't have a licence: he was only 12 at the time. He carried on stealing cars. So the courts endorsed his non-existent licence again - when he was 13 and once again when he was 14 and yet again when he was 15. On the last two occasions, they also disqualified him from driving, never minding the fact that he was still too young to qualify to drive in the first place, never wondering why this small boy was earning a living by stealing expensive cars for adults in Plymouth and taking his pay in cannabis and amphetamine.

From the age of ten, he came up in court for burgling. The bench solemnly imposed a care order on him. But he was already in care. That was part of the problem. They fined him too, even though they knew very well that his only source of income to pay the fines was more burgling. When he was 14, they started locking him up - that year they sentenced him to three bouts of custody, amounting to ten out of the ensuing 13 months, even though it was plain that he was coming straight out of custody each time to carrying on with his offending. When he was 18, they solemnly gave him three years probation, never minding the fact that they had already given him two years probation - not once but twice in the previous ten months. He just went ahead and caried on offending.

After the courts failed, the prisons failed too. By the time the SEU report was published, Andrew O'Connor had been released from custody eleven times and he had never received any effective help for any of the problems which made him an offender. Over and over again, he would come out homeless and jobless, bingeing on drugs and alcohol, haunted by personality disorders from his abusive childhood, unregistered with a GP, still unskilled, still unable to read or write like an adult. Each time, he stumbled straight back into crime - thefts, burglaries, assaults. On one release, he was given £10,000 in compensation for the sex abuse he had suffered in care. He blew the lot in weeks, on drugs and generosity to strangers. Sometimes, it took him only a day to get back into trouble; never more than a few months.

Mr Blair's government already knew quite a lot about this and had started to talk more about rehabilitation. Andrew O'Connor had heard the talk and he had believed it. On his last big release before the SEU report, in March 2000, he had written to his legal adviser, Nicki Rensten of the Prisoners' Advice Service: "I've got a good personal officer. He's set up meetings for my release and is in touch with my probation officer. He's guaranteed me I won't just be dumped on the streets like last time. They're working to set up a package for me with the best support I can have.... Between him and another officer, they have phoned the council housing and DSS on my behalf. He says he'll get probation to pull some strings to get me in my own council flat on relase with help with clothing, furniture etc, and help to set myself up working.... I want to live in this lovely place in Devon called Teignmouth.... I think I'll give prison a rest in future. I think this is going to be a good year."

It never happened. Andrew O'Connor was pushed out of the gate of Long Lartin Prison one morning with no address to go to, no job to go to, no GP to look after him and only £46.75 of discharge grant to last him the several weeks it would take him to sign on. None of the help he had been promised materialised. The housing office in Teignmouth said they did not have to house him, because he was not vulnerable enough to qualify. Without an address, he had even less chance of finding a job. The DSS gave him no help with clothing or any of the other basics which he lacked after a four-year prison sentence; his discharge grant soon ran out and, when he eventually received his benefit, he found the minimal weekly payment had been cut even further to make him repay a crisis loan he had taken out six years earlier. His former GP refused to take him back even when he broke his foot. He ended up sleeping in a friend's car where he was arrested and sent back to jail - for failing to tell probation where he was living, even though they themselves had allowed him to be released without an address to go to.

And what the SEU report captured so vividly from its reservoir of research was that prison was not merely failing to stop the offending of people like Andrew O'Connor: it was actively stimulating it. Homelessness was a cause of crime - but 30% of prisoners lost their homes as a direct result of being jailed. Unemployment was a cause of crime - but two thirds of prisoners who were in work, lost their jobs as a result of being jailed. Debt was a cause of crime - but a third of all prisoners saw their debts get worse as a result of being jailed. Broken family links were a cause of crime - but 43% of prisoners lost these links as a result of being jailed. The failure of the system to deliver its objective was stunning. This failure - this routine, ritualised, systemic failure - was why the SEU report mattered so much. But look how the politicians played their hand.

The warning signs were there from the beginning. This was the first and only report from the SEU which had no action plan at the end of it: the government departments who were responsible for housing, employment, education, benefits etc had agreed to support it only if there was no timetable committing them to change. All they would accept were 'recommendations' without any specific policy commitment. The SEU went along with this in the hope that these departments would have to deliver a real commitment once the report was published if only to allow the government to produce its response, which was due six months later at the end of 2002. What happened was very different.

There was no government response after six months. There was to be no response for two long years, because the government departments whose co-operation was vital, were determined not give extra money or priority to a bunch of politically unpopular offenders, no matter how much crime was generated as a result. As one direct witness told us: "The SEU were shafted. These department officials don't really see the 'social dregs' at the bottom of the pool. They have no sense of the link between the decisions they make and the crime that will be committed as a result. There was two years of acrimonious meetings, with these departments sitting on their hands, digging in their heels, blaming each other. It was really disgraceful."

The Home Office set up a team to implement the report. It was considerably smaller and far less senior in its leadership than teams which had been set up for other comparable issues. The Treasury was interested in saving money by turning people away from prison, but it was not prepared to take any political risks, so Treasury officials backed the SEU's plans for more education and job advice for prisoners - they were politically easy to sell - but they would give nothing for the other causes of crime. Overwhelmingly, the Treasury's investment stayed with prisons, even though the SEU had found so much evidence of their failure - even though the Home Office team came up with their own cost-benefit analysis to show how, in the long term, investment would save public money.

Other government departments were even worse. The small Home Office team spent months trying to secure housing for released prisoners, waving the SEU evidence that a stable home cut re-offending by 20%. All they wanted was for the Office of the Deputy Prime Minister to tell local authorities to treat more prisoners as 'vulnerable' and, therefore, entitled to housing. The ODPM would not have it. They said prisoners' housing was a job for the prison service. The prison service said that was fine but they didn't have any housing. The ODPM said it was no good coming back to them, because the law would not allow them to treat prisoners as 'vulnerable'. One of those involved said this was "nonsense - they just didn't want to do anything". The ODPM got their way: even if it did mean more crime, there was no effective priority on housing for released prisoners.

And there was no help for the thousands of prisoners each year who rented homes and then lost them when they were jailed because they could not get housing benefit to pay the rent. A relatively new rule meant that those who did manage to get housing benefit lost it immediately if it looked as though they were going to spend more than 13 weeks behind bars. Before the report was published, the SEU had persuaded the Department for Work and Pensions to extend this to 26 weeks, but then Andrew Smith had become Secretary of State for Work and Pensions and ruled that single men were not a priority for his department and stopped the whole move. Now, the Home Office team tried to tackle some local authorities who were refusing to pay prisoners any housing benefit at all on the grounds that if they were jailed, it was their own fault, so in the eyes of the law they were intentionally homeless and barred from benefit. The ODPM variously refused to admit that the problem existed and/or declined to solve it.

One after another, the recommendations of the SEU report were killed off: the prison service refused to increase the discharge grant; the Department for Work and Pensions refused to pay prisoners' benefits quicker. At one point the DWP agreed to grant released prisoners immediate crisis loans to stop them running out of money while they waited for benefit, but then they reneged on the deal. There was nothing to make sure that released prisoners had GPs to go to. Nothing at all to help prisoners with alcohol problems, just a strategy document without any resources (a decision which was disguised with classic Whitehall elegance as "building on good practice within existing resources").

The Home Office team tried to re-unite imprisoned mothers with their children who had been taken into care. Nothing. They tried to help the 125,000 children a year who lose a parent to prison - children who subsequently prove to be three times more likely than their peers to have mental health problems. Nobody was interested. Two Whitehall officials sat through a meeting where they refused to make prisoners' children a higher priority for the Sure Start programme and walked away declaring quite audibly: "Well, that was a load of old bollocks, wasn't it?"

The Prison Service itself was asked repeatedly to require all governors to assess the needs of all new prisoners so that their problems were at least known. "That was thrown out with real scorn," according to one witness. "This was a key thing - to know what problems they were dealing with, so they had a chance to solve them - and they just refused point blank."

The government did not reject everything. They agreed to set up Jobcentre Plus advisors in prisons and to fund new programmes to link drug treatment inside prison to agencies on the outside. They transferred prisoners' healthcare to the Department of Health with Inreach teams to work with the most mentally disordered on prison wings. But these moves were politically unavoidable. The fact remained that when the government had the copper-bottomed evidence they needed to attack re-offending, they had chosen to be dishonest about crime and dishonest about the causes of crime.

The civil servant who had headed the Home Office team, Julian Corner, subsequently resigned and wrote a little-noticed article in Safer Society, the magazine of the prison rehabilitation group Nacro. Corner recalled the strength of the evidence in the SEU's report, which he had helped to write: "The premise was that seven government departments would go on to forge a united front against reoffending, and prison would only be used as a last resort. What actually happened was that they cherry-picked the most politically acceptable and convenient actions, and rubbished the rest - namely, the social inclusion measures."

But all this left the government with a problem: they still had to publish a response to the SEU's report. How could they justify rejecting so much evidence of what really stopped reoffending? On the face of it, the Prime Minister who personally had commissioned and welcomed the report should himself hold a press conference to respond to it. Or perhaps one or two of the relevant ministers would take on the job. But they did nothing like that. The response, when it came, was brilliant.

On Monday July 19 last year, the Prime Minister went to a community centre in north London and made a speech about crime in which he blamed our problems with law and order on what he called the 1960s liberal consensus. It is fair to say that this was an idea which was unsupported by any criminological research anywhere on the planet.

That speech was leaked in advance to half-a-dozen newspapers, including The Guardian, all of whom ran it prominently on that Monday morning. During the day, the Home Office put out details of future plans for attacking crime - on-the-spot fines for children, electronic tagging, satellite tracking. This ensured that the speech enjoyed another round of reports on the Tuesday morning. The Daily Mail's headline said: "Blair the crimefighter (and about time too)." Then the columnists and the leaderwriters joined in. The alleged link between the historic rise in crime and 1960s liberalism was still being reported and discussed in the Sunday papers at the end of the week.

And on that Monday afternoon, while all the crime specialists and all the leader writers and the columnists were chasing the decoy of a speech which was unsupported by a single grain of criminological evidence, in the midst of that, without any press conference or press briefing or statement in the House, the government quietly placed their response to the SEU report on the Home Office website. And while the Prime Minister's speech received tens of thousands of words of coverage, spread over a whole week, their response to the SEU's vital research work received.... not one single word.

In the same Safer Society article, Julian Corner reviewed the response which was presented as 'a national action plan' with 60 action points: "There's a commitment to ‘mapping and analysing existing housing and service provision'. Has it actually taken all that time to build the resolve to map provision? How about ‘developing a strategy for a more coherent information and advice service'? Isn't this the work needed to produce the action point? Then there's ‘developing a timetable to introduce a measure' - a new definition of action, surely. And ‘work will examine the possibility of working with a particular region to establish best practice'. Beyond parody. These are, in reality, 60 inaction points."

The 60 points did not even mention the particular problems of prisoners on remand, or on short sentences, or of women prisoners or young prisoners or those from ethnic minorities. Corner continued: "We had, after the SEU report, the near ideal circumstances for putting into place the social care framework that might, conceivably, have made imprisonment compatible with the reducing reoffending agenda. But ministers and officials lacked the resolve to follow through on the report's recommendations." And nobody noticed. The government had hacked the SEU's work to bits and now they had quietly buried its remains.

By now, >>> Of course, Andrew O'Connor was dead. While the government had been sitting on the evidence of the failure of prisons, the numbers being jailed had carried on climbing and Andrew O'Connor had been behind bars again. During the two years when government departments were busy blocking most of the SEU's findings, he had managed three months at liberty, ended up with a crack habit and a bed in the Salvation Army hostel in Devonport, Plymouth. One night, he walked into a snooker club and asked for a drink. They told him he would have to pay £10 for membership. He told them he would pay £5 this week and £5 next. The bar staff told him to get lost. He told them to give him a drink or he would take everybody in the bar hostage. He was unarmed, but the club called the police and he ended up doing 16 months for affray. When he came out of that sentence, he lasted one day before being sent back for breaching his licence: they said he had stumbled drunk into a probation hostel; he said he was having an epileptic fit.

He spent another four months in prison, in Exeter. He did a course in enhanced thinking. When they asked him for his objectives, he wrote "No objectives - you won't get anywhere." He told them: "I never had a proper childhood so am unsure a lot of the time how I should respond or even if what I am thinking or feeling is appropriate for the situation." He passed the course.

He filled in a form at the prison: "Paramount, I need structured care plan to make sure of good recovery. I would like to achieve outside counselling for the root causes of my problems, so the right support and right environment, ie around my family, with intense counselling, this would be the best plan for me." He had a big reconcilation with his mother, who said he could come home to live with her. He wrote to a friend: "My mum is taking three months off work to love and support me, take me to meetings and appointments, help me fill out forms, take me places and re-integrate me into society generally."

As usual, it never happened. There was a meeting in the prison to discuss his release. His probation officer failed to turn up. The prison workers agreed that he should go to his mother's with help from a local agency to deal with his drug habit. But the probation officer then over-ruled this and said he must go to a hostel miles away from her, in Cornwall. The hostel housed sex offenders and, recalling his past abuse, Andrew O'Connor refused to go there and, in protest, then refused to work with the local drug agency. His mother spent the three weeks before his release phoning the police and probation and the prison, warning them that Andrew would not go to the hostel and must come to her. Most of her calls went unreturned.

On Thursday October 23 2003, he walked out of prison. On Wednesday October 29 at 7.30 in the morning, at the Peppermint Park caravan site in Dawlish, south Devon, he was found dead. The pathologist said he had had an epileptic fit in the night. If he had been at home, if he had had a GP, if he had had a life, maybe it would have been different. As it was, he was living a half life. He was fully clothed. His mother said he must have slept that way so he could run away if the police came to get him. She wrote afterwards: "Andrew was a victim before he was born and continued to be a victim. He wanted to come home. In the last twenty years, he never had his own bed. It breaks my heart and fills me with so much pain to know one of my children that I loved, suffered so much for the biggest part of his life, because of my mistakes, to let him go into care, and the mistakes of a system that was supposed to care for him. Why was it so hard to get any real help? Is it easier to just lock someone up to keep them off the streets?"

The SEU's work was not entirely wasted. Its research has become a reference point. There is now a Reducing Reoffending Committee in Whitehall where senior officials carry forward the national action plan which eventually emerged. There are signs of new life - some new money for housing advice, a pilot project in the South West to help released prisoners find homes, "invest to save" money for working on family links. And yet the central thrust remains diverted, away from an effective attack on the causes of crime, towards the acquisition of political power through the infliction of hardship.

The Guardian recently re-published a letter from a former nurse who had started to work with young offenders in Manchester: "It is a complete non-sequitur that because a boy stole your watch, he should be supported on your rates in jail, perhaps for life... The punishment of jail is not a deterrent.. 'Punishment' is perhaps not a word in God's vocabulary at all and, if so, ought not to be in ours." The letter was written in 1890, by Florence Nightingale. Maybe nothing really changes.

Additional research by Roxanne Escobales

Killing Crime Reduction

The Guardian, July 2003

There are a lot of chief constables who would happily strangle George Dixon. It's not so much that the old BBC copper with his folksy winking ways makes any real officer look inadequate, nor even that he had the infuriating advantage of scriptwriters to deliver his perfect results. The real problem is that Dixon is cemented into the public imagination - and he's not very good at his job.

Consider the case of the car park in the country. Ramblers used to turn up there, park their cars, head off on a hike and then return to find their car windows smashed and their belongings stolen. Dixon's answer would have been to get out his handcuffs, hide behind a bush until the thieves turned up, nick them and wobble off home on his bicycle. In the real world, another thief would have been back breaking into the cars before he reached Dock Green and the offenders he arrested would have been bailed to join in again the next day.

For decades, British police would have been happy enough with that failure: they had no option. But in the last 20 years, researchers in the United States and Britain have been urging them to think in a radically different way - to stop relying on the conventional tools of arrest and conviction and to start looking beneath the surface to manipulate the causes of crime, to stop being dragged around by events in favour of stepping back and trying to change them. They call it problem-oriented policing. More and more chief constables can see that when it works, it is a great deal more effective than the esteemed Dixon.

In the case of the car park, instead of reaching for their handcuffs, the local police stopped and analysed the crime. They realised that the real problem was simply that, once the ramblers hit the hills, there was nothing and nobody to stop the thieves doing exactly as they pleased, so, instead of making pointless arrests, they decided to solve the problem at its root - by building picnic tables at the side of the car park. To make sure that picnickers turned up, they arranged for a licenced vendor to sell drink and food up there. Natural surveillance. They cut the thieving by 48% in a year.

In the same way, there was a city force who had a problem with a take-away pizza parlour. Every Friday and Saturday night, groups of drunken people would gather outside, obstructing the pavement, disturbing the locals, occasionally fighting. The old-fashioned answer would have been to send a uniformed officer to stand there and arrest anybody who broke the law - inefficient and ultimately ineffective, because after a week or two, the officer would have had to be deployed elsewhere. Instead, they analysed the problem and discovered that the crowd was building up because the pizza parlour could not cope with the sudden rush of customers after the pubs closed, so they installed hot lines from all the local pubs to the pizza maker, who took their orders in advance and had their pizzas ready when they turned up: no more crowd on the pavement, no more trouble.

This problem-oriented approach is the heart of a far wider effort to bury George Dixon: to say that arrests and convictions have their uses but they also have their limits; to break out of the boundaries of the criminal justice system by pushing resources and energy into working with other agencies and delivering something other than punishment; to attack the infinitely complex roots of crime with infinitely flexible tactics. It's not so much criminal justice as crime reduction.

It is the simple equivalent of doctors admitting that they can't cure cholera with medicine alone: somebody has to find the source of the diseased water and clean it. It means reversing John Major's simple edict on crime that "society should understand a little less and condemn a little more" and working at three levels: to look at the deep reasons why some adolescents devote their lives to crime while a sibling in the same family will not, to deal with family, schooling, housing, health; to see the opportunities for crime as problems which can be solved (like the rural car park and the pizza parlour); to make maximum use of the 'target hardening' techniques of traditional crime prevention. It means giving George Dixon a brain as well as truncheon.

This is the story of the boldest departure in British criminal justice strategy in the last hundred years, the opening of a second front in the war against crime, drawing on the thinking of radical criminologists and pressure groups who were previously locked out of power. It has unfolded almost invisibly, largely unreported and sometimes misunderstood by news media, effectively camouflaged (by accident or design) by the government's relentless release of hard-line criminal policy statements. Most important, at this particular moment, this is a story which looks like it may have a miserable ending. Behind the scenes, this ground-breaking campaign has turned into the policing equivalent of the Flanders fields - misguided generals, destructive commands, confusion in communication, disorder in the ranks, brave young ideas broken and bruised on the ground. But the fight is not yet over. The great danger is that the ministers who launched the campaign may now desert it in its hour of need.

It all started in the spring of 1997. Labour had won the election, and Jack Straw moved into a Home Office which was unlike any other government department in two key respects. First, it had not been starved of resources and so, even though Gordon Brown imposed existing Tory spending plans across Whitehall, the Home Office still had money to spend. Second, other departments were trapped by Tory policies which were destructive but deemed to be politically untouchable (the sale of council houses, the privatisation of railways, Kenneth Baker's education reforms), but the Home Office had nothing. It was a policy wasteland. For nearly 30 years, the masters of the criminal justice system had resorted to a belief which they summarised in two simple words: 'Nothing works'. The last Tory Home Secretary, Michael Howard, had exploited the vacuum to embark on a cynical exercise in political populism, with other two other simple words: 'Prison works'. It didn't. Jack Straw inherited a police service which was detecting fewer crimes, courts which were clogged with delays, fines which were not being collected, community sentences which were not being served and overcrowded prisons which were failing to change the behaviour of prisoners. But he had money to spend and the political opening to do so.

Better still, he had the Prime Minister on side. Under the influence of 'radical realist' criminologists, Tony Blair and the Labour electoral machine had finally seen through the left's blind spot on law and order, understanding that crime was not some kind of romantic rebellion by an alienated working class attacking their middle-class tormentors: it was a miserable plague which was overwhemingly likely to be inflicted on the working class for whom they claimed to speak. For nearly 20 years under the Tories, an epidemic of poverty had swept through the old public housing estates, infecting them with burglary and mugging and harrassment, and it was no longer good enough to have a criminal justice policy which consisted of leaning back over the After Eights and sneering at the cops. Five years earlier, as shadow Home Secretary, Blair had come across the germ of a strategy.

He fastened on to a report by a management consultant, James Morgan, who had been hired by the then minister, John Patten, to look for a solution to rising crime. Morgan had come back with a radical plan to make it a legal requirement for all local authorities to form partnerships with their police in order to attack the causes of crime. Patten took one look at it, took fright at the idea that a Tory government would pass any power at all to local authorities and shelved it. In opposition, Blair met informally with chief constables and local government leaders who encouraged him to adopt the Morgan Report as Labour policy; he goaded the Tories over their failure to act on it. Now, Jack Straw took the Morgan Report down from the shelf.

At the same time, Straw told the Home Office's research department to trawl the archives for evidence of effective tactics. This exercise finally swept away the defeatism of Nothing Works and produced a report, Reducing Offending, which drew heavily on American experiments to pour a stream of fresh thinking into the Home Office. "Nothing works" gave way to "What works".

In effect, the researchers took the rule-book of law enforcement and tore it up. With tightly-argued evidence, they exposed the weakness of traditional policing technique (police patrols "do not have a marked effect on crime levels"; laying more charges against offenders "does not have any noticeable effect"; reducing crime by attacking drug "does not happen") and of traditional punishment (prison was no more effective than fines or community sentences and was so expensive that it would cost £380 million to cut total crime by only 0.6%.) Then they showed the alternatives: deep social programmes like the American pre-school scheme which worked with problem children so that only 7% of them went on to be arrested by the age of 27, compared to 35% of their unaided neighours; using design to cut the opportunities for offenders, like the Dutch programme which cut burglary by 70% by insisting that all houses be secured against crime, the banks who had redesigned their credit cards and cut fraud in half, the local authorities who had redesigned their wheelie bins to stop burglars using them as climbing aids. They recorded how the British car industry had finally started building security into their cars after the Home Office in 1992 published a Car Theft Index which diverted buyers from the riskiest vehicles.

One of these researchers, Professor Ken Pease, of Huddersfield University, later captured the theory of crime reduction in a single sentence: "It is probably not going too far to say that the best strategy for crime control is now clearly a combination of proven techniques for the reduction of individuals' tendency to commit crime through intervention in childhood, and the manipulation of environments to make that more difficult."

By the autumn of 1998, both these initiatives had taken off. The thrust of the Morgan Report had been converted into a legal requirement to set up 376 Crime and Disorder Reduction Partnerships (CDRPs), spread across England and Wales, in which police were compelled to meet with their local social workers, teachers, housing officials and health workers to deliver crime reduction. They were to recruit the wishes of local communities to set their priorities, using their local knowledge and contacts to come up with the infinitely complex tactics to fit each area's own particular needs. And the researchers' radical findings had been developed into the outline of a Crime Reduction Programme. University specialists were hired to direct it and to check its progress. It was backed by the Home Secretary, by Downing Street and by the Treasury. For its first three years, from April 1999, the programme was given £250 million, which soon rose to £400 million with a huge investment in CCTV. Those who were involved speak of a buzz of excitement inside the Home Office - like a birth.

The baby was no sooner born, than it crawled into trouble. Ministers took a decision which, at the time, was controversial and which, in retrospect, was startlingly destructive. They ruled that the CDRPs should have none of the money from the new crime reduction programme. Not a cent. James Morgan had wanted each of them to be given a small budget, just enough to hire one or two staff, an office and a phone, to keep the wheel turning. Ministers, however, over-ruled Morgan's plan and insisted that police and local authorities must fund the new partnerships from their mainstream budgets. The practical result on the ground was that, without staff or cash, the CDRPs had no fuel.

That immediately exposed the CDRPs to a second weakness, which was to cause even more problems: most of the partners were not interested in partnership. The whole initiative was being launched on to a public sector which was exhausted by years of Tory spending cuts, none of which had been reversed. Demoralised and distracted by survival, numerous local authorities could not see what crime had to do with them. In some cases, too, the police were suspicious that this was all about invading their patch. In other cases, the police tried to kick-start the scheme, only to find that this alienated some of their partners, who saw their own patch being invaded.

If the CDRPs themselves had no fuel to run on, and if many local agencies were not willing to invest any effort or funds of their own, the crucial task of analysing local crime simply did not happen. In the autumn of 1998, they were told to provide local crime audits by April 1999, but the deadline arrived, with the alarming news that 117 of the 376 CDRPs had failed to deliver any audit at all. Among those who did, according to a Home Office review, many simply supplied an executive summary without any real data and, even those who tried to collect data found that their partners were not willing or able to provide it: only 38% of the audits had data on school truancy, only 27% of them had health data on drug treatment, only 27% had social services data on offenders.

This was a serious failure, right at the root of the evidence-based, problem-solving philosophy of the programme. Having snapped off the root of the new plant, ministers now dumped manure on it. From April 1999 they started to release the £400 million from the crime reduction programme: the intention was apparently benign, but the result was destructive. They broke the money down into a series of perfectly sensible priorities - burglary, CCTV, prostitution, drug workers, school management etc - but these did not necessarily overlap with local plans, so, for example, a rural CDRP which was most concerned with traffic speeding through its villages would get nothing unless it could find some sex workers to help. Second, they did not offer the money specifically to the CDRPs, with the result that individual local agencies reached out for the cash, bypassing the new partnerships. Finally, they handed out cash only in response to bids, and the most fragile CDRPs lacked the time and energy to submit them effectively. Many CDRPs continued without any funds at all. Others eventually received cash through the police to tackle problems which had never been their priority.

Less than a year had passed, and the whole structure of the CDRPs had already been weakened at the base by three problems which were to continue to haunt it: the clumsy decisions of ministers; the lethargy and weakness of local partners; the heavy hand of Whitehall control. Now, the top of the pyramid also became weakened as, behind the scenes in Whitehall, a kind of craziness took over the other flank of the new strategy, the £400 million crime reduction programme.

The engine house for the programme was the Home Office's Police Research Group which had been given £25 million to feed research to the police and to evaluate their work. Since 1983, the group had been part of the Home Office's police directorate, where it could work directly with forces around the country. Just as the new programme was launched, it was moved, against the wishes of many of its staff, and became part of the Home Office Research and Statistics Unit. It lost its link to the real world. Worse, the group then became caught up in feuding between civil servants in different departments of the Home Office who had an interest in their work, variously ignoring it or attempting to control it. Some junior civil servants were reduced to drifting into other departments to complain about the antics of their bosses. One official simply shut himself in his room to escape the tension. Disunited and dysfunctional, the Home Office then became vulnerable to the demands of the Treasury.

Some of the researchers wanted to go out onto the ground with the police, to help them run their projects; but the Treasury said they must stand back and make objective evaluations without interfering. The researchers wanted to spend the money slowly, partly because they had so much of it that they were running out of academics to spend it on, but, more importantly, because it was the core of the new approach to work slowly, gathering data, analysing it, searching for the precise strategy that would work. At one point, the Treasury agreed to drop their demand for 'annuality' - that each tranche of cash must be spent before the end of the financial year - but rapidly they reneged on the agreement.

One source who was closely involved describes the administrative equivalent of a three-way bar-room brawl: "There were the ministers who had basically said 'Get on with it', which was fine. There were senior Home Office civil servants who were just desperate to spend the money before the Treasury started clawing it back. And there were the researchers saying 'Hold on a minute, this is wrong.' The civil servants were saying 'Spend it. Just get the money out of here.'"

Arguably the most important single initiative was a £30 million programme to run problem-oriented projects with the police. Like the rural car park and the pizza parlour, this can be simple: Oakham Castle in Leicestershire, for example, suffered a long spate of broken windows caused by vandals chucking rubble at the building. For several years, the police tried and failed to control the vandals by occasionally hiding in the building and running out to arrest them - "both expensive and ineffective" as a Home Office report later concluded. Then they adopted the problem-oriented approach, thought laterally and instead of going after the vandals, they went after the rubble. It was swept away, and the windows were safe again. In other cases, it can be hugely complex.

The American criminologist George Kelling was recruited to advise on the epidemic of crime in the New York subway in the late 1980s. The police were trying to deal with the problem by harrassing homeless people who slept in the subway; crime kept rising. Kelling spent a year simply analysing the problem, asking who was in the subway, why, where did they live, what did they do. Armed with this data, he discovered that all of the police officers and pundits who blamed the robbery on homeless people were wrong. The crime was being committed by a relatively small number of people who had homes and who treated the subway as a private playground. Once he had that under his belt, he could devise a strategy which targetted law enforcement on the core of robbers, helped the homeless people through social services, alerted the whole community through a publicity campaign and finally cut subway crime by 80%. Kelling now reports that there are days when the whole subway system is entirely free of crime.

In Britain, Lord Justice Taylor's 1989 report into the Hillsborough disaster is another striking example of wiping away crime with a prolonged and detailed analysis of its roots: his advice that all football spectators should be given seats, instead of wandering loose on terraces, by-passed conventional law enforcement and transformed crime and disorder in stadia.

But the very complexity of the approach was a danger. HM Inspector of Constabulary had just echoed the warnings of a sequence of earlier research that the police generally had not mastered the problem-oriented approach. HM Inspector found ("Beating Crime", 1998) that in 32% of the problem-oriented projects he examined, there was no evidence that the supposed problem even existed; that only 27% of the projects showed evidence of proper analysis; and that only 7% claimed to have succeeded. The researchers warned their civil servants that the new initiatives would succeed only if the police were given two vital supports: time and training. But the civil servants said that was impossible: they had to spend the £30 million before the Treasury's deadline. The result was that police were invited to bid for the money and given only one month to come up with projects. And, in order to get rapid results, they were given no time or funds for training. The researchers watched in dismay as their most important project was launched into dangerously choppy water.

At the same time, they launched another key part of the programme - the Reducing Burglary Initiative, funded by £24million. The plan was to use the problem-oriented approach to develop a package of measures tailored to local needs in a strategy that had been startlingly successful in earlier pilots: in Kirkholt where they cut burglary by 75% over three years; and later in Huddersfield where they cut it by 30%. The police bid for the money in two rounds: 60 special projects which were to develop innovative tactics; a further 186 which would use a more traditional approach. The researchers wanted to divide all the money between the 60 ground-breaking schemes, but senior civil servants over-ruled them and divided the cash equally between all 246. Less than £100,000 each: it was not enough, the researchers warned, to guarantee success from any of them. And why were all of them being given the same money regardless of their scope?

By early 2000, some researchers were so despondent that they considered by-passing the civil servants and going direct to ministers to warn them of what was happening. But they shrunk back from mutiny. In the meantime, the CDRPs had hit a new wave of troubles. Ministers who had already damaged them by denying them fuel now grabbed for the steering wheel and deliberately pushed them off course. Under pressure from Downing Street and the Treasury, who wanted quick, measurable and electorally attractive results, they used the 1999 Local Government Act to require all CDRPs from April 2000 to set new five-year targets for vehicle crime, domestic burglary and robberies. There was no longer any pretence of being guided by the voice of local communities.

In the original CDRP audits, 87% had put domestic violence at the top of their list, but this was nowhere in the new key targets; only 8% of the partnerships wanted to focus on robbery, yet now it was one of the three main national objectives. Those CDRPs who had come up with the same priorities as the government had offered very different aims. The Home Office itself recorded that CDRPs chose targets for reducing vehicle crime by anything between 2% and 25%; Whitehall now ordered all of them to make a 30% cut. In the same way, the local schemes had set targets to cut burglary by between 1% and 16%; Whitehall ordered a cut of 25% from all of them. Beyond that, local needs changed, but Whitehall's new targets were set in stone for five years. And this whole move was produced, in the case of vehicle crime, by an unscripted public remark by the Prime Minister; and, with burglary and robbery, by internal negotiations between the Home Office and the Treasury. There was no attempt at consulting the CDRPs or even the police, many of whom to this day remain privately indignant at the distortion of their work.

Cash was now pouring downwards. Ministers belatedly heeded James Morgan's advice to fund the CDRPs, but now the funding became part of the machinery of Whitehall control. All ten regional governments in England and Wales were told to hire a regional crime director whose job was not simply to get the partnerships working but to ensure that each of them hit the Whitehall targets, which meant a regime of business plans and audits and data-collection - "shedloads of paperwork" as one senior police officer put it - and more and more staff to handle more and more paper in the regional crime directors' offices. Simultaneously, as this series has already described, the same regime of micromanagent was being imposed on the police and the Drug Action Teams, their working lives silting up with bureaucracy.

Beneath the surface, the impact of this central drive began to erode the foundations of the new approach, forcing police back onto the conventional tactics of arrest and conviction in order to deliver the rapid and measurable results which Whitehall was demanding. George Dixon was making his return.

And while central control was tightening, local arrangements were coming unstuck as every outpost of the public sector was re-organised. One partner would reach out for another only to see it lurching away like a drunkard dancng. When the 376 CDRPs were created in 1998, they were told to link up with the 370 police Basic Command Units (the new jargon for a police division). Within 18 months, the police had shuffled their internal boundaries so that there were only 318 BCUs: some of them included several CDRPs; some of them shared a single CDRP with several other BCUs; only 120 now matched the area of their local CDRP. In the meantime, the network of less than 100 Drug Action Teams was expanded so that by the autumn of 2001, there were 149, only a few of which matched BCU and/or CDRP bundaries. (In Wales, the five DAATs were no sooner up and running than they were abolished and replaced by 22 Community Safety Partnerships). In the background, there were 54 probation areas, none of which matched the boundaries of any BCUs or CDRPs or DATs; and in April 2001, they were rearranged into 42 areas. A year later, the structure of eight regional NHS offices was radically altered into a new national network of 304 primary care trusts. The education authorities which had been organised at county level were ordered to devolve funding and powers to individual schools. And annually, the government issued new targets, new guidelines, new funding streams, new law.

There is a perfect glimpse of partnership life on the ground in a recent report by the Inspector of Probation, Rod Morgan, who looked at what happened when the government told the probation service to run the new Drug Treatment and Testing Orders, which allowed courts to order drug-users to accept treatment. The order came through in June 2000. Probation set off in October without any infrastructure to run the scheme. By that time, they were half way into reorganising themselves for April 2001 into their 42 new areas, eight of which were amalgamations needing completely new budgets and plans. By December, the DATs, who were their key partners in the exercise, were also being reorganised - new teams, new boundaries, more new plans. In April 2001, probation were given £36 million to run the scheme. Weeks later, half of it was taken away from them and given to the new National Treatment Agency which was to run the DATs. More new plans - but not until the NTA agreed who was going to spend what, and they didn't start work until the autumn. That dispute was still running in early 2002 when the entire national health service, which was delivering the key treatment for the drug users, was completely reorganised - more new boundaries and plans.

By now, some of the DATs had decided that the DTTOs were too expensive and so they refused to hand over the money that was finally being passed down by the National Treatment Agency. And some of the new primary care trusts, struggling for funds, refused to hand over valuable beds to help law-enforcement when they needed them to hit their own targets. To deal with that, probation had to agree to let the primary care trusts assess the drug users, in addition to probation's own assessment, with the result that only 20% of the assessments were completed within the three weeks required by government. The system was up and running, but only 24% of offenders were actually undergoing the twice-weekly urine tests which lay at its heart - and almost half of them were failing their tests more than 90% of the time. Many probation officers registered this failure but decided it was best to allow the offender to continue with treatment rather than being shunted back to the dead end of imprisonment. Among those who were taken to court for their orders to be revoked, some escaped because there was nobody to impose the revocation.

Around the country, police officers were trying to link up with probation, only to find that probation were too short of funds to do their work; drug action teams referred addicts to treatment, knowing that there was almost no effective treatment out there; social workers sent excluded children for education, but there were almost no out-of-school facilities; courts jailed homeless beggars because there was nowhere else to house them. Even when agencies had the funds and the plans to work together, they could be prised apart by clashing targets from the centre - like the CDRP plan in Bristol to send drug users on DTTOs to live in a probation hostel. The hostel had space, local probation liked the idea, but they had a national performance target which required them to keep their hostels at least 90% full and, since chaotic drug users were the least reliable residents, they had to block the plan.

In the autumn of 2000, the Home Office received a clear signal that all was not well on the ground when HM Inspector of Constabulary and the Audit Commission jointly reported that the new CDRPs were struggling. They urged government to stop over-riding local plans, to measure police against 'community sentiment', to set crime targets for health and education. And, in particular, they urged them to extend the Crime and Disorder Act so that it required partnership work not only from local authorities but from all central government departments as well. The advice was rejected.

As time passed, the researchers began to see the results of the crime-reduction initiatives which they had launched into this confusion. Their prized £30 million scheme to promote problem-oriented policing had crashed into the same barriers which had obstructed earlier efforts. Without the time or training to do the job properly, schemes generally had failed to gather the right data, failed to analyse it properly and failed to device effective solutions. This project was central to the whole crime reduction programme. In a report published earlier this year, two researchers hired by the Home Office, Karen Bullock and Nick Tilley, recorded: "The conclusions are not encouraging."

They noted the destructive impact of the Treasury's timetable: "A much longer initial period would have been needed to make progress.... In so far as the funding regime itself provoked premature closure over problem-definitions and measures to address them, ironically it may, >>> Of course, have undermined the problem-orientation it was intended to promote." And they noted the equally destructive impact of ministerial priorities. British police were now employing more analysts for problem-solving, they wrote: "What they currently do, though, appears to be more oriented to a detection and enforcement agenda and to satisfying performance indicator requirements, than it is to an agenda concerned with dealing with police-relevant community problems."

The former head of the Police Research Group at the Home Office, Gloria Laycock, co-wrote an analysis of the failure, pointing to the capitulation of the police to Home Office pressure to carry on arresting: "There are far too many cases where persistent problems are ignored and where, even when the police do think about solving a problem, they fall back on the standard approach of arrest and prosecution." The bold new programme had wandered far from home.

The £24 million Reducing Burglary Initiative had also suffered from the same cramping of the problem-oriented approach and from the Treasury's attempt to stop researchers intervening to help the police. Three consortia of university specialists were hired to follow the 60 most innovative schemes, to measure changes in burglary rates and to check the funding against the £2,300 average social cost of a burglary. The results were far from successful.

One of the consortia, led by Professor Tim Hope of Keele University, reported that only one of its 20 schemes had secured a significant cut in burglary at a cost-effective price; five others had scored a less significant cut at less than £2,300 per crime. The rest had failed: seven of them had had a more or less neutral effect on burglary at an unacceptably high price, with one of them costing eleven times the £2,300 for each offence it cut; and, worst of all, seven other schemes had stimulated a clear rise in burglary, sending it spiralling upwards in one case by 39%, usually at vast cost. Prof Hope told us: "We found all the implementation problems on the ground that need time and thought and effort to get right. Most projects were on a hiding to nothing from the start."

Most of the schemes had made a particular effort to protect 'repeat victims' - the 4% of the population who suffer 44% of all crime. But Prof Hope's researchers found that, out of their 20 sample schemes, none had reduced the risk of these victims, and six had actually increased it. And this turned out to be a crucial clue to what was going wrong. Just as conventional law enforcement often failed because it assumed that offenders would behave rationally and be deterred, so now crime reduction discovered that the victims of crime would not necessarily behave rationally either.

Most of the burglary schemes had used 'target hardening' as the core of their strategy, offering free packages of extra security to any household which wanted it. The problem was that the most vulnerable households were also the least likely to accept help: the victims were alienated from officialdom and suspicous of the approach; the houses were owned by absentee landlords who could not care less about protecting their tenants; the victims lived chaotic lives in short-term housing and had moved before the offer was made. In some cases, when the police target-hardened the most co-operative houses in the area, the effect was to divert all the burglary onto the most vulnerable, with a dramatic increase in the suffering of the repeat victims.

This clue, in turn, led back to the structural weakness which had been inflicted on the CDRPs who had been created to run schemes like this with the police. The researchers found clear evidence that the most successful burglary projects were planned from the bottom up, using community knowledge and contacts; when they went wrong, they rapidly understood why and adapted. The least succesful ones were imposed by the police alone who sometimes saw the emerging failure, knew no way to adapt the plan and reacted by simply deciding to spend all the money before it was clawed back.

For several years, the Police Foundation think-tank have been trying to ring alarm bells warning that constables can be inherently conservative and even cynical and tend not to do very much about initiatives that are handed down from on high without any kind of consultation with them. A scheme works its way from Whitehall down through the ranks until finally it arrives on the desk of a duty sergeant along with a whole lot of other paper and becomes, for the constables, just the latest fad that they are expected to deal with. It is an exact echo of the problem of crime victims who do not co-operate with burglary initiatives which are passed down from distant desks. From the Home Office's point of view, abandoning the reality of consultation, it was the opposite of rolling a snowball down a hill and watching it gather weight, more like throwing a vase off a rooftop and watching people on the ground pick up bits and wonder where that came from.

Supporters of the burglary initiative were dismayed by the results but not defeated. The research had done precisely what it was supposed to do: by mid 2002 they had identified the problem and so now they could deal with it by strengthening the partnerships and using their local knowledge to adapt tactics on the ground. But now something else went wrong: the Home Office lost its nerve. They simply sat on the research, denying the police and the CDRPs the chance to learn the lessons. A year passed, and last month (June 2003), a brief summary was published amidst such controversy about its claims of success that Prof Hope's consortia demanded that its name be removed from it on the grounds that it was "highly misleading and selective".

At a conference on criminology last month, Prof Hope complained that the Home Office summary simply failed to say anything about whether the burglary schemes had been cost effective and undervalued the fact that burglary had been falling across the country, crediting the schemes with a success to which they were not necessarily entitled. The Home Office summary, he said, "produces results that at best contradict our own and at worst seriously distort the actual pattern of project outcomes as far as we can judge. These tend to point in a much more positive and favourable direction than our own." This looked, he said, like "the regrettably familiar government game of policy-led evidence". "What works" had started to give way to "Pretend it works."

In the background, the Home Office was rapidly discarding other elements of the new programme to beat property crime. In 1999, they commissioned a group of specialist insurance executives, police officers, fire officers and academics to advise them on imaginative new approaches to burglary, arson and criminal damage. They were known as P-crat, the Property Crime Reduction Action Team. In May 2001, they produced an initial report which was full of energetic ideas.

They wanted building regulations to include a requirement for all new homes to be secured against crime; a £1 computer chip installed in all new electronic goods so that they could be rendered inoperable if they were stolen; a national database of stolen valuables to make it harder to fence them; a new library of good practice in crime prevention; an expansion of the CDRPs to include more local agencies, including fire officers. They had already negotiated preliminary figures from insurance companies who were willing to slash premiums for home owners who took up their ideas and they had set up five working parties to carry forward their plans.

Their work was welcomed by the then minister, Charles Clarke, who said: "I look forward to the delivery of their full report and to doing all I can to help them to achieve reductions in property crime." But there never was a full report. P-crat was quietly disbanded. None of the ideas has materialised in the Home Office, although their arson ideas have been taken over by John Prescott's Arson Control Forum. One of those closely involved with P-crat told us: "There was an election; there was a new Home Secretary. The whole thing lost momentum."

At the same time, the Home Office had funded work by a Department of Trade inquiry, chaired by Lord Sharman, into technology and crime. They too produced innovative ideas, based on the advice of 60 experts and 26 seminars, to protect computer systems from hacking, to allow the police to record crime and to appeal for help through the internet, to build crime prevention into the life cycle of every product, and to tie central government into the kind of partnership work which was now required of the local CDRPs. They undertook to make sure that all their ideas were acted on. "Crime reduction is not the priority for everyone that it should be, and we need to change that," Lord Sharman wrote in his foreword to the report. Thus far, the government has set up funding for more research, but none of the other ideas has materialised. Sharman warned, in particular, that mobile phone companies should design their products so that they could be blocked if they were stolen: nothing was done until the theft of mobile phones soared.

Part of the difficulty seems to have been a resistance from mainstream business. One of P-crat's problems was that they pushed hard for a compulsory scheme for new houses to be 'secured by design' with high quality doors and window frames and estate lay-outs which expose thieves to the natural surveillance of neighbours. As a voluntary scheme, this scored startling success in the early 1990s, with some projects reporting that burglary fell to zero. But the big construction companies soon discovered that buyers were not willing to pay extra for the security and stopped co-operating. P-crat wanted to compel them, but one insider suggests that the steady fall in burglary since 1993 has weakened the government's will to take on the construction industry.

Some of the big car park operators have resisted a long-standing programme to win 'secure car park status' by installing CCTV and other crime-prevention, even though car parks are the site of 22% of all vehicle crime and the secure status cuts crime by an average of 70%. The government aimed to have 2000 car parks in the scheme by the end of 2000: more than two years later, they still have only 1,210. Like the hoteliers, car park operators profit from consumer law which omits them from the definition of 'goods and services' which would otherwise make them liable for thefts on their property. Similarly, at various points, the Home Office has looked at amending the 1956 Hotel Proprietors Act to stop hotels denying liability for the theft of their guests' possessions. In France, for example hotels are legally responsible and, therefore, are far more security conscious. But British hoteliers have dug in their heels.

Four years after its official launch, the crime reduction programme now is buried in a rubble of disappointment. But it is not dead. There are signs of some partnerships finally beginning to work. In Bristol, for example, where the CDRP was particularly weak in its early years, there has been a revival, brought about from an unlikely source. Tony Blair's 2002 Street Crime Initiative on mugging, which was a classic example of centralised control reinforced with a rigid performance regime, appeared to be a further blow to the CDRPs by imposing yet another law-enforcement requirement on the police, but ironically the sheer political pressure to come up with a result forced local agencies to work together. The chief constable of Avon and Somerset, Steve Pilkington, chaired a special leadership group to run the initiative: he pulled together the same agencies who were failing to work together in the CDRP and gave them a chance to succeed. That has now had a knock-on effect on the CDRP, which finally is showing signs of life.

And some of the schemes run by the crime reduction programme appear to have done better than their initiatives on burglary and problem-oriented policing. The Youth Justice Board has established Young Offender Teams and Youth Offender Panels in every area to divert juveniles from prison and into constructive schemes. Parenting Orders and Anti Social Behaviour Orders - both of which received a bad press when they were announced - have proved to be less repressive and more constructive than was feared. The huge Sure Start programme has generated projects around the country to work with pre-school children, on the same model as the successful American schemes, but the long-term nature of the work means the evaluation will not be finished until 2007.

And whereas P-crat was dumped, the equivalent action team for vehicle crime, V-crat, has done better, with a little help from the Department of Transport. Their key idea, of having immobilisers fitted to all cars so that they can not be driven away by thieves, fell foul of technical problems but they persuaded government to tackle the wrecking of stolen cars by scrap-metal firms and to make life more difficult for car thieves by improving the data flow to the police from the DVLA, car insurance companies and MOT records, with the aim of clamping down not only on drivers without correct paperwork but on the apparently large pool of cars which exist nowhere in official records and are routinely used for crime.

But the drum beat from the Home Office is not so encouraging. It recently posted a one-line message on its website, noting that the crime reduction programme which was originally conceived by researchers as a ten-year project, had closed in March last year after only three years. The Home Office say it has been replaced by a new funding stream, the Safer Communities Initiative, but its £20 million last year was a trickle compared to the original programme's £400 million. The officials who ran it have been dispersed throughout the Home Office. Some of the local schemes are still running, but there is no more central funding or training for problem-oriented policing or radical new initiatives on burglary. Indeed, the Treasury, which was blamed for many of the problems with the burglary projects, used the weak results to claw back £26 million which had originally been earmarked for a second phase of 250 more schemes. Funding for other schems including design against crime drained away last year. Crucially, there is no more central guidance from researchers; and no more central funding for the kind of evidence-based innovation which lay at the heart of the original programme.

The CDRPs themselves now benefit from direct funding and will receive some £84 million a year for the next three years - but this is a substantial cut compared to last year, because £25 million of their funding has been diverted to police BCUs. Although some local police commanders are now committed supporters of the CDRPs and will spend the money with them, others are being encouraged by the Home Office to spend it on DNA banks and car number-plate recognition systems and new finger-print databases, all the new technology of old law enforcement.

As the Home Secretary ratchets up his rhetoric for more detections and longer sentences, there are clear signs that the balance of policy has tipped back towards law enforcement. The crime reduction programme aimed to cut the prison population by 10%; but it has risen as the government pushes a conventional agenda. It has just set up new criminal justice boards across the country with the specific task of achieving its central performance targets.

The whole radical new strategy is now balanced between the power of its underlying thinking and the weakness of its implementation. The law still requires that police and local authorities work in partnership. There is still money flowing through the CDRPs. More than ever before, there are police officers and council officials and health workers and teachers and social workers who can see the value of solving problems instead of merely trying to punish their results. They can see that if police make the roads safer, hospitals will have fewer casualties to treat; that if schools work well with their truants, police will have fewer burglaries to handle; that if any of them can break through on drug treatment, all of them will be better off.

In a speech in Hackney last September, Tony Blair declared: "It is crucial to address poverty, social exclusion, unemployment, lack of education, the sense of hopelessness that is so often the breeding ground for crime." The message from the battered body of the crime reduction programme is that he can still do that, but his Home Office ministers too often have looked for quick fixes when the real solutions are long term; demanded measurable results when the real progress is often invisible; forced great simplicities on a problem which is notoriously complex; imposed rigid centralism where local flexibility is needed; and, most of all, failed to stand up for their project, failed to educate the public in the truth about the weakness of conventional law enforcement, failed to explain the virtues of their crime reduction programme and the new partnerships, and reverted to the easy populist rhetoric which made George Dixon a star.

Political Policing - how central government has gate-crashed the independence of police

The Guardian, July 2003

David Blunkett has not been getting on too well with his chief constables. Last autumn, for example, the Home Secretary unveiled his brand new National Policing Plan, which is to guide the 43 constabularies of England and Wales in all their efforts to deal with crime and disorder.

For many months, in private, the most senior officers in the land told Mr Blunkett their opinion of this central manifesto of the government's attack on crime. One chief constable captured for us in a single vivid sentence the message which his colleagues conveyed: "Frankly, sir, with respect, this is crap."

This friction is rather more than an argument among friends. It is a profound clash of ideas about how to generate success from the structural failure of the criminal justice system, which we described yesterday - a system which succeeds in bringing to justice only 3% of the offences that are committed around it, which then frequently fails to enforce its sentences against those whom it does catch and which routinely fails to prevent them offending again.

Faced with the limits of conventional law enforcement, this Home Secretary has inherited his predecessor, Jack Straw's, attempt to work outside the system with some radical alternatives, which we will examine tomorrow. But Mr Blunkett also has applied the most intense pressure to the existing system in an effort to force it to deliver better results. And it is here - in what Mr Blunkett likes to call 'narrowing the justice gap' - that he has found himself at odds with many chief officers who complain bitterly that his law-enforcement agenda, now captured in the new National Policing Plan, is narrow, simplistic and essentially ineffective.

This conflict of ideas is, in turn, a conflict of power. Chief officers around the country worry out loud that they are faced with a Home Secretary who is dealing with their doubts about his policy by trespassing on their operations, crashing through constitutional law in a way which, for the first time in this country, involves government in the political control of the police.

According to the rules, the Home Secretary is allowed to establish the level of funding and the strategic priorities for police, but that is all; it is for the local police authorities to decide how the funds are spent; and it is for the chief constables alone to decide how officers are deployed. This awkward balance is rooted in historic fear - that none of these three could be trusted with the power which police have to take our liberty and to invade our privacy. Now the balance is tipped towards the centre. As one chief officer put it: "We have always been answerable to the law - and only to the law. Suddenly, we are answerable to a politician."

This fretwork of tension has provoked several unseen crises. One of the earliest and most important involved the chief constable of Avon and Somerset. Steve Pilkington is one of a new generation of highly-educated chief officers who are willing to tackle crime and disorder with a far more sophisticated approach than their predecessors. Believing that there is more to policing than simply arresting and convicting offenders, Pilkington has championed 'geographic policing', using beat managers to build links with communities so that finally and genuinely, his officers can claim to be policing by consent; and he has trained all his officers in problem-solving so that they can try to cut crime before it reaches the criminal justice system, by working with other agencies to unravel the problems that lie behind the offence.

Pilkington's philosophy was tested to the limit late one night last summer when two men opened fire on a couple of his detectives, who were unarmed. The men with the guns ran, the detectives ran after them, radioing for help. An armed response vehicle caught up with them. A minute or two later, one of the gunmen was dead on the pavement. A police bullet had hit him in the back.

This was nightmare time for Avon and Somerset police. All this had happened in St Paul's, an impoverished mixed-race community in the centre of Bristol which is beset with crime problems. The man with the bullet in his back was black. St Paul's notoriously rioted in the early 1980s. As a young inspector in London, Pilkington himself had seen Brixton burn at the same time (he'd been caught in a van which was petrol-bombed). Those past riots had been triggered by heavy-handed policing. Indeed, one of the Brixton riots was caused directly by a police shooting.

Using the tools of conventional law enforcement, a chief constable would have reacted to the threat by flooding the area with officers, including riot squads, trying to use force and the threat of arrest to hold down the lid. Pilkington's officers did something very different. They had spent years trying to persuade this community to trust them, so now they put their trust back in them and simply gave them the facts. They set up a team of local leaders and a telephone tree to link up with their contacts in St Paul's and, through their beat managers, to people on the street. Each day, as they found out more about the shooting, they fed out more information: the two men were not local but Jamaican; they were carrying a loaded gun; they had fired on unarmed officers; witnesses had heard the armed response unit shout a warning; the men had ignored it and fired back. They held public meetings. Some officers left their private mobile phone numbers so that people could call them to check out any rogue rumour. Pilkington himself went down to St Paul's to talk to people. And... there was no riot, no disorder. Nothing.

In old-fashioned terms of arrest and conviction, this was a non-event, and yet it was a policing triumph. It was not simply that the facts were on the side of the police action, but that this community, which could so easily have rejected those facts, accepted them and accepted that the police were not strong-arm enemies - because Avon and Somerset had spent years building a relationship with the people they were policing. And the important point is that while Pilkington's philosophy was rescuing Bristol from riot, just about every chief constable in the country was watching the Home Secretary apparently attempting to sack him.

Within days of taking office, David Blunkett had made his mark by elbowing the chief constable of Sussex, Paul Whitehouse, out of his job, sending a clear signal to every chief officer in the country that, regardless of the legal nicety that it was up to the local police authority to hire and fire the chief constable, they were vulnerable to his will. Behind the scenes, he had taken two steps further to curb their power by creating a new regime of centralised control.

Clearly borrowing from his previous job as Education Secretary, Mr Blunkett transferred the role of the former Ofsted chief, Chris Woodhead, to a new role for HM Inspectors of Constabulary. Instead of reviewing the work of forces and giving advice to the Home Secretary, as they had done for years, the inspectors would now inspect individual Basic Command Units (the new jargon for police divisions). This was a slap in the face for chief constables who were by-passed by the process, but the punch on the nose came from Mr Blunkett's second new step - the creation in the Home Office of a Police Standards Unit which could be called in to any BCU which was deemed to be failing. This was the most direct threat to chief constables, usurping their authority over their own officers, cutting straight through their own policing plans and effectively requiring their resignation.

In July 2001, a team from HM Inspectorate of Constabulary, working to this new agenda, arrived in Bristol to review the work of the Central District BCU, which is responsible for the commercial heart of the city as well as the university, the pubs and clubs in the new '24-hour city' and a cluster of deeply deprived wards, including St Paul's. Before they even set foot in Bristol, the inspectors reached for the weapon which is now at the heart of the struggle between the Home Secretary and his chief constables: they asked for the numbers.

Chief constables, like the chief executives of every other branch of the public sector, are now surrounded by numerical performance indicators. In negotiations with the Home Office, they have expressed their deep and often bitter opposition to this. They have warned ministers that the numbers do not measure the reality of their work (the non-riot in Bristol is the exactly the sort of example they quote). Beyond that, they have complained that the numbers actively distort their operations, pushing them away from real local needs in order to hit national targets and giving them a perverse incentive to ignore serious criminals in favour of "hunting down the young and stupid, the impulsive, the mentally frail and the absolutely desperate", as one chief constable put it to us.

One of the most troubling examples has been the government's recent decision to require all police forces from April of this year (2003) to deliver an annual increase of at least 5%in the number of persistent offenders who are brought to justice. Apart from some strong reservations about the impact of this kind of conventional law-enforcement, the chief constables were prepared to go along with the idea: their intelligence networks identify the people who commit most of the crime on their patches and they want to focus on them. But the Home Office, attempting to satisfy the Treasury's demand for numerically measurable results, converted the initiative into a bureaucratic obstacle. They insisted that persistent offenders must be defined as those who had been convicted of at least six offences in the previous 12 months.

The chief constables angrily complained that that was a definition not of a persistent offender but of a stupid one; that it diverted their efforts from the clever offender who had evaded conviction and also from the imprisoned offender who was about to be released without any recent convictions simply because he had been locked up; that it invited them to go after the school girl who had been caught shoplifting six times but not the habitual burglar who had been caught only twice. Even worse, we have picked up reports from around the country of forces who early this year deliberately gave a holiday to their persistent offenders, choosing not to arrest them until April 1 when the counting of the new target officially started.

When HM inspectors descended on the central Bristol BCU, the National Police Plan was still in the pipeline, but most of its numbers were already pouring down from Whitehall, which had ignored almost all of the chief officers' warnings. These numbers dominate police strategy and permeate every corner of their decision making, but the numbers which matter most - the ones which Mr Blunkett was already waving in front of the press, the ones which Home Office ministers inserted into the new role of HM inspectors despite the opposition of chief constables - are the levels of burglary, car crime and robbery. Those were the numbers which the inspectors asked for before they even arrived in Bristol. This was bad news for Steve Pilkington and the central Bristol officers.

For several years, their own approach had been cutting the local crime numbers but in early 2001, they had suddenly leaped upwards - by 27.3% in only three months. In part, this was just numbers, nothing to do with reality: Avon and Somerset had adopted new rules for counting crime, throwing out years of dubious practice and simply recording every allegation from a member of the public as an offence which needed to be cleared up. Pilkington had opted to do this before most of the country and he had made it very clear to all his officers that he wanted this done ethically. In a deliberate break from the national police history of fiddling figures, he insisted that Avon and Somerset crime statistics must tell the truth. That alone had sent the numbers jumping upwards by some 10%.

In addition, central Bristol had been invaded by Yardie gangsters who had arrived at the end of 2000 and embarked on a campaign of casual violence which had seen opponents abducted, stabbed or shot (sometimes as a reprisal for some gang dispute back in Kingston) and who had also succeeded in boosting the market in crack cocaine, whose heavily-addicted buyers were now frantically burgling and robbing and breaking into cars to pay their Yardie suppliers. Pilkington's officers were fighting back, but they had a real struggle on their hands.

That year of the Yardies' arrival, 2000/01, saw a 72% increase in serious crime in central Bristol - a 52% increase in grievous bodily harm, a 100% increase in kidnaps, a 128% increase in abductions, a 75% increase in rape. The government's most favoured targets were doing just as badly - robbery, burglary, theft of cars and from cars were all substantially higher. The impact had spread across the whole of Avon and Somerset.

The chief constables had warned the Home Office that random local events - often much smaller than a Yardie invasion - could have a dramatic effect on crime figures. One problem family could move into an estate and send BCU burglary statistics through the roof. One football riot could distort the whole BCU picture on crimes of violence. Random events could cut the numbers too: robbery statistics in the BCU in Somerset East, for example, plummetted when the Glastonbury festival was cancelled by the foot and mouth epidemic.

The Association of Chief Police Officers had been particularly scathing about the Home Office plan to group BCUs together in 'families' which were supposed to be similar in terms of miles of motorway or rates of unemployment, but which were often quite different in terms of crime. ACPO even commissioned research from the Police Foundation think tank, who ran a binary logistic regression and found that fewer than 5% of the 318 BCUs in England and Wales had low clear-up rates for burglary or robbery which could not be explained by local factors.

All these underlying difficutlies now came to the surface in Bristol. Months before HM inspectors arrived, Steve Pilkington and his senior officers agreed that the only way to tackle the sudden crime boom in central Bristol was to tackle its underlying cause - the Yardies. If they could do that, they could kill off the motive that was driving something like 70% of property crime in the city. That meant more than just arresting and processing Yardies. They would work with the immigration department to stop the Yardies wandering through the airports where they were sometimes spotted and stopped only to be released a few hours later because there were no holding cells. They would negotiate with customs too to work with them as the Yardies' mules arrived in the country: there had been examples of police in various parts of the country tipping off customs with specific details of mules flying in only to find that customs took no action because their own performance targets diverted them into the interception of much larger drug consignments. They would also carry on working with the community, so that they accepted their work without seeing it as a threat.

But this was Avon and Somerset's approach - not the Home Office's. With this strategy, there would be no spare officers to run any kind of campaign for Mr Blunkett's car crime target. If the strategy worked, it would work slowly, waiting for immigration and customs to play their parts, potentially failing to yield the burglary figures as quickly as the Home Secretary wanted. Fundamentally, the operation against the Yardies was part of a wider effort to work outside the frame of conventional law enforcement by taking a 'problem-solving approach', recognising that most of their persistent offenders were drug addicts and/or illiterate and/or homeless and/or mentally ill and/or jobless and that there was simply no benefit to the community in arresting them yet again and sending them off on a tragic roundabout of ineffective consequences.

As a related example of problem-solving, the local commander at the Central Bristol BCU, Chief Superintendent Mike Roe, wanted to extend the network of beat managers into local schools, not to arrest children but to spot those who were getting into trouble and to work with them and their families before they slipped all the way into crime. But Roe could not do this on the scale he wanted while his officers were being diverted by Mr Blunkett's targets. And, even if he found the officers, the impact would not show up in the crime numbers for years. Mr Blunkett wanted his results before the next election.

When HM inspectors produced their draft report, in September 2001, they scarcely noticed the force's problem solving. The inspectors' report, which we have seen, made no allowance for the new counting rules or the Yardies. And they made a fetish of the numbers. They said Central Bristol had the worst numbers per thousand of population in its family of 17 BCUs - even though they had failed to count more than 100,000 commuters, nightlifers and students who came into the area, all of them potential crime victims and offenders, but none of them on the electoral roll. The inspectors went on to say that Central Bristol was underperforming compared to the other 17 BCUs in its family, even though none of the others contained a city centre (15 of them were suburbs, 2 of them provincial towns) with all that that involved for late-night drinking and crime; and none of them had been invaded by several hundred armed Jamaican gangsters with a ready supply of crack cocaine.

As part of their attempt to police in the way that their community wanted, the BCU had set up special units to deal with domestic violence, sex offences and racist crime. Officers were enormously proud of their success, but in the world of New Labour numbers, their success looked like failure, because the victims of these crimes were now trusting the police to deal with them so they were coming forward and reporting more offences. Women who are battered by their partners, for example, typically suffer 35 assaults before they approach the police. Now, Bristol women were coming forward much quicker. It was the same with race crime and sex offences. But HM inspectors complained that their numbers for crimes of sex and violence were increasing and stood well above the other 17 BCUs.

Faced with the impact of the Yardies, central Bristol officers were working long hours, but instead of reflecting on the difficulty of dealing with a huge increase in crime without an increase in resources, the inspectors simply asserted that manpower should be better managed. The inspectors produced eleven coloured graphs and two statistical tables, they worried about the absence of 'robust audit trails linking targets and objectives through to operations and their outputs' and they went on to make seven recommendations, dealing with internal structures, the cycle of meetings and the monitoring of performance.

For police officers on the ground and for residents of central Bristol, it was a report which had no link to the real world. What good were meetings and monitoring in the face of a Yardie crime wave? (And, indeed, how could the Home Office attack the police when it was their immigration service which was failing to stop the Yardies and the mules pouring through the airports at will, carrying an estimated 4.5 tonnes of cocaine a year?) How could they possibly enjoy the consent of the people they were policing if they started dancing to a political tune? Avon and Somerset officers wanted the report re-written, but HM inspectors stuck to their new agenda and insisted on publishing it without any significant change.

Now, chief constables around the country were watching closely. Their concern was not just that Pilkington was in danger but that all of them were in danger from a Home Secretary who was apparently willing to upset the delicate balance of power over their work. They had seen the danger dawning before Blunkett took over the Home Office. Some of them had been deeply opposed to Jack Straw's idea of a new national police radio system, Airwave - the biggest capital investment ever made by the police in this country. They said it was unnecessary and grossly overpriced but when they took their complaints to the supposedly independent Police Information and Technology Organisation, who had to make the decision, they found that Straw's officials had gatecrashed and were insisting that the scheme must be adopted - because it was the biggest PFI scheme in the government's gift and, regardless of benefit, regardless of cost, it must go ahead. And it did, in 2000. (The deal has subsequently been mauled by the Public Accounts Committee as 'disastrous', by the National Audit Office for its 'lack of competition', and by some experts in pulsed frequencies as a health risk comparable to asbestos.)

In the same way, as David Blunkett was closing in on Steve Pilkington, his officials were intervening in the supposedly independent Police Negotiating Board, some of whose members report that, while they were attempting to strike a new deal for police pay in the normal way, the Home Secretary was dictating terms and using leaks to the press to undermine the board's position. "It was outrageous," according to one board member. "There was one Sunday morning when we were pursuing our position with the staff organisations - and there in the Sunday papers was the Home Secretary putting completely different points. We are supposed to be independent." One result was that the Home Secretary forced a national cut in police overtime, even though that was the fuel which police were using to hit his targets.

But what worried them most was that the Home Secretary was top-slicing their money: instead of handing cash down to the 43 forces, he was holding more and more of it in the centre, inviting bids on which he and his ministers would make decisions. This stole power from the police authorities as well as the chief constables. And some of these central decisions were unwelcome. For example, David Blunkett wanted forces to hire Community Safety Officers as a cost-effective way of putting uniformed patrols on the streets. This was popular with some of the big city forces who could provide rapid back-up to the CSOs, but it was despised by most chief constables who believed uniformed patrols were a politically popular waste of time and who feared that untrained CSOs could easily cause friction. And yet they needed the money, so they bid for the Home Office's pot of CSO cash. Kent, for example, bid for 18 CSOs even though their chief constable, Sir David Phillips, as president of ACPO, had said publicly that he thought they had 'very little merit'.

It was not just that specific decisions about deployment were being stolen by the Home Office. The top-sliced funds tipped the whole balance of power. Crucially, David Blunkett had made a great public splash about increasing the number of police officers, but all of the money for these new recruits was held centrally and since every chief constable was desperate for more officers, all of them were vulnerable to Home Office pressure. (Not one chief constable would talk to us on the record about the Home Office for fear that their force would suffer. Even retired chief constables were worried. Almost all of them used the same word to describe the Home Office's attitude to them - 'spiteful'.) It was in this context of centralised power that the numerical targets became such a powerful tool.

There was now a real risk that Avon and Somerset were about to have one of their BCUs taken over by the new Police Standards Unit, that Steve Pilkington would be forced out of his job, that the whole community-based, problem-solving approach to the policing of Avon and Somerset would be crushed. Avon and Somerset decided to play the Home Office at their own game: they would accept all the recommendations of HM inspectors; they would rearrange their meetings and their monitoring and their audit trails; and they would let events prove the point for them. They warned Home Office ministers directly that unless they admitted that there was a national crime problem with Yardies and crack cocaine, not only would HM inspectors fail to help central Bristol but the government nationally would fail to reach its own crime targets.

The following summer, July 2002, HM inspectors published a draft follow-up report on the Central Bristol BCU, in which they patted themselves on the back, crediting their first report as 'a catalyst for change', and noted that the BCU had done all that had been asked. This time, they acknowledged that the BCU's similarities to the others in its family were 'somewhat limited' but neverthless produced eight different graphs to make comparisons. This time, they acknowledged that new counting rules had had a 'serious impact' on their figures but nevertheless used the figures to conclude that this was 'what can only be described as a comparatively poor crime performance'. This time, they noted 'the influx of drugs into the Bristol area, mainly through couriers from Jamaica' and recorded the BCU's strong view that they could not do much unless the government did something to stop the Yardies entering the country. But nevertheless they went ahead and delivered the judgement which most worried chief constables around the country: the inspectors suggested that the Police Standards Unit should be called in to the BCU.

At this point, David Blunkett was finally confronted by one of the many significant differences between education and criminal justice: the police are organised. The teacher unions were divided and swept aside by Blunkett. The police groups are much tougher. The Association of Chief Police Officers, in particular, runs a 100% closed shop, it has powerful political allies in Fleet Street and Westminster, and more than any other group of organised labour in this country, it has succeeded in infiltrating its employers by inserting its various sub-committees as national policy-making bodies. ACPO covers 43 forces, each of which has its own structure and philosophy, some of which have no time at all for problem-solving and geographic policing, but all of whom were threatened by what was happening to Steve Pilkington.

Within days of the inspectors' follow-up report, the Police Standards Unit were in touch with the local police authority in Avon and Somerset to inform them that they would be making 'a formal intervention'. Members of the police authority found themselves being phoned by MPs who were hostile to Pilkington (several chief constables around the country report being 'bullied into line' by local Labour MPs, apparently acting at the behest of the Home Office). Some members of the police authority were upset that they were being over-ridden: if the chief constable really was at fault, then it was for them to act, not the Home Office.

Their chairman, John Christensen, insisted on meeting the head of the standards unit, former police superintendent and businessman Kevin Bond. Christensen, knowing that this meeting could set a national precedent, took with him the head of the national Association of Police Authorities, Melanie Leech. Bond listened to them and undertook to write to Christensen to spell out what was happening.

In the meantime, the then president of ACPO, Sir David Phillips, travelled from Kent to Bristol to find out what was happening. According to ACPO sources, he left the city appalled at the naivety of HM inspectors and determined to stop the standards unit moving in. The ACPO machine started to mobilise. According to one source, they confronted senior Home Office officials with a passionate complaint about the Home Office's theft of power from chief constables and police authorities and with the specific prospect that if Pilkington was ousted, he would be entitled to sue them, thus making a public spectacle of the power struggle.

Behind the scenes, Sir David Phillips is believed to have visited HM inspectors at the Home Office, where he warned that they were on the brink of a move which could be deeply counter productive. Some in HMI were not keen to encourage the standards unit, which was trespassing on its patch. Sir David is believed to have pointed out that the standards unit could not intervene unless HM inspectors formally recommended it. Although they had written their report, the inspectors had not yet made the formal move. And they never did. At the standards unit, Kevin Bond never wrote his promised letter to the Avon and Somerset police authority. HMI's follow-up report was stamped 'confidential' and, unlike the orginal report - and in breach of HMI's stated policy - it was never published. Steve Pilkington, who had been hanging over the edge of a precipice, was hauled back to safety.

As a compromise, the standards unit agreed to 'collaborate' with the central Bristol BCU, offering advice and even helping them to access new money, but not formally intervening in their strategy. Since then, the Home Office's own researchers have confimed what central Bristol was trying to tell them, that there is a national surge in the black market for crack cocaine, which is jeopardising the whole of the government's crime strategy. The Home Office, in December last year, belatedly produced a national crack plan.

In central Bristol now, the crime figures are falling again. Sticking to their problem-solving approach, Avon and Somerset sent officers over to Jamaica and brought Jamaican police to Bristol to spot known offenders. They intercepted £10 million of Yardie cash leaving the country; closed down the bogus college which was giving cover to Yardie mules entering the country as students; forged links with immigration and customs; and saw the Home Office belatedly bring in visas which may make travel more difficult for the Yardies. They also worked so closely with the community that they were able to put armed officers onto the streets without losing their support. The result of their work is reflected now in a newsletter version of their annual report showing sharp falls in the number of crime victims in the BCU, under the headline "Partnerships - the key to cracking crime."

Nobody at Avon and Somerset wants to dig up the past. The head of central Bristol BCU, Chief Supt Mike Roe, stresses the good side of what happened: "It is true that we were disappointed with the HMI report, that they did not see our crime in its context and did not make the link to the Jamaican organised crime groups. But at the end of it all, you could argue that HMI did help us to do our job better and the standards unit also had a positive role to play. They did support us, they also got us extra funds. "

In the meantime, the performance regime has been tightened still further. It has been recast into a new Policing Performance Assessment Framework which sets yet more targets under five new domaine headings (it was six, but they cancelled 'Helping The Public'), all still grouped into 'families' of forces so that their average performance can be compared. The government has just created a new national network of criminal justice boards, pulling together the chief officers of every criminal justice agency in every region in order to ensure that all of them meet their numerical targets.

And this year, at the behest of the Treasury, Activity Based Costing is to be introduced across the country, requiring that each year, for a three-week period, every police officer should pause every 15 minutes in order to fill in a card to summarise the work they have been doing so that Whitehall can monitor efficiency. In diplomatic words, HM Inspector of Constabulary, Sir Keith Povey, noted in a recent report: "This can be regarded as absurd".

And this whole regime is enforced with the same engulfing bureaucracy which we described earlier in this series with the Drug Action Teams. Police officers meet as a senior management team and work their way through service level agreements and tendering protocols, producing business plans with audit trails and budget codes and business score cards to monitor the result. They bid for funds under dozens of different headings, wait months for approval and then scramble to spend the cash before the financial year ends.

There are very few police officers now who do not expect to be held to account for their work. And they understand the importance of numbers - as a clue to crime patterns, as an intelligence tool to steer their work, not as a concrete overcoat. Earlier this year, the chief constable of Thames Valley, Peter Neyroud, had the courage to announce publicly that he was setting aside Mr Blunkett's targets for burglary and robbery. He said they "fly in the face of reality" and threatened his ability to protect his community from other crimes. Other chief officers in mute rebellion have quietly adopted their own targets.

Down on the ground there are people who know a lot about crime but who are running a criminal justice system which does not deliver results. In the centre there is a government which is desperate for results and therefore takes over from the people on the ground even though - and here's the point - they know just about nothing at all about crime and even less about the different needs of different communities. Pounding a system to deliver results which are beyond its reach, the horrible outcome is that the Home Office are actively making the fight against crime more difficult. The symbol of David Blunkett's police service is a blue light wrapped in red tape. "Frankly, sir, with respect, this is crap."

Additional research by Tamsen Courtenay

Street Crime

The Guardian, July 2003

In April last year (2002) Tony Blair launched a crusade against street crime. He personally chaired eight meetings of ministers and chief constables, which chose to spend £261 million on a concerted drive to arrest, try and convict street thieves in the ten forces where the problem was worst. Blair assigned a minister to each of the ten forces and made it their personal responsibility to deliver results and then declared publicly that they would crack the problem by September - only six months after the initiaitive started.

This was the most acute example of the government intervening directly in the work of the supposedly independent chief constables. It threw their own carefully-drawn local plans into confusion. Ordinary divisions were forced to donate officers; traffic policing in some areas was cut back to the bone as more manpower was diverted. The intervention reached a pitch where, for example, according to Police Review magazine, civil servants in London were calling police in Merseyside to tell them to double the number of officers working on the initiative, a degree of central control which would once have been dismissed as unlawful.

But did it work? When September came, the Home Office announced that the project had cut street crime by 16%. The truth was not quite as simple as that: there was success but there was also failure. The whole initiative is a snapshot, first of the inherent weakness of the criminal justice system as a mechanism for cutting crime; second, of the inherent difficulties in allowing politicians to drive supposedly independent local police forces from the centre.

The Home Office scored its 16% success by fudging the figures in two ways. First, they moved the baseline. When the initiative was launched, the ten chief constables were told that their performance over the six months from April 2002 would be measured against their figures for the same period in 2001. But when they announced the results, the Home Office abandoned that plan and took advantage of the fact that street robbery had risen right up to the beginning of the initiative: they used the figures for the single final month before the initiative to give themselves a higher baseline from which to measure their result. If they had stuck to the original plan, they would have recorded an improvement of only 10%.

Second, the improvement was scored only by smearing together the figures for all ten forces. If you look at each force separately, you find that only five of the ten saw their robbery figures improve compared to the same six months of the previous year. Of the remaining five, Manchester managed to cut the total number of street robberies by a statistically insignificant 0.14%. But West Yorks' street robbery rose by 7%; Notts soared by 19%; and Merseyside and South Yorkshire each shot up by 22%. No minister high-lighted this disappointing outcome.

Furthermore, the original brief for the Street Crime Initiative explictly included "the illegal possession or use of firearms that threatens the safety of our streets". The Home Office simply ignored this whole area in its claim of success. Four months later, two Birmingham teenagers were shot dead at a party, and new crime statistics showed that gun crime had increased by 35% in the last year. Publication of HM Inspector of Constabulary's report into the initiative, due in April of this year, has been delayed.

The bottom line is that the six-month blitz succeeded in cutting the number of recorded street offences by 5,068 compared to the same period a year earlier. With a total bill of £261 million (which was spent through agencies across the whole criminal justice system) each offence which was prevented came with a bill to the taxpayer of £51,500.

That figure is a little blurred. It may understate the real cost, since it is clear that some forces cut their street crime figures by changing the way they record offences; and there is some evidence that, where the street crime was being committed by drug users funding their habit rather than by schoolchildren stealing mobile phones, the crime was displaced, with burglary and car crime rising.

On the other hand, some of those costs may have a longer-term benefit, because the street crime initiative was extended beyond its original six months; and there was a hidden profit in some areas where the sheer heat of the initiative finally forced local authorities to forge real working partnerships with their police.

Several of the chief constables who were most irritated by the initiative now acknowledge that it had its virtues, not least that it gave them extra money. And yet they remain deeply worried by the government's narrow focus on trying to cut crime by driving up detections; and by the role of politicians in disrupting local plans which draw on local knowledge and police expertise. One told us: "The Street Crimes Initiative has arrested hundreds of people and charged most of them, but we have probably arrested them 15 times before. We can arrest them 20, 25 times. It won't make a difference. As long as they are on the streets, they commit crime. While they are behind bars, they stop. And then they come out and start again.

"The Home Office were wedded to the idea of arrest before they even started talking to us about it. They are advised by people who do not understand the system. I have never seen any evidence to justify the policy of arresting persistent offenders - it does not work in the long term. It is irrelevant."

Additional research by Tamsen Courtenay

The real roots of crime

The Guardian, June 2003

They hanged Huffum White in August 1813. When he was offered a last wish, Huffum told the priest he'd quite like somebody else please to take his place on the scaffold, but criminal justice had its way with him and celebrated a great achievement: they had just hanged the last highwayman in England. In truth, it was no achievement at all.

The passing of Huffum's trade had very little to do with hangmen or magistrates or parish constables. Indeed, it is an object lesson in the limits of conventional criminal justice. Huffum's colleagues were born from a co-incidence of history: the growth of a national network of horse-drawn coaches carrying the wealthy and their cash; and a vigorous crop of Royalist ex-soldiers who had lost their homes and their prospects of a lawful life in the civil war.

For 150 years, generations of highwaymen robbed their way around the country while the forces of law and order struggled to deal with them - hiring informers, offering rewards, giving full pay to constables (who had previously caught only those robbers who carried a reward), hanging them in public and then, adding horror to death, ordering that highwaymen be left to dangle in chains, rotting on the gibbet. Really, none of it worked. The highwaymen carried on robbing.

Huffum's trade died, as it was born, from forces well beyond the reach of the criminal justice system. The growth of banking and the introduction of cheques cut the amount of cash in transit. William Pitt's Act For Restricting Cash Payments (designed to gather gold for his war against France) cut it still more. The enclosure acts swallowed Hounslow Heath and Finchley Common and the other common land which had provided the highwaymen with cover. And then the railways diverted travellers from the highways. So, Huffum hanged alone.

All crime grows and dies beyond the reach of the state. It is arguably the most essential human perversity. What is obvious is that no law-and-order initiative is going to touch the social chemistry which produces highway robbery or piracy or drug dealing or any other major crime pattern. It's like asking doctors to prevent car crashes - they can't change the construction of motorways, or the design of cars, or the habits of drivers. What is less obvious is that when the law-and-order machine goes to work on those things which it can touch - when it sets out to catch and punish offenders - it also tends to fail, in a way that generally speaking doctors will not. This is because (to pursue the medical parallel) modern criminal justice still relies on the equivalent of leeches and bleeding. The system may or may not be badly run; it might or might not do a little better if it had more resources but at root, it tends to fail because it uses failed tools.

Conventional criminal justice rests on the foundation of three Big Ideas which prove to be particularly weak when they are used against the offenders who matter most, the persistent offenders or 'lifestyle criminals' who amount to only 20% of the offending population but who account for 80% of recorded crime.

First, ever since 13th century China, the state has been sending its uniformed watchmen out to patrol the streets, to detect villainy and to preserve the peace. In Kansas City, in1972/3, police decided to test the value of this. They divided their beats into three areas in which patrols variously were trebled in frequency; removed completely, with officers entering the area only if they were called; or simply left at the previous level. They then spent 12 months watching 648 different indicators of crime, fear of crime and attitutudes to police. The conclusion: the patrols made no difference. Police could flood an area, disappear or carry on as before - their visible presence on the street changed nothing. Six hundred years of patrols. For what?
This is not news to most chief police officers in this country. The recently-retired president of the Association of Chief Police Officers, Sir David Phillips, publicly derided the idea that bobbies on the beat are an effective device to prevent crime or catch criminals. He said last year that the whole idea came from an 'Enid Blyton world', whereas, in the real world, he said, most forces could put only one officer on patrol for every 14,000 people.
But, >>> Of course, there is more to policing than patrolling. The second fundamental, though more recent, activity of police is to use detectives. We obtained an ACPO internal paper, which analysed the crime rates in three-quarters of the police divisions in England and Wales and came up with a devastating conclusion. They discovered that detecting villains does not cut crime: "It is suggested that increasing detections will reduce crime. This intuitive appeal is not supported by the evidence." The paper went on to consider the impact of detections on the Home Office's two other major priorities - reducing anti-social behaviour and the public's fear of crime - and it noted that not even the Home Office had been able to find any link.
Indeed, the paper concludes that the reality of detective work is exactly the opposite of popular (and official) belief: detections do not drive crime rates up or down; it is the crime rate which drives the detections. Simply put, this means that if there are lots of fish in the barrel, you will shoot a lot of fish. Or, as the ACPO research concluded: "We find a strong positive relationship between total offences and total detections as it stands to reason that the more offences you have, the more there are available to detect. As total offences increase, so do total detections."
That is why, for example, police have increased their detection of robbery by 500% in the last 15 years, while the amount of recorded robbery has soared by 1000%; and, conversely, why the police detection of burglary has slumped from 31% to only 12% at the same time as the amount of recorded burglary has fallen from 1.2 million to 878,000.
We obtained another internal ACPO paper, in which the chief constables stood back and looked at the system as a whole in its dealings with lifestyle criminals. According to the paper: "All too often they can become untouchable and are amenable to justice only when they are unlucky....They are not amenable in general to traditional police enquiries, and other agencies ‘back off'.....Routine policing will not affect such criminal activity in any significant way." Just absorb that last sentence, written by the most senior police officers in this country about the offenders who commit 80% of our recorded crime.
Third, look more closely at the core concept of the system, the idea of punishment and, in particular, of imprisonment, and consider the extraordinary experience of Prof Jerome Miller, now of West Virginia, formerly of the state of Massachusetts, where, during the early 1970s, he was Commissioner for Youth, responsible, among other things, for some 2,000 young offenders who were then locked up in the state's juvenile prisons. Prof Miller did something which is probably unique in the developed world: he opened the prison doors.
He told us he did this because conditions were so bad that it was the only move which he could make to protect the juvenile inmates. State law gave him the power. The state governor gave him support. And so he released them, all but a dozen whom he considered to be demonstrably dangerous. And to make sure that nobody tried to put them back, he sent bulldozers to flatten most of the buildings in which they had been held. There was quite an outcry, there were scare stories in the press, and the police and the courts reacted by redoubling their efforts to arrest more young offenders and sentence them to custody. Miller simply turned them loose and diverted the cash from his prison budget to providing jobs, housing and supervision for the adolescents he had released. And, >>> Of course, there was no crime boom in Boston.
The National Council for Crime and Delinquency followed the crime careers of Miller's juveniles over the following ten years and compared them with juveniles who had served their full time behind bars. Essentially, they found there was no difference: the worst of them, from both groups, carried on offending; but there was a clear tendency for those in Miller's group to commit less serious offences while those who had been in prison tended to commit increasingly damaging crimes.
Between 1950 and 1990, England and Wales doubled the number of people who were locked up in jail - and yet the rate of recorded crime roared upwards, increasing ninefold. (These figures discount the underlying growth in national population). In the United States, the courts are incarcerating offenders at six times our rate; a third of young black men in the US now are either in prison, on parole or on probation; California alone has the third largest prison population in the world. And yet, the USA remains the most violent society in the developed world with a murder rate seven times higher than the UK. (For a complete analysis, see Crime and Punishment in America by Elliott Currie.)
Prison succeeds in the limited sense that an offender behind bars is not free to commit his crime. As a means of changing the behaviour of offenders, incarceration is peculiarly ineffective. The Home Office's own research department concluded in the early 1990s that to cut recorded crime by just 1%, they would have to increase the prison population by 25%.
The heart of the whole problem is that the system makes a set of assumptions about the behaviour of regular offenders - that they are making rational calculations about their behaviour, that they are worried about getting caught and that they are fearful of being punished. Those assumptions may apply to the law-abiding majority, but they are overwhelmingly false in relation to the generation of adolescents, usually male, who are based in the wreckage of the old public housing estates, whose values have been distorted by a childhood in collapsing communities and broken families, and whose ambitions have been swallowed by the one style of life which offers them status, excitement, a decent income and the prospect of promotion - crime and particularly the blackmarket in drugs. These are the lifestyle criminals who commit 80% of recorded crime: patrols don't inhibit them, detectives don't catch them, prisons don't deter them.
There is no shortage of politicians and thinktanks who are happy to cherry-pick their way through the facts to defend conventional law and order. But where is the evidence that this huge system with all of its resources and all of its power is an effective method of cutting crime? The ACPO paper on detections notes that there is only one piece of serious academic research which this government likes to quote in defence of the system (Criminal Deterrence and Sentence Severity by Von Hirsch et al). This does find a correlation between a higher certainty of arrest and lower crime rates but, as the ACPO paper notes, the reseachers explicitly warn that "this does not suffice to confirm a deterrent effect." Which is why they could hang Huffum, but they couldn't halt the highwaymen.

(1,866)

Failing schools and failing solutions

The Guardian, July 2000

In the bizarre world of Britain's target-driven schools, it is not only teachers who have joined children in cheating to get good results. The Department for Education are in there, too.

We decided to test the DFEE's claims to be 'turning around' failing schools, by analysing the academic results of every secondary school which has ever been put into 'special measures', the programme of intensive reform and inspection which, according to the repeated claims of the Education Secretary, sets schools "back on the path to success". "We are turning them around more quickly than ever," he declared last year.

In the latest available list, there are 166 secondary schools who have gone into special measures. The first point is that seventy per cent of them are either still in (87, including one that has been there for six years) or they have closed (29, including nine that have been re-opened as Fresh Start schools, which we examine later). Setting aside one school which has been merged, we looked at the remaining fifty for the signs of success which are celebrated by the Secretary of State.

We found that in the year before they went into special measures, on average only 13.24% of their pupils were scoring at least five A to C grades at GCSE, the government's chosen measure of academic success. This was seriously low. Mr Blunkett announced in March that in future he would consider closing any school which failed to deliver at least 15% A to C grades. Then we looked at the average achievement of pupils in each of the schools for every year since it went into special measures and found out that Mr Blunkett was in some difficulty. On average, in these schools which are "back on the path to success", there have been only 13.66% of pupils scoring five A to C grades - well below Mr Blunkett's threshold for survival.

This tiny overall improvement has been secured at an estimated average cost of £500,000 per school, a total bill of £25 million. And it has taken place against a background of intense stress for teachers and heads, some of whom have lost their jobs in the process; and tumbling morale not only amongst staff but also among parents, some of whom have reacted to the imposition of 'special measures' by withdrawing their children.

>>> Of course, our averages refer to schools of different sizes and they conceal wide variations. Among the fifty, there are some who have moved upwards sharply: St Mary and St Joseph's in Bexley was scoring 35% A to Cs and now scores 45%; Hayes Manor in Hillingdon was scoring 18% and now scores 27%; Fairham in Nottingham has moved from 17% to 25%. Almost all of the fifty schools now have fewer students who fail to pass a single GCSE at any level (only six have deteriorated in this respect). Clearly, there is some genuine improvement.

However, the signs of continuing failure are striking. On Mr Blunkett's own 15% benchmark, twenty one of these success stories are liable to be closed down: fourteen of them are not even scoring 10% (one school scores only 2%, two others score 4%). Nearly a third of them have actually declined: sixteen are turning in results for A to C grades which are worse than they were before they went in. (This includes the troubled Ridings School in Yorkshire, which went in with 8% and came out with only 6%). Four others have delivered a net improvement of only one per cent over five years.

We also looked to see if there was a trend for results to improve over time. We found that after twelve months in special measures, the schools showed a small average improvement in the number of children scoring five A to Cs. But after two years, there was a small drop, followed by a two-year plateau and then a marked fall for those who had been in special measures for more than five years.

The signs of failure touch even the most renowned success story. Northicote School in Wolverhampton was the first secondary school ever put into special measures, in November 1993, and when it emerged two years later, it was greeted with a chorus of official acclaim. Its headteacher, Geoff Hampton, was given a knighthood for his success. He became a national consultant on techniques for 'turning around' failing schools and was subsequently invited to Downing Street to tell the Prime Minister about his methods.

Last year, a team of Ofsted inspectors returned to the school and, although they found strengths - "GCSE results rising much faster than the national average... majority of students making good progress... good links with the community... financial planning is very good" - they also found just as many weaknesses. They re-ported: "Not enough teaching that is good or very good... monitoring of teaching is unsatisfactory... students' personal development is unsatisfactory... quality of sixth-form provision is poor... school does not meet all statutory requirements... level of students' attendance is below the national average."

The underlying point here is the one that has been made repeatedly to Mr Blunkett - that schools can be improved by shaking up teaching and management, but this improvement is limited by the school's resources and by its intake of children. In the case of Northicote, where the per centage of students scoring five A to C grades has reached 20%, there has been a clear change in intake. In 1995, Ofsted found that a massive 70% of the pupils had special educational needs. Last year, they found only 33% had. The number of children whose families were poor enough to claim free school meals had also declined, from 33% when the school went into special measures; to 30% when it emerged in 1995; to 26% now.

Phoenix School in Hammersmith has done everything which Mr Blunkett could ask to improve its teaching and management. Last year, Ofsted reported that five years after the school went into special measures, the leadership of its headteacher was 'excellent', the governing body's link with the school was 'first rate', the LEA's help was 'effective and enduring', and teaching had improved to the point where 60% of the lessons were either good, very good or excellent. And yet despite all this, the number of children at the school who scored five A to C grades at GCSE last year was only five per cent. In the year before the school went into special measures, it was more than three times higher, at 17%.

Mr Blunkett may not understand the reasons for this fall, but anyone at Phoenix School can tell him that the school was damaged directly by being put into special measures in early 1994. This triggered a rash of vitriolic pubicity which, in turn, created an immediate flight of teachers and of parents of motivated children. This left classes to be taught by supply teachers; it also took high-achieving children directly out of GCSE groups and drained many of the most able children from the new intake in September. The school's results immediately started to slide, from 17% to 11% in 1994 and then 5% in 1995. As the improvements in management took hold, they rose again, to 16% in 1997 only to be slammed downwards again when the intake of eleven-year-olds which had been most weakened by the bad publicity reached Year Eleven in 1999 and sat their GCSEs, with only 5% of them scoring five A to Cs. In other ways, the school can show real improvement: more children attend, fewer are excluded, they behave better; Ofsted said their moral and cultural development was very good. But special measures has not delivered the academic results which are claimed for it. In truth, in some respects, it has damaged them.

Earlier this year Mr Blunkett tried to justify his sidelining of the impact of a poor intake on school performance by citing a school which turns in less than 25% A to C grades but where only six per cent of the children are poor enough to claim free meals. What he chose not to tell his audience was first, that this is a secondary modern school surrounded by three grammar schools which systematically skim off the brightest children in the community; and, secondly, that the school, in Lincolnshire, has been forced to adopt a new system for registering free school meals. Children can now claim them only if their parents physically attend school to confirm that they qualify: the headteacher says that, without this new system, they would have some 20% of children on free meals.

For those schools which fare particularly badly in special measures, Mr Blunkett has created the Fresh Start programme, in which the school is closed and its entire staff are sacked before it is re-opened with a new name, a new head and a new staff, which may include some hand picked from the old school. In March, he declared that "our Fresh Start policy is already being used by LEAs to tackle failing schools and is beginning to have an impact." A month later, he referred to the scheme again as an example of 'rapid progress' in tackling failure and added: "A successful example of this is Firfield School in Newcastle."

Since then, Firfield has been caught out by Channel Four News trying to get rid of difficult pupils by persuading parents to claim they were going to educate them at home; its 'superhead' has resigned; and this year, with 120 vacancies for Year Seven students, it has been chosen as first choice by only 60. Our understanding is that the LEA are now planning to take this 'successful example' and close it down for good.

Most of the nine other Fresh Starts have also run into trouble. In Wolverhampton, the new head of Kings School, Tim Gallagher, recently told the Times Educational Supplement that that he had been given no warning of the Fresh Start decision ("We were told we would be part of the Fresh Start initiative. We thought 'What's that?'"). They had been given no extra funds (a common complaint in the Fresh Start schools) and he complained that essential building works were being stalled by Whitehall bureaucracy: "In effect, we were given a millstone when we started, not a fresh start." In Hull, Kingswood has seen 23 of its 50 teachers, including four heads of department, hand in their notice since the school went into Fresh Start last September. Riverdeen in Nottingham are expecting only half of their Year Seven places to be filled in September. Bishopsford in Merton has recruited only 60% of the new pupils it hoped for.

In Brighton, the new College of Media Arts similarly lost 18 of its 58 staff within two terms of its Fresh Start, before also losing its headteacher, Tony Garwood, and its chair of governors. When they tried to find a new head, five of their six short-listed candidates pulled out of recent interviews, and the only remaining applicant was considered unsuitable. Earlier this month (July), they finally found a new head from the private sector, but she cannot start until next Easter. The school has been dogged by debt, computer foul-ups, friction with the LEA, truancy and indiscipline. For September it has filled only 58% of its vacancies. Now it has been put back into special measures. One senior figure at the school told us: "The requirement was for a radically new way of doing things. Unless we were going to change the children as well, that approach was a mistake."

The much-celebrated Fresh Start at the Islington Arts and Media School has also crashed in flames. For two months after the school opened, there was only one phone line in the whole place, no hot water, no kitchen, no fire alarm (they used a fog horn for fire drills), no science labs (the builders had gutted them by mistake) and no locks on the doors. When the locks finally arrived, they were the wrong ones; when the right ones finally turned up, they were wrongly installed. The electronic registration system did not work, because there were not enough cables. All this reflected a state of stunning chaos in the LEA which had no idea how many children would come to the school, no idea what the school budget would be, no extra Fresh Start funds because they failed to apply for them, and no New Deal money because they applied and were turned down.

When the head teacher's guestimate of the number of pupils turned out to be too low, the LEA provided no extra money for the extra pupils and the classrooms which had been planned for 22, were suddenly filled with 30 pupils. At the beginning, there were so few classrooms that pupils had to come to school in shifts. Most of the toilets did not work. There was a surge in bullying, but when the head tried to exclude pupils, he was blocked by the new policy of 'inclusivity'. When two girls started fighting in the playground over a bag of chips, it turned into a running battle involving 40 students. A new 'privatised' LEA took over, at which point the school discovered that the few decisions which it had managed to wring out of the old LEA were all null and void because none of them had been minuted. The new head resigned and, like Brighton, the school is now being put back into special measures.

None of this failure should surprise Mr Blunkett. The Fresh Start scheme is based on the idea of 'reconstitution' developed in San Francisco in 1984. It spread to other cities, but by 1997 it was thoroughly discredited. In December of that year, the American Federation of Teachers described the initiative as "politically popular but educationally bankrupt". That was when Mr Blunkett grasped the idea - just as it was being abandoned not only by its pioneers in San Francisco but also by other converts in Chicago, Cleveland, Cincinnati, Memphis and Minneapolis, all of whom agreed with the AFT that it would be better to try a more collaborative approach, in which teachers and officials worked together to draw up action plans for struggling schools and offered teachers the chance to stay on or to leave.

Even Gary Orfield, who chaired the committee of experts which launched the San Francisco experiment, now recognises the limits of reconstitution. He told us: "My basic conclusion is that this is like open heart surgery. It is necessary in some cases, but very costly and needs a very strong supporting team to give it a reasonable chance at success. It produces strong resistance and anger from faculties when it is done in the wrong way, and it cannot produce miracles. It should not be done on a massive basis because it requires a great deal of investment in leadership in creating a brand new school in a situation which is inherently difficult."

Furthermore, Mr Blunkett has been warned repeatedly that his whole approach to school improvment is flawed. The same experts who pioneered the techniques which he is using, have urged him to recognise that their benefits are "valuable but limited". The improvement which has occurred is precisely within the narrow range predicted by these experts, and yet he continues to try to use their methods to deliver far more.

A senior Ofsted inspector told us: "A poor school is fantastically hard to turn around. The DFEE deems schools to have been turned around by concentrating on criticising teachers and managers and watching for signs of change in behaviour, particularly truanting. But in order to do so it has to turn a blind eye to its own professed target, an increase in academic standards. So a school is turned around if its behaviour improves, even though its education may remain quite unchanged. In order seriously to turn that school around, they would have to look at its curriculum, exams, teaching technique and they would have to look at the fundamentals but they absolutely refuse to do that."

Mr Blunkett is thrashing the wrong horse. There is widespread agreement now that, in the 1980s, the Tory government were right to complain that schools were suffering from some bad teaching and idle management. It set up some of the most powerful systems that have ever been brought to bear on a public service and, with a few exceptions, they have purged the problem. Now almost everyone at every level of education knows that the DFEE need to switch their attention to other causes of failure, some of them structural, some of them in specific policy, most of them the direct product of DFEE decisions. And yet Mr Blunkett is still thrashing the horse in the stable instead of the one with the cart, still smiling and claiming to be pleased at the progress of his journey.

Additional research by Helene Mulholland

Poverty invades the classroom

The Guardian, September 1999

This is the moment. The teacher with the Bleeper has legs like an ostrich and takes the stairs three at a time. Within 30 seconds, he has reached the classroom which has called for help and there, he wades into the confusion. The trouble is Terence.

Terence is on a computer but he is refusing to work on the exercise he has been set. Instead he is fooling with graphics, simply ignoring instructions, his chin resting insolently on one palm. The classroom teacher is torn between Terence, who is stealing his attention, and the other children, who are beginning to wander and chatter. The teacher on Bleeper Patrol tells Terence he must leave the room. Terence sullenly refuses and carries on toying with the screen in front of him. Two girls come over to eavesdrop on the confrontation. In the background, several boys break into Punjabi to swap insults.

This is the moment that lies at the heart of the often frantic debate about Britain's schools - when a teacher stands up in front of a class and the teaching simply fails to take place. It is the moment which haunts a Prime Minister who famously declared that his three highest priorities are "education, education and education". And this particular moment happens to be occuring in a school which sits right in the centre of Sheffield, the political cradle of the current education secretary, David Blunkett.

The Bleeper Man persists. He has been on this kind of patrol many times before, acting as a kind of fireman who can be called out to deal with any crisis in the building. Tantrums, fights, breaking windows, smoking cigarettes, all riddled in amongst the daily rituals of a stable school. On a bad day, the Bleeper will call for help 40 or 50 times - a crisis every eight minutes or so. He knows he has to be careful. A few weeks ago, a boy went up on the roof and dangled one leg over the edge, threatening to jump unless he was left alone. That time, the Bleeper man quietly talked him down.

For several minutes, with the whole class wobbling on the verge of disintegration, Terence simply ignores the requests to leave, until suddenly he jumps to his feet, crashes his way through several unused chairs, snears at the classroom teacher and surges out into the corridor where he marches off, drumming one fist loudly against the wall. In the doorway at the end, he bumps into a 12-year-old girl, kicks her in the shin and vanishes around the corner. The class calms down, the teacher teaches and the Bleeper man goes off in search of Terence, the electronic alarm already squealing once more in his pocket.

Abbeydale Grange was once the cream of Sheffield's schools, a well-endowed comprehensive which was built out of three grammar schools with a tradition of high achievement and old-fashioned discipline. In many ways, it still succeeds and yet now it is beset by trouble. It struggles to survive; its numbers have disintegrated, from more than 2,000 to just over 500; only 22% of the pupils score five A-C grades at GCSE; its budget is drowning in deficit. It is one of the 40% of secondary schools in Britain which are said by Ofsted to fall below the required standard.

Why do some schools fail to deliver the best academic results? The big problem is trendy teaching methods (according to just about everybody on the right); it's a chronic shortage of resources (just about everybody on the left); it's teachers (Ofsted); it's Ofsted (teachers); it's a culture of low expectation (George Walden, the former Tory education minister); it's an overdose of intervention (the teacher unions); it's the abolition of grammar schools; the existence of private schools; the rigging of exam results; the shortage of nursery schools. It's the most important question in British public life. And yet the answer is torn like a fox between hounds.

This matters not simply as an exercise in failed analysis - there are plenty of policy questions which remain unanswered - but because in the last 15 years, education has attracted more intervention than any other area of goverment. From the sweeping Tory reforms of the late 1980s to the volley of initiatives since May 1997, this cacophony of answers has generated a cross-fire of activity by the state. If the analysis is wrong, much of this activity has been shot into the dark.

Why do schools fail? Despite all the confusion, there is an answer to the question. The strange reality is that in an area which is so peculiarly riven with controversy and genuine doubt, there is one clear, undeniable truth - one factor which more than any other determines whether a school will succeed or fail in delivering academic results. It is something which is recognised by almost everyone who is directly involved in schools, and yet it remains overlooked by almost all outsiders and sidelined by almost all official discourse. The answer is revealed by the Bleeper Man.

*

It is ten o'clock in the morning at Abbeydale Grange, and already the Bleeper has been busy: Dave has casually walked out of his class and gone to see his mates two doors away; a Somali lad with a baseball cap has downed tools and will not work; Joey is dancing on a table, whistling loudly so he cannot hear his teacher's protests. The Bleeper Man ricochets between them, ferrying the unruly to the Time Out room, where he finds Darren who is not supposed to be there at all. He was excluded yesterday, but his mother has sent him to school just the same. It is a contest with disorder.

Here on the wall is Shane's poem, one eleven-year-old boy's image of education: "School's crap, school's good
Every day, we come to school
It always rains
And every day I get off my bus and go for a fag
I say to myself 'Well, there goes another day'
Teachers talking, students shouting
When is all this noise going to stop?"

Here comes Imran, long and lean and full of mouth, sauntering late into class with a bag of crisps on the go, stopping to chat to his friends on the way to his desk. Never mind the lesson struggling to survive. Never mind anything. Imran is already on a last warning. He threatened to take off his belt and thrash someone who crossed him and the headteacher has told him he is on the edge of the precipice staring down at permanent exclusion. He has promised to produce "a ten-out-of-ten" day. Now, he grins as he swaggers towards his seat, a little lord of disorder.

What is going on in this place? It is not that the school is in chaos. There are no riots or rapes. Indeed, there are classrooms full of children who are learning. There are charismatic teachers and some brilliant kids - charming, clever kids, sporting stars, girls taking their GCSE's two and three years ahead of schedule. But then there is this fragility, this constant bubbling of trouble threatening to erupt as if the teachers were pulling off a miracle every time they reached the end of a lesson without an explosion. As the Bleeper Man lopes through the school, juggling crises, the outlline of the truth begins to emerge, slowly through the blizzard of contradictory claims.

He talks about the day he followed an eleven-year-old boy who had skived off class. When he caught up with him, he asked him simply "What's up?", and the boy slumped on to his stomach on the floor and started beating the lino with his fists, groaning with some inexpressible pain. He talks about the boy with the elfin face, who is sent to the Time Out room three times today: he has no father, his mother cannot cope; he has an alarming medical problem and he knows it; he is eleven but he has the reading age of a six-year-old; his dearest wish is to be excluded permanently so that he does not have to deal with life in the classroom.

There is the boy who comes to school from some kind of hell with his mother and spends the day hiding in the hood of his coat; the girl who has lost her mother and her father and whose grandparents were so harsh with her that she went to social services and begged to be taken into care; the boy whose home burned down, killing his pet while he fought with the firemen who would not let him go into the flames to save it (and who is now obsessed with doom and destruction). There are girls who get pregnant, boys who get drug problems, kids who have been taught at home to beat the crap out of anyone who irritates them, a boy who has moved home seven times in seven months because neighbours keep attacking his mother - and several times a week, he disappears from school and trails back to their latest refuge to protect her.

There are twelve-year-old girls who are the main carers in their family, feeding, clothing and supervising a cluster of younger siblings. This girl's brother has been beaten up by the local street gang. This boy's father is in prison. Somebody's mother is a drunk. Somebody's house has been torched by the neighbours. Here's an art class of thirteen-year-olds who have just spent the day in Derbyshire: half of them had never left Sheffield city in their lives. Here's a girl who has gone to the school office because her leg hurts, only to find that the police want to talk to her about reports that her step father has been assaulting her.

The children who are caught up by the Bleeper Patrol have more stories than Hollywood, but almost all of them have one thing in common. They are poor. And that is what matters. It is a simple thing. Every teacher knows it. There was a time when every government minister admitted it. The banal reality is that the single factor which more than any other determines a school's performance is its intake - the children who go there.

A small part of this is gender: girls at secondary school do better than boys. Everybody puzzles over it, but nobody can deny it. No single-sex girls school, for example, has ever been failed by Ofsted. While 40% of Sheffield girls scored at least five A to C grades at GCSE last year, only 33% of boys did so. But the big factor is poverty.

If a school takes in a substantial proportion of children who come from a disadvantaged background - if their parents do not read, if they have no books at home, if they are awake half the night and then half asleep all day, if they have been emotionally damaged by problems in their family or in their community, if they have suffered from an environment which, more than any other, is likely to expose them to drug abuse and violence and alcohol abuse and the collapse of social boundaries, then the school is more likely to fail academically. A school which is based in a disadvantaged community will struggle with its children, while one that is based in a more affluent area will prosper.

This not an occasional problem, but an endemic one. There are about 13.3 million children in Britain. On any available measure, some 4.6 million of them live in poverty - and they are all enrolled in schools. The evidence that poverty undermines education is overwhelming - and has been for years. Yet governments deny it. The last government denied the poverty itself. This government admits the poverty but denies its impact.

By obscuring this simple reality, the public discourse on our school system has entered the realm of the absurd and become lost there. This is not to deny that there are good and bad teachers, that there are good and bad approaches to teaching, that schools can make a difference. It is not to quarrel with the complaint that schools in the 1960s and 70s were allowed to drift into a state of unsupervised complacency or that ideologies of social engineering may often have interfered with education or that the Guardian among others on the left was seduced into some naive and unsupportable positions. But when those factors take their proper place in the picture, they slip out to the margins while the children take up the centre - and the host of political initiatives which ignore the children are revealed as mere alibis. Like all alibis, those initiatives may contain some element of truth but like all of the most dangerous alibis, they are essentially dishonest.

*

Until a few years ago, Dr Phil Budgell was the chief inspector of schools in Sheffield. Like his opposite number in every other Local Education Authority, he visited all the schools on his patch and noted their strengths and shortcomings but, unlike most of his opposite numbers, Dr Budgell is a trained statistician. In search of an understanding of what he saw, he began to sift through the river of statistics which flowed into his department, panhandling it in search of patterns.

The poverty was obvious. Since 1979, South Yorkshire has lost 24% of its jobs, and nearly a quarter of Sheffield's children now live in families with no earner. Dr Budgell started using census material to tot up the indicators of poverty in each household - no earner, no car, overcrowding, single parent, ethnic minority. Then he switched to the database for the city's schools, pulled out the post codes for every single pupil, matched this against the districts for which he had census data and produced an index of disadvantage for all the schools in Sheffield. He added in the distribution of girl pupils and also figures for those who simply failed to turn up for school in Year Eleven, when exams were being taken, and he produced a table which ranked all 27 secondary schools in the city according to the difficulties of their intake.

Then he looked at the academic outcomes of the 27 schools. He used seven different measures of exam results, including average scores, mean scores, A to C grades, A to G grades. There were small variations but essentially the picture was clear: the league table of schools who did well in exams was simply the reverse of the league table of difficult intakes. Using multiple regression analysis, Dr Budgell found that more than 90% of the difference in exam results between schools was accounted for simply by the poverty, gender and final-year attendance of the children who were enrolled there. What was being done by the schools was influencing only the remaining five to ten per cent.

"I'm not saying that schools don't make a difference," he told the Guardian. "There are incompetent teachers, but in order to explain the failing of inner city schools in terms of incompetence you have to make the bizarre assumption that these schools have hired a mass of incompetent teachers while good schools have hired none. There is a volume of evidence that schools are not playing on a level playing field. When you look at these intake factors, the level playing field is more like the side of Mount Everest."

Three secondary schools in Sheffield have been condemned by Ofsted and put into 'special measures': Earl Marshall, Hinde House and Myrtle Springs. All three of them are in the North East of the city, with an intake which is dominated by the children of poor families. At Fir Vale School, which has taken over from Earl Marshall, the head teacher Ken Cook has a pupil body of whom only 16% speak English as their first language. Most of his parents speak no English at all. On a Monday morning last term, he had seven Somali children turn up for their first day at school, fresh out of a war zone, without a sentence of English between them. "Within an hour they are in the classroom," he said, "and we are accountable for their performance."

At Hinde House School, the head teacher, Sarah Draper, deals with a similarly poor intake: "If there are 25 kids in a classroom, there may be 15 with behavioural problems. I am past being shocked, although I know that people out there don't understand." She recalled the children who had failed to turn up for school during England's first match in the World Cup last year. Some of their parents had insisted that the children were right to stay at home. "They thought football was more important than school. The trouble is that education is a middle class value which we are trying to operate in a working class culture."

Abbeydale Grange draws its children from a wedge of deprivation, which takes in the Sharrow area where unemployment is the highest in the city, infant mortality is the highest in the city, 30% of children come from families on income support, 12% of the adults are diagnosed as suffering from depression and 25% of the children live in homes which are officially deemed to be overcrowded. Fifty three per cent of the school's students claim free school meals: on national trends, a further ten per cent would be poor enough to qualify but fail to lodge the claim. The poverty invades the school like water flooding a ship, reaching into every weak point.

Poverty, at first, means sheer material hardship: the numerous children who come to school without breakfast; the boy who falls down in the mud at school and turns up for the rest of the week in the same muddy clothes; the homes which have no books, never mind computers or Internet connections; no quiet place for homework; no cash for school trips; no cash for bus fares in the morning.

Poverty often means parents who gained nothing from school and expect nothing more from it for their children, like the Somali father who keeps his daughter at home so that she can translate for him; or the man who has kept his 13-year-old boy off school for the last two years so he can help look after their animals. Abbeydale teachers last term told Wayne, who is 12, that he should start to think about what GCSEs he wants to sit. He blinked and shook his head and said that no one in his family - none of his numerous older brothers and sisters, and certainly not his parents or their siblings - had ever passed any exam of any kind, so, well, why would he?

Headteachers and officials at the town hall agree that in the old public housing estates, education has never been highly valued. In the good old days up until the early 1980s, that was because there were apprenticeships more or less on demand in the coal and steel industries. Now, Sheffield's traditional economy has been destroyed, 60% of the old industrial jobs have been lost for ever, and the reasoning is reversed: last year, one out of every five young people who left Sheffield's schools had no work to go to, and one out of nine of them had no qualifications at all. Overwhelmingly, those young people came from the old estates where they remain now, as the neighbours - and role models - of this year's students.

The best available source of income and status is the drugs economy, which reaches into every school in the city. One of the Abbeydale Grange teachers recalls the primary school where he taught where one of the eight-year-old boys daily stole the lunchboxes off other kids: he was the son of the local drug baron, so no one could argue with him. In the library at Abbeydale, Mark, now aged 15, tells how the previous day he bumped into a friend from primary school. "I never knew him that well. He wasn't exactly a friend. But he gave me his bleeper number in case I wanted to buy any blow off him. Which was helpful." Why bother with school?

Poverty steers children off course long before they reach secondary school. Of the 115 eleven-year-olds in last term's Year Seven at Abbeydale Grange, twenty five of them arrived at the school with a reading age of less than eight. Their Non Verbal Reasoning Scores were just as low. The effect on the rest of their schooling is catastrophic.

Many of the deprived children come from families of recent immigrants who do not speak English as their first language. Of the 521 pupils in the school last term, 204 are from the Indian subcontinent, together with children from Colombia, Brazil, Somalia, Venezuela, Kosovo, Senegal, Portugal and China. Half of the pupils in Years Seven and Eight are in the process of learning English. In Years Eight and Nine, the position is even more difficult, with 70% of students adapting to English as a second language. Last term, one boy completed a French test by translating the text into Albanian. By the end of the term, the teacher had found someone to make sense of it and discovered that he had got almost all of it right.

Poverty does its worst damage with the emotions of those who live with it: parents who are too tired or depressed, too stretched trying to juggle too many young children, too damaged to cope; children whose development is distorted from their earliest days. Forty five per cent of the students at Abbeydale Grange are classified as having special educational needs, many of them suffer from emotional or behavioural problems. Twelve per cent of them have a need so serious that they are 'statemented' by the local authority as cases requiring the involvement of outside agencies: IQ as low as 50, very short concentration, hyperactivity, disruptive behaviour, attention-seeking, dyslexia, clinical depression.

Josh suffers from a classic cluster of problems. He is twelve, his father has not been seen for years, his step father cannot be bothered with him, his mother drinks and simply does not like him. She criticises his every move and has called the school to complain about their sending home letters in praise of his better behaviour. When a teacher tried to take him to the cinema as a reward for trying, she blocked it. Josh expects to make a mess of everything he touches and he spends his day in school avoiding work for fear of failing; looking for attention with disruptive jokes and antics; smoking and attempting to wander off to the shops. There are children like Josh in every class in Abbeydale Grange.

*

So a school like this is logged as a failure, its academic results limping far behind the private schools and the state schools in rural towns and pleasant suburbs. Back on the Bleeper Patrol, however, a very different picture begins to emerge - signs of success, hidden beneath the surface of daily school life.

During the night, it rained, and, as usual, the puddles on the flat school roof have leaked through to the modern languages room below. Now, there's a whole Spanish class roaming the corridors in search of a home. The Bleeper man races down the corridor, finds an empty room, races back to the Spanish class but, before he can reach them, he finds a small girl wandering in search of a teacher who has failed to show up. He sends the girl to tell the Spanish class to go to the empty room, pops his head in the door of the class without a teacher and calms the children, gallops off down the stairs to the staff room, finds the name of the missing teacher on the rota, heads to the school office who have no idea where he is, charges back up stairs, shepherding stray Spanish students as he goes, tells two boys to stop spitting and a third not to swear, grabs some litter off the floor, finds a spare teacher, sends him to the class who have lost theirs, checks that the Spanish class has found its home and sees that all is well, heads for the class without a teacher and sees they are still fooling around, discovers the spare teacher has gone to the wrong classroom, finds him, redirects him, takes a breath... and realises that all is well, all is quiet. He has created order.

While the outside world looks at the league tables and sees failure, for the teachers inside the school, life is thick with success.

One of the Bleeper Man's most regular customers was Catherine, who left the school this summer. She came from a violent and broken home and, from her first day in the school was an almost constant source of disruption, walking out of classes, refusing to fit in, threatening violence against herself and others. The school poured its attention into her. They attached a support assistant to her, designed a special timetable for her, allowed her to go to a Quiet Room to escape from her most worrying classes, drew up a contract for her behaviour, reviewed her progress every week, gave her counselling, liaised with her unstable home. And it worked. Last summer term, she sat her GCSE exams. After five years of domestic turmoil and emotional pain, the school had no illusions at all about the grades she would get. But she sat them - decided it was worth the effort, decided to revise, decided to turn up on time and even to sit through them writing, decided to try. That is success.

When the Bleeper Man sees Josh in the playground being approached by two older boys on their way for a cigarette, and Josh turns them away, that, too, is success. When Terence rushes out of his computer class and disappears and the Bleeper Man quietly follows him and finally discovers him already sitting in the Time Out room, having decided to co-operate and not to run or to fight, that is success.

These are children who are so tough on the street that policemen won't go on their estates without back-up and flak jackets; yet a lone teacher in shirtsleeves deals with them thirty at a time. But that doesn't score points in the league tables. The same children who fail their SATs tests also write the school prospectus and sit on interview panels, with a power of veto, when the school hires new staff.

There is hidden success in sport - like the Abbeydale Grange football team which struggled to win matches but scored a city-wide record by playing for the five full years of a school career without ever arguing with a single referee's decision. They were rewarded by Sheffield United, who invited them to use their ground at Bramall Lane to play their final match. There is social success, in the tumultuous combination of cultures in the playground without any kind of race hatred; in the rarity of bullying; in the sheer delight of the Year Seven cricket team who have only tennis balls and four elderly bats for practice but who took on the local Birkdale prep school with their brand new kit - and thrashed them.

When a Year Ten student passes GCSE maths a year early, when two Year Nine students do it two years early, when a Year Eight student does it three years early, that is straight forward academic success. Although none of it shows up in the official school tables which record only Year Eleven results. When the government last year produced a 'value-added' table, concentrating not on exam results but on signs of academic improvement between Year Nine and Year Eleven, Abbeydale Grange was one of only three schools in the whole of Sheffield to record a dramatic improvement. But other schools, who normally come out top of the tables, complained - and the whole exercise was watered down.

The bell rings for the end of the last lesson. The Bleeper Man heads out to the drive way, where the children mill around the busses. The outside world is waiting to invade. Two young men with pimples and baseball caps start handing out advertising flyers for a free evening at a new nightclub in the city centre. The eleven-year-old boys and girls grab the flyers and mount the busses. Two girls from Year Seven hug each other tight and say goodbye for the day. A couple of boys slip away for a cigarette.

Schools are defined by the children who go there. Take the children from Abbeydale Grange and parachute them into Eton, and Eton will start to fail academically. Take Eton's teachers and plant them in Abbeydale Grange, and they will struggle to teach. The truth is masked by academic results. They simply disclose how well the children did in their exams, but they don't tell you how well the school did with its intake. And it is the intake that matters. For years, Britain has been shovelling children into poverty - taking away their parents' work, cutting their family's welfare, embroiling them in a war against drugs which has plunged them into crime and violence, breaking up their communities - and now these children are in the schools, messed up, screwed up, damaged and delinquent, shunted over the edge. A school with a poor intake is like an ambulance at the bottom of a cliff: sometimes, it can pick up the children and patch up the damage; most of the time, it's too late.

One of the teachers at Abbeydale Grange left at the end of last term, abandoning his career twenty years early. "Opinion formers seem to have no concept of what is going on," he said. "The general level of achievement of children with these problems is very low and it has got worse with poverty. " This particular teacher has found his own solution. He is a Christian and he has gone off to join the Church Army in the hope that God might succeed where governments have failed.

The point here is not that governments should introduce a more sensitive measure of achievement like value-added tables. The point is much bigger: the vast majority of government interventions over the last 15 years have been built on the foundation that schools can be blamed for the failure of their children; if that foundation is essentially false, the whole structure of reform is wrong. Millions of pounds and a mass of energy have been poured into projects which at best succeed only partially and at worst, actively damage the schools they are claiming to help. Tomorrow, we disclose evidence that that damage is now profound.

* To protect the privacy of children at Abbeydale Grange, most of their names and some minor identifying details have been changed.

Additional research by Helene Mulholland

Poverty and education: the evidence

The Guardian, September 99

This is the secret that everyone knows: the children of poor families are far less likely to do well in school than those whose parents are affluent. For the last ten years, this has been almost buried in denial. "Poverty is no excuse," according to the Department for Education. Neverthless, it is the key. As everyone knows.

The ministers and pundits who want to deny or diminish the link are keen to present it as the invention of soft-focus lefties trying to justify a socialist theory of education or to excuse incompetent teachers. However, the clearest and most persuasive recent evidence for the link was produced earlier this year, not by a teacher's union or a liberal academic - but by the Treasury, in its fourth report on the modernisation of Britain's tax and benefit system.

Reviewing nearly 30 years of research, the Treasury reported: "Children from disadvantaged backgrounds are much less likely to succeed in education... On 'difficult to let' estates, one in four children gain no GCSEs (the national average is one in twenty) and rates of truancy are four times the national average... There is considerable evidence that growing up in a family which has experienced financial difficulties, damages children's educational performance...

"The differences between advantaged and disadvantaged children are apparent from a very early age. At 22 months, children whose parents are in social classes one or two are already fourteen percentage points higher up the educational-development distribution than children whose parents are in social class four or five.... The data from the National Child Development Survey show that there is a strong relationship between children's performace in maths and readings tests between the ages of six and eight, and their parents' earnings, with the children of higher earning parents performing better... If one father's earnings are double the level of another, his son's maths test score is on average five percentile points higher than the other's... Going to school does not reduce the differences in early development between advantaged and disadvantaged children."

The link is strong. It is also central to the experience of Britain's schools because, as the same Treasury document confirms, poverty in Britain has trebled since 1979 to the point where a third of Britain's children - more than four million of them - now live below the poverty line. This torrent of poor children poured into the classroom at exactly the same time as standards of behaviour and achievement slumped. Our levels of pupil failure are higher than most of the rest of the developed world, but our levels of child poverty also are higher than most of the rest of the developed world. According to Eurostat, for example, 32% of children in the UK live in poor households, compared to 20% in the rest of the European Union. According to Treasury figures, we have higher poverty levels than Greece and Portugal.

The physical, emotional and social damage which is inflicted on children who live in poverty, is clearly reflected in the latest academic results. The independent group, Research and Information of State Education, trawled through Ofsted reports and matched the standards of students against the number who were claiming free school meals, the nearest available measure of poverty in the classroom. In schools with only a few poor children, one in every five pupils was scoring Grade 1; at the other end of the spectrum, in schools with a well above average number of children on free school meals, only one in a hundred was doing so.

A disadvantaged intake can make life tough for a whole school, not just for individuals. Researchers at Durham University looked at schools which have been failed and subjected to 'special measures' by Ofsted and then matched them against the six bands of disadvantage which are used by the Department of Education to reflect the proportion of pupils on free meals. Not one of the schools in special measures fell in any of the three 'affluent' bands. A small group fell close to the national average, but almost all of them - 96.5% - were in the two upper bands, schools with a proportion of children on free meals which is clearly above the national average.

The evidence goes on and on - from the US, from the OECD, from the EU. There are literally dozens of academic studies which confirm the link between poverty and academic failure. But in the Department of Education, anxious to deliver policies which appear to have a chance of short-term success, the new orthodoxy remains the same. "Poverty is no excuse."

Additional research by Helene Mulholland

How the government used and abused educational research

The Guardian, September 1999

The greatest dream of all good experts is to find a government who will listen and turn their research into reality. Some succeed. Peter Mortimore did. But the greatest frustration for any expert is to have found a government who finally listened - and ended up misunderstanding.

Twenty four years ago, Peter Mortimore abandoned a nine-year career as a school teacher to join a team of researchers who were about to embark on a special project for Professor Michael Rutter at the Institute of Psychiatry at the University of London. The goal which Mortimore and his colleagues set themselves was to try to identify the seeds of success in the classroom by spending four long years studying a dozen schools in London. They were to ignite one of the longest running theoretical disputes in the world of education.

In 1979, they published "Fifteen Thousand Hours: Secondary Schools and Their Effects on Children." It was a ground-breaking work. It challenged the conventional wisdom of the time by showing that although the social and economic background of pupils was a very powerful factor in deciding academic results, nevertheless it was not the whole story. Schools could make a difference, and they tried to identify the kinds of things which successful schools could do for their pupils to start to overcome their inherent social disadvantage.

This was important. There was no disguising the fact that state schools were in trouble. Numerous schools had been renamed ‘comprehensive' without any training for staff or change in their curriculum. Local education authorities allowed their schools to be accountable to just about no one. Headteachers could be as secretive as they pleased, concealing their results and disguising their problems, failing to heed criticism or complaint. There was widespread concern that some teaching methods were sloppy and ineffective and that ‘child centered' learning had reached a point of absurdity where teachers declined to mark their students' work for fear of appearing critical. Many schools appeared to have low expecations of their students, reflected in indifferent results. Now, finally, there was a chance to spread the best practice, to make schools more effective.

It took nearly ten years for the message to get through to the Department of Education. By that time, the research had been confirmed and refined repeatedly, in the United States as well as in Britain, and there was real excitement about a cluster of possibilities for school improvement - some involved different approaches to leadership and management, some to classroom technique. But, when they finally acted, Conservative ministers grabbed the wrong end of the stick and started beating teachers over the back with it. They fired off a volley of reforms - league tables, SATs tests, Ofsted inspections - all of which was aimed at the kind of School Effectiveness which Mortimore and his colleagues had identified and all of which utterly ignored the fact that the social and economic background of pupils remained a very powerful factor.

Mortimore and others tried to warn them. They explained that the most you could hope to achieve by improving schools was an increase of between eight and ten per cent in results. It was important and yet it was only a fraction of the whole. It was like watching a furious motorist pumping up the tyres on a car which had run out of petrol: it might eventually help the car to run more smoothly, but if he refused to address the real problem, it would do very little good and sooner or later, if he just carried on pumping regardless, something was bound to blow.

By now, Mortimore had become one of the leading experts on education in the country and had been appointed head of the Institute of Education at the University of London. In a book called Road to Improvement, he warned: "It is crucial that policy makers desist from claiming that school improvement - by itself and in the absence of extra resources - can solve all the problems. Whilst this might be true in ‘advantaged' schools, it is certainly not true in disadvantaged schools." Rather like the atom scientists who saw their work hijacked by government for immoral ends, he saw his warnings ignored.

The difficulty was that School Effectiveness was immensely attractive to politicians. By pinpointing the work of teachers and administrators, it completely absolved central government of all possible responsibility for failure. By sidelining the impact of intake, it permitted policies which focussed on detail in the school and were therefore relatively cheap, and which promised to deliver results quickly and were therefore electorally attractive. And so the Department for Education and Ofsted were already committed to hunting down failing schools and attributing their failure entirely to the weakness of teachers and managers, ignoring the destructive impact of an intake which had become progressively more delinquent as the new poverty swept through the country. The government's supporters were determined to recognise part of the truth and nothing but that part of the truth. Conservative columnists savaged Mortimore's book, effectively accusing him of not understanding his own research.

Ofsted and education ministers justified themselves by pointing to the performance of a group of schools with a disadvantaged intake, who appeared to have succeeded against the odds. The National Commission on Education organised studies of eleven such schools and found there was real evidence that, despite their intake, these schools had succeeded, using a combination of strong leadership, the setting of clear targets, and the involvement of parents and staff and community. And yet this study warned, first that none of the eleven schools saw government policy as helpful; and secondly that "the nature of school improvement.. has yet to be thoroughly understood and measured in a sensible and sensitive way." Researchers expressed real doubt about whether such schools would succeed in the long term, once the special effort of rescuing them subsided; and whether, in any event, there was any prospect of schools generally achieving these exceptional results; and, finally, whether some of those who were being credited with success against the odds had not simply attracted an easier intake of children.

One of those who took part in the eleven studies and who celebrated the achievement of these schools was Professor Peter Mortimore. However, this support for his early work only left him struggling all the harder to persuade the government to understand the rest of the story: "Whilst some schools can succeed against the odds, the possibility of them all doing so, year in and year out, still appears remote, given that the long-term patterning of educational inequality has been strikingly consistent throughout the history of public education in most countries.... We must be aware of the dangers of basing a national strategy for change on the efforts of outstanding individuals working in exceptional circumstances." None of which has stopped the government pursuing precisely such a strategy.

Additional Research by Helene Mulholland

A book about schools

The Guardian, November 2000

Nothing I have ever written has produced a reaction like the Guardian series on schools which is now being published as a book - a torrent of readers' letters spilling over with passion, more than a hundred invitations to speak at public meetings, a couple of journalism awards and a personal denunciation from the prime minister and the secretary of state for education. The current editor says the response was of a different order to anything else he has seen since he took over the paper (and this is the editor who presided over the demise of Aitken, Hamilton and Mandelson). What was that about?

I think something rather odd happened. Normally, when you publish an investigation in a newspaper, you hope you are uncovering something which nobody knows. With these stories, however, we did the reverse - we delivered something which masses of people knew but which no one with any power would admit. That torrential response was not shock or horror but a clamour of recognition of a reality that was being denied.

In the eighteen months I spent researching them, I got deeper into the workings of a government department than I have ever done before. It was not a reassuring experience. I started with a vague feeling of unspecified benevolence towards the secretary of state for education, David Blunkett. I came out with a feeling close to contempt, having peered into a department which, I eventually concluded, was habitually lying and cheating and was presiding over a shambles - something which it was enabled to do mainly by the scale of that self-same dishonesty.

I came away with a picture of ministers beset by problems and lost for solutions - lost, either because they simply didn't understand the issues well enough to know what to do, or because they did know and couldn't bear the political consequences. Rather than admit they were stuck, they were reaching for pseudo-solutions, policies which gave them something to talk about, gave the appearance of action, gave the pundits something to chew on while the real problems sat unsolved in the background, because (as the ministers often very well knew) the alleged solutions were entirely bogus.

We looked at Fresh Start, David Blunkett's brave new cure for the weakest schools - and found a mass of evidence which had discredited the idea in the United States years before Mr Blunkett ever pulled it out of his briefcase and started pretending it should be taken seriously. We looked at 'special measures', the routine procedure for dealing with a school that fails its Ofsted report - and (after months of negotiations) we dug out statistics which completely contradicted the ministers' grandiose claims of success. We looked at the official claims for the virtues of our 'mentoring' scheme - and then unearthed the department's own devastating findings of its miserable weakness. These initiatives are high-octane guff. And they reflect the great underlying problem, that the entire strategy of Mr Blunkett's department is based on an analysis of school failure which has the intellectual weight of a joke in a Christmas cracker. Everything else flows from that analysis, and the simple reality is that it's phoney.

And then there are the outright lies. In one of the early stories, we gave credit to David Blunkett for securing a £19 billion increase in funding for education - a few months later, we discovered that we had been conned. Well, the whole country had been conned. What was worse, the conman got away with it. On the single occasion when he was challenged about it - in the House of Commons, by the Liberal Democrat education spokesman, Phil Willis - Hansard records the secretary of state's reaction. He laughed. And that was that (although he never again repeated the £19 billion lie).

This is a lesson not just about the cynicism of politicians but also about our own gullibility - particularly our gullibility as journalists. We allow Whitehall to manage us: Mr Blunkett makes a big announcement about extra funds for Fresh Start schools; we report it, as though there were some kind of sense in it; and then we go off into micro-criticism of its detail, whether this is quite enough extra cash, whether there is really such a thing as a superhead, without explaining that the whole project is a proven failure. We are pushed into a sideshow.

Within a week of Mr Blunkett first telling his £19 billion whopper, he was caught out by the Treasury Select Committee, who exposed the most successful of his several sleights of hand. They did the same for the then health secretary, Frank Dobson, whose claims about new cash for his department were similarly fictitious. The point is that the press missed the story. The Treasury Select Committee's report sunk like a stone, the journalists chased off to the next press conference, and the various ministers carried on fibbing for 18 months before they were caught out. (Frank Dobson's budget was exposed as a hoax by BBC Panorama, by coincidence, in the week after we took apart Mr Blunkett's.)

I spent the first three or four months of this research being as stupid as it is possible for a journalist to be without getting sacked. I started, in the approved fashion, by reading files of old newspaper stories; I read the book by the former education minister, George Walden; I went to see senior people at the Department for Education and Ofsted. And I emerged with the clear view that school failure was primarily caused by bad teachers and, in particular, by bad teachers who had been led astray by 'trendy teaching methods' from the 1960s. Then two things happened. First, I read a book called Failing School Failing City by Martin Johnson, a veteran teacher and now president of the NASUWT union. Then I started going into schools. And I realised that my working theory was complete garbage, that the truth was simpler, nastier and very plain to see, as the first two stories, set in Sheffield, attempted to make clear: you cannot make sense of why some schools fail and some succeed without taking account of the corrosive impact of child poverty, which has soared in this country in the last twenty years. Combine that with the effects of the Conservative education reforms of the late 1980s and you have a design for educational failure.

You can look at any area of our schooling system - the effect of private schools on their state counterparts, the scale and distribution of funding, teacher stress and teacher pay, syllabus and teaching technique, truancy and exclusion, the outbreak of teacher cheating in exams - and you cannot explain what is happening unless you take primary account of child poverty and Kenneth Baker's reforms. There are other factors in there as well, but those two are essential. The reality is that unless Mr Blunkett acknowledges this and until he finds the political courage to scrap almost all of the market-driven reforms of the late 1980s, none of the dinky little schemes which he has launched will save our schools from crisis.

* The School Report, published today by Vintage, £6.99.

Zarqawi Letter, Feb 04

From the White House website, February 11, 2004

Global Message

A memo believed to be written by suspected al-Qaida operative Abu Musab al-Zarqawi reveals al-Qaida's participation in an effort to spark civil war in Iraq.
This memo describes an intention to tear Iraq apart by provoking violence toward Shi'a people and their leaders.
The Iraqi people have demonstrated repeatedly that they aren't willing to participate in these activities. They are looking forward to a free, united and sovereign Iraq.
The memo states that the biggest bulwark against terrorist efforts is the continued standing up of Iraqi security forces, continued American resolve, and the hand over of sovereignty to an effective Iraqi government.
Coalition and Iraqi security forces are capable of using intelligence to kill or capture those who would try to create anything but a safe and secure environment in Iraq.
This makes clear that the terrorists understand that failure to defeat Coalition forces will be a major setback for their overall terror war.
This development is an important reminder of why it is so critical that we forge ahead with our plan in Iraq to bolster Iraqi security forces.
Moving forward on the hand over of sovereignty to the Iraqi people will do more to isolate the terrorists than anything else. Coalition resolve remains unwavering.

Below is the text of the letter, as translated and distributed by the Coalition Provisional Authority.

1. The foreign Mujahidin: Their numbers continue to be small, compared to the large nature of the expected battle. We know that there are enough good groups and jihad is continuing, despite the negative rumors. What is preventing us from making a general call to arms is the fact that the country of Iraq has no mountains in which to seek refuge, or forest in which to hide. Our presence is apparent and our movement is out in the open. Eyes are everywhere. The enemy is before us and the sea is behind us. Many Iraqis would honor you as a guest and give you refuge, for you are a Muslim brother; however, they will not allow you to make their homes a base for operations or a safe house. People who will allow you to do such things are very rare, rarer than red sulfur. Therefore, it has been extremely difficult to lodge and keep safe a number of brothers, and also train new recruits. Praised be to Allah, however, with relentless effort and searching we have acquired some places and their numbers are increasing, to become base points for the brothers who will spark war and bring the people of this country into a real battle with god's will.
2. The present and future: there is no doubt that American losses were significant because they are spread thin amongst the people and because it is easy to get weapons. This is a fact that makes them easy targets, attractive for the believers. America, however, has no intention of leaving, no matter how many wounded nor how bloody it becomes. It is looking to a near future, when it will remain safe in its bases, while handing over control of Iraq to a bastard government with an army and police force that will bring back the time of (saddam) Husayn and his cohorts. (headquarters comment: it is not clear to whom "it" is referring, but it appears to mean the united states.) There is no doubt that our field of movement is shrinking and the grip around the throat of the Mujahidin has begun to tighten. With the spread of the army and police, our future is becoming frightening.
3. So where are we? Despite few supporters, lack of friends, and tough times, god has blessed us with victories against the enemy. We were involved in all the martyrdom operations - in terms of overseeing, preparing, and planning - that took place in this country except for the operations that took place in the north. Praised be to Allah, i have completed 25 of these operations, some of them against the Shi'a and their leaders, the Americans and their military, the police, the military, and the coalition forces. There will be more in the future, god willing. We did not want to publicly claim these operations until we become more powerful and were ready for the consequences. We need to show up strong and avoid getting hurt, now that we have made great strides and taken important steps forward. As we get closer to the decisive moment, we feel that our entity is spreading within the security void existing in Iraq, something that will allow us to secure bases on the ground, these bases that will be the jump start of a serious revival, god willing.
4. Plan of action: after much inquiry and discussion, we have narrowed our enemy to four groups:
A. Americans as you know, these are the biggest cowards that god has created and the easiest target. And we ask god to allow us to kill, and detain them, so that we can exchange them with our arrested shaykhs and brothers.
B. Kurds these are a pain and a thorn, and it is not time yet to deal with them. They are last on our list, even though we are trying to get to some of their leaders. God willing. ?C. The Iraqi troops, police, and agents these are the eyes, ears, and hand of the occupier. With god's permission, we are determined to target them with force in the near future, before their power strengthens.
D. The Shi'a in our opinion, these are the key to change. Targeting and striking their religious, political, and military symbols, will make them show their rage against the Sunnis and bear their inner vengeance. If we succeed in dragging them into a sectarian war, this will awaken the sleepy Sunnis who are fearful of destruction and death at the hands of these Sabeans, i.e., the Shi'a. Despite their weakness, the Sunnis are strong-willed and honest and different from the coward and deceitful Shi'a, who only attack the weak. Most of the Sunnis are aware of the danger of these people and they fear them. If it were not for those disappointing shaykhs, Sufis, and Muslim brothers, Sunnis would have a different attitude. ??5. Way of action: As we have mentioned to you, our situation demands that we treat the issue with courage and clarity. So the solution, and god only knows, is that we need to bring the Shi'a into the battle because it is the only way to prolong the duration of the fight between the infidels and us. We need to do that because:
A. The Shi'a have declared a subtle war against Islam. They are the close, dangerous enemy of the Sunnis. Even if the Americans are also an archenemy, the Shi'a are a greater danger and their harm more destructive to the nation than that of the Americans who are anyway the original enemy by consensus.
B. They have supported the Americans, helped them, and stand with them against the Mujahidin. They work and continue to work towards the destruction of the Mujahidin.
C. Fighting the Shi'a is the way to take the nation to battle. The Shi'a have taken on the dress of the army, police, and the Iraqi security forces, and have raised the banner of protecting the nation, and the citizens. Under this banner, they have begun to assassinate the Sunnis under the pretense that they are saboteurs, vestiges of the Ba'th, or terrorists who spread perversion in the country. This is being done with strong media support directed by the governing council and the Americans, and they have succeeded in splitting the regular Sunni from the Mujahidin. For example, in what they call the Sunni triangle, the army and police are spreading out in these regions, putting in charge Sunnis from the same region. Therefore, the problem is you end up having an army and police connected by lineage, blood, and appearance to the people of the region. This region is our base of operations from where we depart and to where we return. When the Americans withdraw, and they have already started doing that, they get replaced by these agents who are intimately linked to the people of this region. What will happen to us, if we fight them, and we have to fight them, is one of only two choices:
1) if we fight them, that will be difficult because there will be a schism between us and the people of the region. How can we kill their cousins and sons and under what pretext, after the Americans start withdrawing? The Americans will continue to control from their bases, but the sons of this land will be the authority. This is the democracy, we will have no pretext.
2) we can pack up and leave and look for another land, just like it has happened in so many lands of jihad. Our enemy is growing stronger day after day, and its intelligence information increases. By god, this is suffocation! We will be on the roads again. People follow their leaders, their hearts may be with you, but their swords are with their kings. So i say again, the only solution is to strike the religious, military, and other cadres of the Shi'a so that they revolt against the Sunnis. Some people will say, that this will be a reckless and irresponsible action that will bring the Islamic nation to a battle for which the Islamic nation is unprepared. Souls will perish and blood will be spilled. This is, however, exactly what we want, as there is nothing to win or lose in our situation. The Shi'a destroyed the balance, and the religion of god is worth more than lives. Until the majority stands up for the truth, we have to make sacrifices for this religion, and blood has to be spilled. For those who are good, we will speed up their trip to paradise, and the others, we will get rid of them.
By god, the religion of god is more precious than anything else. We have many rounds, attacks, and black nights with the Shi'a, and we cannot delay this. Their menace is looming and this is a fact that we should not fear, because they are the most cowardly people god has created. Killing their leaders will weaken them and with the death of the head, the whole group dies. They are not like the Sunnis. If you knew the fear in the souls of the Sunnis and their people, you would weep in sadness. How many of the mosques have they have turned in to Shi'a mosques ("husayniyas")? How many houses they have destroyed with their owners inside? How many brothers have they killed? How many sisters have been raped at the hands of those vile infidels?
If we are able to deal them blow after painful blow so that they engage in a battle, we will be able to reshuffle the cards so there will remain no value or influence for the ruling council, or even for the Americans who will enter into a second battle with the Shi'a. This is what we want. Then, the Sunni will have no choice but to support us in many of the Sunni regions. When the Mujahidin would have secured a land they can use as a base to hit the Shi'a inside their own lands, with a directed media and a strategic action, there will be a continuation between the Mujahidin inside and outside of Iraq. We are racing against time, in order to create squads of Mujahidin who seek refuge in secure places, spy on neighborhoods, and work on hunting down the enemies. The enemies are the Americans, police, and army. We have been training these people and augmenting their numbers.
As far as the Shi'a, we will undertake suicide operations and use car bombs to harm them. We have been working on monitoring the area and choosing the right people, looking for those who are on the straight path, so we can cooperate with them. We hope that we have made progress, and perhaps we will soon decide to go public - even if gradually - to display ourselves in full view. We have been hiding for a long time, and now we are seriously working on preparing a media outlet to reveal the truth, enflame zeal, and become an outlet for jihad in which the sword and the pen can turn into one. Along with this, we strive to illuminate the hindering errors of Islamic law and the clarifications of Islamic legal precepts by way of tapes, lessons, and courses which people will come to understand.
The suggested time for execution: we are hoping that we will soon start working on creating squads and brigades of individuals who have experience and expertise. We have to get to the zero-hour in order to openly begin controlling the land by night and after that by day, god willing. The zero-hour needs to be at least four months before the new government gets in place. As we see we are racing time, and if we succeed, which we are hoping, we will turn the tables on them and thwart their plan. If, god forbid, the government is successful and takes control of the country, we just have to pack up and go somewhere else again, where we can raise the flag again or die, if god chooses us.
6. What about you? You, noble brothers, leaders of jihad, we do not consider ourselves those who would compete against you, nor would we ever aim to achieve glory for ourselves like you did. The only thing we want is to be the head of the spear, assisting and providing a bridge over which the Muslim nation can cross to promised victory and a better tomorrow. As we have explained, this is our belief. So if you agree with it and are convinced of the idea of killing the perverse sects, we stand ready as an army for you, to work under your guidance and yield to your command. Indeed, we openly and publicly swear allegiance to you by using the media, in order to exasperate the infidels and confirm to the adherents of faith that one day, the believers will revel in god's victory. If you think otherwise, we will remain brothers, and disagreement will not destroy our cooperation and undermine our working together for what is best. We support jihad and wait for your response. May god keep for you the keys of goodness and preserve Islam and his people. Amen, amen.

Leaking the Zarqawi letter, Feb 04

The New York Times - February 9, 2004 Monday - Late Edition - Final

THE STRUGGLE FOR IRAQ: INTELLIGENCE;

U.S. Says Files Seek Qaeda Aid In Iraq Conflict
BYLINE: By DEXTER FILKINS; Douglas Jehl contributed reporting from Washington
for this article.
SECTION: Section A; Column 4; Foreign Desk; Pg. 1
LENGTH: 1456 words
DATELINE: BAGHDAD, Iraq, Feb. 8

†††American officials here have obtained a detailed proposal that they conclude
was written by an operative in Iraq to senior leaders of Al Qaeda, asking for
help to wage a "sectarian war" in Iraq in the next months.
††† The Americans say they believe that Abu Musab al-Zarqawi, a Jordanian
who has long been under scrutiny by the United States for suspected ties to Al
Qaeda, wrote the undated 17-page document. Mr. Zarqawi is believed to be
operating here in Iraq.
††† The document was made available to The New York Times on Sunday, with an
accompanying translation made by the military. A reporter was allowed to see the
Arabic and English versions and to write down large parts of the translation.
†††The memo says extremists are failing to enlist support inside the country,
and have been unable to scare the Americans into leaving. It even laments Iraq's
lack of mountains in which to take refuge.
†††Yet mounting an attack on Iraq's Shiite majority could rescue the movement,
according to the document. The aim, the document contends, is to prompt a
counterattack against the Arab Sunni minority.
†††Such a "sectarian war" will rally the Sunni Arabs to the religious
extremists, the document argues. It says a war against the Shiites must start
soon -- at "zero hour" -- before the Americans hand over sovereignty to the
Iraqis. That is scheduled for the end of June.
†††The American officials in Baghdad said they were confident the account was
credible and said they had independently corroborated Mr. Zarqawi's authorship.
If it is authentic, it offers an inside account of the insurgency and its
frustrations, and bears out a number of American assumptions about the strength
and nature of religious extremists -- but it also charts out a battle to come.
†††The document would also constitute the strongest evidence to date of contacts
between extremists in Iraq and Al Qaeda. But it does not speak to the debate
about whether there was a Qaeda presence in Iraq during the Saddam Hussein era,
nor is there any mention of a collaboration with Hussein loyalists.
†††Yet other interpretations may be possible, including that it was written by
some other insurgent, but one who exaggerated his involvement.
†††Still, a senior United States intelligence official in Washington said, "I
know of no reason to believe the letter is bogus in any way." He said the letter
was seized in a raid on a known Qaeda safe house in Baghdad, and did not pass
through Iraqi groups that American intelligence officials have said in the past
may have provided unreliable information.
†††Without providing further specifics, the senior intelligence officer said
there was additional information pointing to the idea that Al Qaeda was
considering mounting or had already mounted attacks on Shiite targets in Iraq.
†††"This is not the only indication of that," the official said. The intercepted
letter also appears to be the strongest indication since the American invasion
last March that Mr. Zarqawi remains active in plotting attacks, the official
said.
†††According to the American officials here, the Arabic-language document was
discovered in mid-January when a Qaeda suspect was arrested in Iraq. Under
interrogation, the Americans said, the suspect identified Mr. Zarqawi as the
author of the document. The man arrested was carrying it on a CD to Afghanistan,
the Americans said, and intended to deliver it to people they described as the
"inner circle" of Al Qaeda's leadership. That presumably refers to Osama bin
Laden and his deputy, Dr. Ayman al-Zawahiri.
†††The Americans declined to identify the suspect. But the discovery of the disc
coincides with the arrest of Hassan Ghul, a Pakistani described by American
officials at the time as a courier for the Qaeda network. Mr. Ghul is believed
to be the first significant member of that network to have been captured inside
Iraq.
†††The document is written with a rhetorical flourish. It calls the Americans
"the biggest cowards that God has created," but at the same time sees little
chance that they will be forced from Iraq.
†††"So the solution, and only God knows, is that we need to bring the Shia into
the battle," the writer of the document said. "It is the only way to prolong the
duration of the fight between the infidels and us. If we succeed in dragging
them into a sectarian war, this will awaken the sleepy Sunnis who are fearful of
destruction and death at the hands" of Shiites.
†††The author offers his services and those of his followers to the recipients
of the letter, who American officials contend are Al Qaeda's leaders.
†††"You noble brothers, leaders of the jihad, we do not consider ourselves
people who compete against you, nor would we ever aim to achieve glory for
ourselves like you did," the writer says. "So if you agree with it, and are
convinced of the idea of killing the perverse sects, we stand ready as an army
for you to work under your guidance and yield to your command."
†††In the period before the war, Bush administration officials argued that Mr.
Zarqawi constituted the main link between Al Qaeda and Mr. Hussein's government.
Last February at the United Nations, Secretary of State Colin L. Powell said,
"Iraq today harbors a deadly terrorist network, headed by Abu Musab al-Zarqawi,
an associate and collaborator of Osama bin Laden and his Al Qaeda lieutenants."
†††Around that time, the Americans believed that Mr. Zarqawi was holed up in the
mountains at the Iranian border with Ansar al Islam, a group linked to Al Qaeda
that is suspected of mounting attacks against American forces in Iraq.
†††Since the war ended, little evidence has emerged to support the allegation of
a prewar Qaeda connection in Iraq. Last month, Mr. Powell conceded that the
American government had found no "smoking gun" linking Mr. Hussein's government
with Al Qaeda.
†††In the document, the writer indicated that he had directed about 25 suicide
bombings inside Iraq. That conforms with an American view that suicide bombings
were more likely to be carried out by Iraqi religious extremists and foreigners
than by Hussein allies.
†††"We were involved in all the martyrdom operations -- in terms of overseeing,
preparing and planning -- that took place in this country," the writer of the
document says. "Praise be to Allah, I have completed 25 of these operations,
some of them against the Shia and their leaders, the Americans and their
military, and the police, the military and the coalition forces."
†††But the writer details the difficulties that he and his comrades have been
experiencing, both in combating American forces and in enlisting supporters. The
Americans are an easy target, according to the author, who nonetheless claims to
be impressed by the Americans' resolve. After significant losses, he writes,
"America, however, has no intention of leaving, no matter how many wounded nor
how bloody it becomes."
†††The Iraqis themselves, the writer says, have not been receptive to taking
holy warriors into their homes.
†††"Many Iraqis would honor you as a guest and give you refuge, for you are a
Muslim brother," according to the document. "However, they will not allow you to
make their home a base for operations or a safe house."
†††The writer contends that the American efforts to set up Iraqi security
services have succeeded in depriving the insurgents of allies, particularly in a
country where kinship networks are extensive.
†††"The problem is you end up having an army and police connected by lineage,
blood and appearance," the document says. "When the Americans withdraw, and they
have already started doing that, they get replaced by these agents who are
intimately linked to the people of this region."
†††With some exasperation, the author writes: "We can pack up and leave and look
for another land, just like what has happened in so many lands of jihad. Our
enemy is growing stronger day after day, and its intelligence information
increases.
†††"By God, this is suffocation!" the writer says.
†††But there is still time to mount a war against the Shiites, thereby to set
off a wider war, he writes, if attacks are well under way before the turnover of
sovereignty in June. After that, the writer suggests, any attacks on Shiites
will be viewed as Iraqi-on-Iraqi violence that will find little support among
the people.
†††"We have to get to the zero hour in order to openly begin controlling the
land by night, and after that by day, God willing," the writer says. "The zero
hour needs to be at least four months before the new government gets in place."
†††That is the timetable, the author concludes, because, after that, "How can we
kill their cousins and sons?"
†††"The Americans will continue to control from their bases, but the sons of
this land will be the authority," the letter states. "This is the democracy. We
will have no pretexts."

US Department of Defense Information - February 9, 2004

Coalition Provisional Authority Briefing from Iraq
SECTION: DEPARTMENT OF DEFENSE TRANSCRIPTS
LENGTH: 3942 words

†††(Participating were Army Brig. Gen. Mark Kimmitt, deputy director of
operations, Combined Joint Task Force 7, and Dan Senor, senior advisor to the
Coalition Provisional Authority.)
...
†††Senor: Yes, sir?
†††Q: Alan Fryer from Canadian Television. General, I'd like to ask you about
the report in The New York Times today about this letter that they reported on,
purportedly from a high-level al Qaeda person, which essentially lays out a
strategy for sparking civil war in this country by targeting Shi'a. How credible
is it and how seriously do you take it?
†††Kimmitt: We believe the report and the document is credible, and we take the
report seriously -- and we take the threat seriously as well.
†††Q: (Off mike.)
†††Kimmitt: Sorry? Use the microphone.
†††Q: I just wondered if you might elaborate a little bit. And what then might
you be able to do about it or how can you respond?
†††Kimmitt: Well, I think there's a lot that the coalition and the Iraqi people
can do about this. First of all, it is clearly a plan on the part of outsiders
to come in this country and spark civil war, create sectarian violence, try to
expose fissures in this society. And first of all, the Iraqi people have
demonstrated time after time that they are unwilling to participate in any of
these activities by and large. They are looking forward to a free, united and
sovereign Iraq. The coalition has substantial capability, along with the Iraqi
security forces, to use this intelligence and any other follow-up intelligence
to kill or capture those that would try to use this capability to create
anything but a safe and secure environment here in Iraq.
†††Senor: Yeah. I would just add that what is perhaps most striking about that
memo is the extent to which it clearly outlines that the terrorists understand
the stakes in Iraq; that is, they understand that failure to defeat us in Iraq
will be a major setback for their overall terror war. It also outlines what is
working.
†††The memo states that the buildup of Iraqi security forces is putting
increasing pressure on the terrorists. This is a trend that we've been seeing
for some time. Now that we have well over 150,000 Iraqis in security forces in
Iraq, more Iraqis in Iraq today protecting their own country than there are
Americans protecting Iraq, is making it more and more difficult for the
terrorists to operate. It shows that the terrorists are focused on the June 30th
handover of sovereignty; that they recognize that as we politically empower the
Iraqi people, the terrorists will be isolated and it will be harder and harder
for them to operate.
†††As General Kimmitt has said, their strategy is sectarian warfare in an effort
to provoke bloodshed and tear this country apart. Knowing what is working by
their own admission -- ramping up of American security forces, demonstrating
American resolve -- is a very good indication of what we need to continue to be
doing as we move forward to handing over sovereignty to the Iraqi people.
...
†††Jim?
†††Q: Hi. I was wondering, referring again to that New York Times report,
whether you've seen any attacks of this nature, sectarian-type attacks, that
would -- you might blame on al Qaeda, you might have some sort of evidence that
would point to al Qaeda. Things that stick in my mind are the two attacks in
Baqubah, the Shi'ite mosques that were -- one was attacked. There was a car bomb
that failed to go off outside second one. There were a few others. There was
something up in Samarra the other day, the Kurdish attacks, et cetera. I'm
wondering if any of those you can blame on al Qaeda or might be somehow linked
to this memo.
†††Kimmitt: Well, the memo itself says -- the author of the memo himself says
that he accomplished 25 operations since he's been here in Iraq.
†††It is clear that the type of techniques that we have seen in certain of these
attacks, such as what we've seen at the north gate, at the Assassin's Gate,
perhaps in Erbil, perhaps down -- the assassination of Hakim, perhaps the attack
on the U.N. building -- all of these have the fingerprints, as we have said,
month after month, and hallmarks of al Qaeda, fingerprints of al Qaeda and other
foreign fighters. So we can't rule out that the 25 operations claimed inside
that document are untrue. And in fact that just gives us more and more evidence
of these -- that al Qaeda is in fact conducting operations, or people who would
like to work with al Qaeda are operating inside this country.
†††Senor: Yes, sir? In the back there.
†††Q: Kevin Flower with CNN. So back to this letter. Do you think that it is --
is it your belief that Zarqawi is the author of this letter? And who is the
letter to? Was it written to al Qaeda operatives outside the country? And
finally, can we see the letter? Can we -- can you make portions of it available
to us?
†††Kimmitt: Yeah, we are persuaded that Zarqawi was the author of this letter.
It is our understanding that this letter was being taken by a courier outside
this country for delivery abroad. And it is our intent and our -- certainly our
hope that -- in the near future that this letter can be declassified.
†††Let me just give you sort of a picture of the 17 pages of it on the screen
here, not very -- it's not very clear to you. But we are hoping in the near
future to be able to release this, because this document does in fact
demonstrate what we have been assessing all along, and the impact of this letter
on our operations and as we take operations forward is very, very dramatic.
†††Senor: It's an important reminder -- just to add, it's an important reminder
of the -- of why it is so critical that we forge ahead with our plan here to
beef up Iraqi security forces, which are clearly doing an effective job in
combating the terrorists and are making the terrorists feel tremendous pressure
and continuing to move over with the hand-over of political authority to the
Iraqi people.

Zarqawi's Journey: From Dropout to Prisoner to Insurgent Leader

By JEFFREY GETTLEMAN

Published: July 13, 2004

AMMAN, Jordan, July 10 - Ten years ago, fellow inmates remember, Abu Musab al-Zarqawi emerged as the tough-guy captain of his cellblock. In the brutish dynamic of prison life, that meant doling out chores.
"He'd say, 'You bring the food; you clean the floor,' " recalled Khalid Abu Doma, who was jailed with Mr. Zarqawi for plotting against the Jordanian government. "He didn't have great ideas. But people listened to him because they feared him."
According to American officials, Mr. Zarqawi has come a long way from his bullying cellblock days and is now the biggest terrorist threat in Iraq, accused of orchestrating guerrilla attacks, suicide bombings, kidnappings and beheadings. [On Sunday he claimed responsibility for a mortar barrage in Samarra last Thursday that killed five American soldiers and one Iraqi soldier.]
American views of Mr. Zarqawi's relationship to Al Qaeda have varied. Secretary of State Colin L. Powell has described him as a Qaeda operative, but a senior American military official said recently that sources now indicated that Mr. Zarqawi was "a separate jihadist.''
He remains a singular target: American forces are stepping up airstrikes on buildings they believe to be his safe houses in Falluja and have raised the bounty on him to $25 million, the figure offered for Osama bin Laden.
For all that, Mr. Zarqawi remains a phantom, with little known about his whereabouts or his operations.
In Jordan, where he stamped strong impressions on people as he climbed the ladder of outlaw groups, friends and associates described the making of a militant. They say he grew up in rough-and-tumble circumstances and adopted religion with the same intensity he showed for drinking and fighting, though he became far less a revolutionary mastermind than a dull-witted hothead with gruff charisma.
These people, who knew Mr. Zarqawi until he disappeared into the terrorist murk of Afghanistan four years ago, acknowledge that he may have changed. But they say that while the man they knew could be capable of great brutality, they have a hard time imagining him as the guiding light of an Iraqi insurgency.
"When we would write bad things about him in our prison magazine, he would attack us with his fists," said Yousef Rababa, who was imprisoned with Mr. Zarqawi for militant activity. "That's all he could do. He's not like bin Laden with ideas and vision. He had no vision."

Jihad Dreams

Mr. Zarqawi, thought to be 37, grew up fast and hard in Zarqa, a crime-ridden industrial city north of Amman known as Jordan's Detroit.
From his two-story concrete-block house, he looked out on hills dotted with smokestacks. He came from a poor family and has seven sisters and two brothers. His father was a traditional healer. His mother struggled with leukemia. His birth name was Ahmed Fadeel al-Khalayleh.
Childhood friends say he was much like any other boy, chasing soccer balls through gravely streets, doing average work in school, not going to the mosque much. But he liked to fight. "He was not so big, but he was bold," said a cousin, Muhammad al-Zawahra.
At 17, family members say, he dropped out of school. Friends said he had started drinking heavily and getting tattoos, both discouraged under Islam. According to Jordanian intelligence reports provided to The Associated Press in Amman, Mr. Zarqawi was jailed in the 1980's for sexual assault, though no additional details were available.
By the time he cleared 20 he was adrift, his family said, and like other young Arab men looking for a cause, he looked northeast, to Afghanistan.
Saleh al-Hami, Mr. Zarqawi's brother-in-law - who, like many former guerrillas who fought in Afghanistan, has a long black beard and a plastic leg - said Mr. Zarqawi arrived in Khost, in eastern Afghanistan, in the spring of 1989 to join the jihad, or holy war, against the Russians. But he got there a little late. The Russians had just pulled out. So instead of picking up a gun, Mr. Zarqawi picked up a pen.

He became a reporter for a small jihadist magazine, Al Bonian al Marsous, whose name means "The Strong Wall.'' He was 22, with a medium build and shiny black eyes, and roamed the countryside interviewing Arab fighters about the glorious battles he had missed.
Mr. Hami was convalescing in a hospital after he stepped on a land mine when he met Mr. Zarqawi. The two grew close, and he later married Mr. Zarqawi's younger sister.
One night while they were camping in a cave, he recalled, Mr. Zarqawi shared a special dream. He said he had seen a vision of a sword falling from the sky. "Jihad" was written on its blade.

Prison Days

Mr. Zarqawi returned to Zarqa in 1992 and fell in with a militant Islamic group, Bayaat al Imam, or Loyalty to the Imam. He was arrested in 1993 after the Jordanian authorities discovered assault rifles and bombs stashed in his house.
His lawyer said Mr. Zarqawi lamely told investigators that he had found the weapons while walking down the street. "He never struck me as intelligent," said the lawyer, Mohammed al-Dweik.
Mr. Zarqawi was sent to Swaqa prison, on the desert's edge. He was housed with other political prisoners in a large room with iron bunk beds. Cellmates said Mr. Zarqawi turned his bunk into a cave, covering each side with blankets. He sat for hours bent over a Koran, trying to memorize all 6,236 verses.
Friends said this was typical. When he was a drinker, they said, he was an extreme drinker. When he was violent, he was extremely violent.
He strutted around in Afghan dress and a woolly Afghan hat and lived and breathed old Afghan battles. "Back then, he liked Americans," Mr. Abu Doma said. "Abu Musab used to say they were Christian and they were believers."
The Russians were his No. 1 enemy, but this, like many other beliefs, would change behind bars. In the wing where Mr. Zarqawi lived, ideologies scraped up against one other. But cellmates said he shied away from politics. Instead, he pumped iron. Cellmates remember his barbells, made from pieces of bed frame and olive oil tins filled with rocks.
As the years passed, Mr. Zarqawi's arms and chest grew - and so did his role. He mapped out shifts for cleaning, bringing meals to cells and visiting the doctor. He did not talk much. When asked to describe him during this period, almost everyone interviewed began with the word "jad," which means serious.
His firmness was his attraction, fellow inmates said, his remoteness his power. By 1998, when a prison doctor, Basil Abu Sabha, met him, Mr. Zarqawi was clearly in charge.
"He could order his followers to do things just by moving his eyes," Dr. Abu Sabha said.
His religious views became increasingly severe. They had been marinating in a stew of militant beliefs served up by the imams and sheiks in the iron bunks next to him. He lashed out at cellmates if they read anything but the Koran.
Mr. Abu Doma said he got a threatening note for reading "Crime and Punishment."
"He spelled Dostoyevsky 'Doseefski,' Mr. Abu Doma said, laughing. "The note was full of bad Arabic, like a child wrote it."
Fellow inmates said that around that time, 1998, just as Al Qaeda was emerging as a serious threat blamed for the two bombings of United States Embassies in Africa, Mr. Zarqawi started talking about killing Americans.

Adrift Again

In March 1999, Mr. Zarqawi was released under an amnesty for political prisoners. His associates said they expected him to return to jail.
"Because of his views, there was no place for him in Jordan," said Mr. Rababa, explaining that the country, tempered and mostly secular, was no place for an extremist. As for himself, Mr. Rababa said he had found a place in Jordan because his views had matured.
But for Mr. Zarqawi, Mr. Rababa said, "everyone was the enemy."
Mr. Zarqawi also had hopes for a normal life, according to Mr. Hami, who said he had at least two children and had thought of buying a pickup truck and opening a vegetable stand.
"You could tell he was confused," Mr. Hami said.
In early 2000, Mr. Zarqawi went to Peshawar, Pakistan, at the Afghan border. It was a deeply religious city, which made it attractive to him. He even took his aging mother.
But at the doorstep to jihad, he hesitated.
"He said it was Muslims fighting Muslims in Afghanistan and he didn't believe in the cause," Mr. Hami said. "And he liked the air in Peshawar and thought it was a good place for his mother."
Mr. Zarqawi's family said he was especially close to her, kissing her forehead every time he walked in the door.
While he was deciding what to do, his Pakistani visa expired. Around the same time, Jordan declared Mr. Zarqawi a suspect in a foiled terror plot against a Christian pilgrimage site.
"At that point, he had nowhere else to go," Mr. Hami said.

In June 2000, Mr. Hami said, Mr. Zarqawi crossed into Afghanistan, alone. His mother died of leukemia in February of this year at age 62. Mr. Hami said her last wish was for her son to be killed in battle, not captured.

Terrorist Connections

American intelligence officials said Mr. Zarqawi opened a weapons camp connected to Al Qaeda in late 2000 in western Afghanistan. There he took up his nom de guerre, with Zarqawi a reference to his hometown of Zarqa.
United States officials said he was wounded in a missile strike after the Sept. 11, 2001, terror attacks when American forces went after the Taliban and Al Qaeda.
Intelligence officials say he then left Afghanistan, where he had taken a second wife, and made his way to a corner of northern Iraq controlled by a Kurdish separatist Islamic group called Ansar al-Islam.
The next sighting of Mr. Zarqawi was on Sept. 9, 2002, when Jordanian agents said he illegally entered Jordan from Syria.
A month later Laurence Foley, a senior American diplomat, was fatally shot outside his home in Amman. Jordanian agents arrested three men who, the agents said, told them that they had been recruited, armed and paid by Mr. Zarqawi. He was sentenced to death in absentia.
On Feb. 5, 2003, Secretary of State Powell made his assertions about Mr. Zarqawi at the United Nations.
Mr. Powell stands by his statement, a spokesman said this month, even though other parts of that speech have been discredited and Mr. Powell mistakenly identified Mr. Zarqawi as Palestinian. He actually is of the Beni Hassan tribe, with roots deep in the Jordanian desert.
Other American information about Mr. Zarqawi has also been incorrect. At first it was said that he had a leg amputated during a Baghdad hospital visit, but now, a senior United States military official said in an e-mail message, "we believe Zarqawi has both legs, and reporting of the missing limb was disinformation."
At the beginning of the war in Iraq, Mr. Zarqawi and the Ansar fighters were driven out of the country. In August a car bomb blew up the Jordanian Embassy in Baghdad, the first in a deadly wave of bombings. Mr. Zarqawi, because of his history as an anti-Jordan militant, was immediately a suspect.
In February, American officials in Baghdad released a 6,700-word letter - outlining a terror strategy to drag Iraq into civil war - that they said had been found on a CD from Mr. Zarqawi to Al Qaeda's leadership. But people who know Mr. Zarqawi wonder if he was the author. They said the lengthy political analysis, the references to seventh-century kings and embroidered phrases like "crafty and malicious scorpion" do not sound like him.
"The man was basically illiterate," Mr. Abu Doma said, though he acknowledged that a learned acolyte could be helping him.
Americans officials stand by their identification. They said the letter had been seized from a courier working for Mr. Zarqawi, who calls his group the Tawid and Jihad Movement.
The mystery remains. On May 11, a video appeared, titled "Sheik Abu Musab Zarqawi Slaughters an American Infidel." It showed the beheading of Nicholas Berg, the young Pennsylvania businessman. American officials believe that Mr. Zarqawi may have been the killer.
Back in Amman, there are questions. The killer on the video cuts with his right hand. While Mr. Hami said he thought Mr. Zarqawi was right-handed, Mr. Rababa and Mr. Abu Doma, who shared the same room with him for several years, insisted that he used his right hand only for eating and shaking hands.

Abdallah Abu Romman contributed reporting for this article.

How the Zarqawi myth was made in America

US and Britain used stories of Al Qaida operatives to justify the war on Iraq. Loretta Napoleoni exposes the truth

The first time I heard the name of Musab al-Zarqawi was on 5 February 2003 when the then US secretary of state Colin Powell singled him out as the link between Al Qaida and Saddam Hussein.
At the time I knew that this connection was fictitious. I had proof that such a link not only did not exist, but that Al Qaida had tried to approach Saddam in 2000 and he had refused to talk to them.
I began reading jihadist web pages and reports from the Arab press. Little by little the life history of al-Zarqawi began taking shape.
Al-Zarqawi is the ultimate product of Al Qaidism, the new anti-imperialist ideology which has risen from the ashes of Al Qaida.
His childhood and youth coincided with the hardening of the Palestinian diaspora and the struggle against the Soviet army in Afghanistan.
Al-Zarqawi's real name is Ahmed Fadel al-Khalaylah. He was born at the end of October 1966 in Zarqa, Jordan.
Zarqa is an industrial city encircled by Palestinian refugee camps. It is a poor city where unemployment is rampant.

Values

Al-Zarqawi grew up in one of the poorest working class neighbourhoods, Masum, where traditional tribal values clashed daily with rapid modernisation. He was aware of the Palestinian struggle - his father had participated in the battle of East Jerusalem in 1948.
At 16, following the death of his father, he dropped out of school, joined a local gang and eventually spent time in prison.
When he was released he began frequenting the local mosque, where he was recruited to join the jihad (holy war) in Afghanistan.
The idea to become a mujahideen (resistance fighter) appealed to him because he had a romantic idea of the Arab warrior.
He was not aware of the politics behind the war in Afghanistan. He was a very simple and uneducated man.
He arrived in Afghanistan in 1989, after the Soviet army had abandoned the country, and never participated in any battle. He went to work as a junior for the Arab-Afghan bureau in Peshawar where he met al-Maqdisi, a well known intellectual, who introduced him to radical Salafism, a doctrine which calls for a return to the purity of Islam.
In 1993 al-Maqdisi and al-Zarqawi returned to Zarqa where they planned to overthrow the Jordanian regime. They were arrested and imprisoned for five years.
During al-Zarqawi's time in Afghanistan major changes were taking place inside Al Qaida.
The organisation was taken over by Osama bin Laden and Ayman al-Zawahiri, an Egyptian doctor who was a member of Egyptian Islamic Jihad.
During the 1990s bin Laden and al-Zawahiri reshaped Al Qaida into an armed organisation for international jihad.
Al-Zarqawi's journey to becoming an international leader of terror took place in prison. Torture and solitary confinement boosted his determination to challenge authority.
He showed strong leadership qualities and organisational skills. The inmates elected him their leader.
People were impressed by his determination and his kindness. Once he personally bathed a mujahideen who had been injured and had lost a leg.
Al-Zarqawi was convinced that the jihad had to focus on overthrowing the corrupt Arab regimes.
In 1999 when he was released from prison, al-Zarqawi returned to Afghanistan.
In 2000 he met Osama bin Laden who offered him and his followers the opportunity to join Al Qaida.
Al-Zarqawi refused because he was not prepared to fight the Americans. He wanted to continue his struggle against the Jordanian regime.
Al-Zarqawi was able to convince the Taliban to fund a small camp in Herat near the border with Iran.
The camp was frequented by Palestinians, Jordanians and Syrians. It forged suicide bombers. While in Herat, al-Zarqawi established links with a group of Jordanians from the city of Salt.
The group had moved to Iraqi Kurdistan where they joined Ansar al-Islam, an armed organisation linked to Al Qaida. After the fall of the Taliban he found refuge in Iraqi Kurdistan.
The fall of the Taliban regime shattered Al Qaida.
In January 2002 the Kurdish secret service told the US that al-Zarqawi was Al Qaida's man in Iraqi Kurdistan.
The US contacted the Jordanian authorities who immediately blamed him for a foiled attack during the millennium celebration in Jordan, the assassination of an Israeli citizen and of the US diplomat Laurence Foley. Neither the Kurds or the Jordanians could back their accusation with any evidence.

Scapegoat

The Kurds wanted the US to help them get rid of the jihadists in Iraqi Kurdistan and the Jordanians needed a scapegoat for a series of mysterious terrorist attacks.
The US administration welcomed the creation of the myth because they needed a reason to go to war with Iraq.
Today we know that most of Colin Powell's speech on 5 February 2003 was based on false information. But at the time when he mentioned al-Zarqawi as the new international terror leader the entire world believed him.
From that moment, a totally unknown leader of a small and insignificant group became the new international bogeyman.
Colin Powell's speech also provided the jihadist movement with a new, much needed operational leader.
With Osama bin Laden and al-Zawahiri trapped in Pakistan's tribal belt, al-Zarqawi became the new icon of anti-Western struggle.
His myth helped the transition of Al Qaida from a small, highly integrated armed organisation to a global anti-imperialist creed.
All jihadists scattered around the world wanted to be related to al-Zarqawi. Funds and future suicide bombers flocked to Iraq to join his group.
Ironically, while his myth boosted al-Zarqawi's popularity outside Iraq, in Iraq he was regarded with suspicion.
He waited until August 2003 to enter the fighting, after the end of the official war, when the Shia insurgency was already in full swing and the population had turned against the occupation.
The US claimed he was the leader of the resistance in Fallujah, a myth debunked by the discovery of diaries that admit his supporters accounted for only 15 of the estimated 5,000 fighters in the city.
From August 2003 until December 2004, when Osama bin Laden nominated him leader of Al Qaida in Iraq, he sought recognition from bin Laden because he lacked the legitimacy to rally the Sunni population.
Right from the beginning he was determined to drive a wedge between Sunnis and Shias to prevent them from uniting in a national front.
At the same time he became fully committed to the fight against the US. Thus his struggle was conducted from the beginning on two fronts-one against the Shia and one against coalition forces.
The tactics of suicide attacks, kidnappings and beheadings of Western hostages strengthened and confirmed his status.
But al-Zarqawi has not reached the top through natural selection inside the jihadist movement - he has been pushed there by those who have created his myth - the Kurds, the Jordanians and the US.
Loretta Napoleoni is an Italian journalist who has worked for the IMF, the UN and advised the US homeland security department on terrorism. She is author of Terror Inc. Her new book, Insurgent Iraq, is available from Bookmarks. Phone 020 7637 1848

Does Al-Zarqawi exist??

Revealed: truth behind US 'poison factory' claim

09 Feb 2003: The Observer - Page 1 - (1103 words)

Revealed: truth behind US 'poison factory' claim: Luke Harding reports from the terrorist camp in northern Iraq named by Colin Powell as a centre of the al-Qaeda international network

By: Luke Harding
IF COLIN POWELL were to visit the shabby military compound at the foot of a large snow-covered mountain, he might be in for an unpleasant surprise. The US Secretary of State last week confidently described the compound in north-eastern Iraq - run by the Islamic terrorist group Ansar al-Islam - as a 'poison and explosives training camp'. He suggested the camp was a centre for teaching terrorists how to produce ricin and other chemical weapons.

Yesterday, however, it emerged that the terrorist factory was nothing of the kind - more a dilapidated collection of concrete outbuildings at the foot of a grassy, sloping hill. Behind the barbed wire and a courtyard strewn with broken rocket parts are a few empty concrete houses.

There is a bakery. There is no sign of chemical weapons anywhere - only the smell of paraffin and vegetable butter used for cooking. In the kitchen, I discovered some chopped up tomatoes but not much else. The cook had left his Kalashnikov propped neatly against the wall.

Ansar al-Islam - the Islamic group that uses the compound identified as a military HQ by Powell - yesterday invited me and several other foreign journalists into their territory for the first time.

'We are just a group of Muslims trying to do our duty,' Mohammad Hasan, spokes-man for Ansar al-Islam, explained. 'We don't have any drugs for our fighters. We don't even have any aspirin. How can we produce any chemicals or weapons of mass destruction?'

The radical terrorist group controls a tiny mountainous chunk of Kurdistan, the self-rule enclave of northern Iraq. Over the past year Ansar's fighters have been at war with the Kurdish secular parties which control the rest of the area, launching murderous attacks that have resulted in hundreds of casualties. Every afternoon both sides mortar each other across a dazzling landscape of mountain and shimmering green pasture. Until last week this was an obscure and parochial conflict.

But last Wednesday Powell suggested that the 500-strong band of Ansar fighters had links with both al-Qaeda and Saddam Hussein. They were, he hinted, a global menace - and more than that, they were the elusive link between Osama bin Laden and Iraq.

This is clearly little more than cheap hyperbole. Yesterday Hassan took the unprecedented step of inviting journalists into what was previously forbidden territory in an almost certainly doom-ed attempt to prevent an American missile strike once the war with Iraq kicks off.

Ali Bapir, a warlord in the neighbouring town of Khurmal, lent us several fighters armed with machine guns and we set off. We drove past an Ansar checkpoint, marked with a black flag and the Islamic militia's logo - the Koran, a sheaf of wheat and a sword.

The landscape was littered with the ruins of demolished houses, destroyed during Saddam's infamous Anfal campaign against the Kurds in 1988. At the corner of the valley we passed a pink mosque, with sandbagging on the roof. Washing hung from a courtyard. A group of Ansar fighters in green military fatigues smiled and waved us on.

Several of their comrades were in the graveyard across the road. There were numerous fresh plots, each marked with a black flag. After 20 minutes' drive along a twisting mountain track we arrived in Serget - the village identified from space by American satellite as a haven of terrorist activity.

But Hassan was at pains to deny any link with al-Qaeda. 'All we are trying to do is fulfil the prophet's goals,' he said. 'Read the Koran and you'll understand.'

Senior officials from the Patriotic Union of Kurdistan - the party with which Ansar is at war - insist that the Islamic guerrillas based in the village have been experimenting with poisons. (?IE IT WAS THE KURDS WHO PUT THIS UP TO THE US, IN ORDER TO GET THESE PEOPLE OFF THEIR BACKS) They have smeared a crude form of cyanide on door handles. They had even tried it out on several farm animals, including sheep and donkeys, they claim. The guerrillas have also managed to construct a 1.5kg 'chemical' bomb designed to explode and kill anyone within a 50-metre radius, Kurdish intelligence sources say.

Hassan yesterday dismissed all these allegations as 'lies'. 'We don't have any chemical weapons. As you can see this is an isolated place,' Ayub Khadir, another fighter, with a bushy pirate beard and blue turban, said. And yet, despite the fact there appeared to be no evidence of chemical experimentation, Ansar's complex was lavish for an organisation that purports to be made up merely of simple Muslims. Concealed in a concrete bunker, we discovered a sophisticated television studio, complete with cameras, editing equipment and a scanner.

In a neighbouring room were several computers, beneath shelves full of videos. A banner in Arabic proclaims: 'Those who believe in Islam will be rewarded.'

Until recently Ansar had its own website where the faithful could log on to footage of Ansar guerrillas in battle. In small concrete bunkers the fighters operated their own radio station, Radio Jihad. The announcer had clearly been sitting on an empty box of explosives. Hassan denied yesterday that his revolutionary group received any funding from Baghdad or from Iran, a short hike away over the mountains.

'If Colin Powell were to come here he would see that we have nothing to hide,' he said. But Ansar's sources of funding remain mysterious - and their real purpose tantalisingly unclear. 'All Ansar fighters are from Iraq,' Hassan said. 'Iraq is one of the richest countries in the world. Our fighters have brought their own things with them.'

But while they appear to pose no real threat to Washington or London, Ansar's fighters are a brutal bunch. They have so far killed more than 800 opposition Kurdish fighters. They have shot dead several civilians. They have even tried - last April - to assassinate the head of the regional administration in the neighbouring town of Sulamaniyah, the mild-mannered Dr Barham Salih. The plot went wrong and two of the assassins were shot dead. A third is in prison. 'We are fed up with them. We wish they would go away,' said one villager, who refused to be named.

The militia's weapons had been inherited, captured from their enemies or bought from smugglers, Hassan said. Kurdish intelligence sources insist that there is 'solid and tangible proof' linking Ansar both to Iraqi intelligence agents and to al-Qaeda. They say that a group of fighters visited Afghanistan twice before the fall of the Taliban and met Abu Hafs, one of bin Laden's key lieutenants.

Hassan yesterday refused to say how many fighters were holed up in the three villages and one mountain valley under Ansar's control ('It's a military secret,' he said) and claimed - implausibly- that none of his men were Arab volunteers come to fight jihad in Iraq.

Foreign involvement in the Iraqi insurgency

By Ahmed Hashim

The Iraqi insurgency spiked again in August 2004 when Muqtada al-Sadr took the offensive against the transitional Iraqi government of Prime Minister Iyad Allawi and the Multi-National Force of U.S. and other foreign troops, as the former coalition is now known. It was optimistically believed that following the return to Iraqi sovereignty at the end of June, the insurgency by both Sunni and Shi'a groups would wither away. It has not, and the issue of foreign involvement with insurgent groups - which has hovered in the background since last year - came to the foreground in the summer of 2004. U.S. General Richard Myers, chairman of the Joint Chiefs of Staff, highlighted the issue with regard to Syria when he adamantly stated: "We know that the pathway into Iraq for many foreign fighters is through Syria. It's a fact. We know it. The Syrians know it." [1] More recently, the claim by the Iraqi Defense Minister Hazem Sha‘alan in July that Iran was interfering in Iraqi domestic affairs by allowing or promoting infiltration into Iraq has led to a significant contretemps between the two neighbors.

The question of foreign insurgents in Iraq presents a particularly tangled problem. Layers of complexity beneath a seemingly simple surface make it difficult to untangle fact from fiction when discussing the issue. Though the Bush administration has maintained that attacks are the work of "regime dead-enders" and foreign infiltrators, hard empirical evidence - often from the U.S. military forces - indicates that the foreign element is minuscule. Evidence which shows that of 8,000 suspected insurgents detained in Iraq, only 127 hold foreign passports, supports this latter claim. But a simple head-count does not tell the whole story. The insurgency's foreign element has had a greater impact than mere numbers would lead us to believe.

Un-sponsored Foreign Insurgents

This insurgency has seen its share of outraged and disgruntled individuals, Arab nationalists, and "un-sponsored" religious extremists make their way into Iraq to fight the foreign occupation. Many Palestinians were recruited to fight in Iraq in 2002, and some joined the regime's irregular force, the Feda‘yeen Saddam. [2] Similarly, large numbers of Syrian volunteers with close tribal and cultural links to Iraqis across the border felt it was their duty to fight. These individuals received no encouragement from their government. One such fighter, a 26-year-old Syrian named "Abed," decided to fight in Iraq barely a week after the war began because, as he put it, "there was something inside that made me explode." Another, a Saudi captured in Iraq named Mohammad Qadir Hussein, was a poor, disgruntled individual who had no military training, but who was motivated by an abiding desire to help other Muslims in distress. [3]

The collapse of Iraqi border controls facilitated the entry of un-sponsored insurgents into Iraq, while Iraqi middlemen or facilitators provided logistical support (i.e. food, directions, and weapons and ammunition) once these individuals had gained entry into the country. Un-sponsored foreign infiltrators are then "passed on" to Sunni imams who became their mentors. Many of these foreign infiltrators entered Iraq before the start of Operation Iraqi Freedom. While poorly trained and ill-equipped, a substantial number of them fought doggedly and to the death in some of the battles between Iraqi irregular forces and the coalition advancing from the south. After the end of Operation Iraqi Freedom, some returned home, while others remained and fought in the insurgency. Many of these gravitated towards the more disciplined jihadist insurgents.

Non-State Actors and Organizations

Foreign insurgents who come in as part of a "package" sent into Iraq by non-state actors are a more formidable force than un-sponsored foreign infiltrators. There is growing evidence that Iraq has begun to attract foreign Islamists and anti-American groups such as al-Qaeda and the Tawhid organization of the elusive and enigmatic Jordanian-Palestinian terrorist, Abu Mus‘ab al-Zarqawi, for whom Iraq is a new and easily accessible battlefield.

Uncertainty regarding the level or depth of al-Qaeda presence in Iraq remains due to a lack of non-politicized intelligence on its activities in that country. Osama bin Laden and his subordinates did not think much of Saddam Hussein and his regime, with evidence showing that the feelings were mutual. In the early days of the war, when there was an influx of foreign volunteers into Iraq, Hussein apparently warned the Ba‘ath party against close links with outsiders, especially religious extremists. [4] A senior Islamist operative (now deceased) allegedly authored a text entitled "The Future of Iraq and the Peninsula After Baghdad's Fall: The Religious, Military, Political and Economic Future." The work argues that the fall of the Ba‘athist regime was "better for the Islamists than the victory of the Iraqi Ba‘athists, because the collapse of Arab Ba‘athism means the collapse of the atheist, pan-Arab slogans that swept the Muslim nation...the demise of the Ba‘ath government in Iraq heralds the hoisting of the Islamic banner over the debris." [5] Such fighters were attracted to Iraq following the war precisely in order to fight the U.S. presence in that country for the sake of Islam.

Once in Iraq, "sponsored" jihadists needed to create a logistical infrastructure, as infiltrating heavy weapons and explosives across the borders of Iraq's neighbors is difficult. [6] For this they needed the help of Iraqis. Mutual suspicion between Sunni Islamists and former regime loyalists, secular-minded nationalists, and tribal elements actively opposing the Coalition does not mean that the latter groups are averse to providing logistical support for the former. Attempts by foreign jihadist organizations to operate in Iraq depend on the resources, protection and concealment provided to their fighters by Iraqis. Unable to enter into Iraq with the resources they need or blend in with the local population, these foreign elements would be lost without support from within Iraq.

Salafists in Iraq

The importance of the foreign jihadists who adhere to a strict interpretation of Sunni Islam (known properly as Salafism but popularly as Wahhabism) lies in three distinct areas. Firstly, these foreign jihadists have coupled with local Iraqi Salafists - who emerged into the open following the downfall of the Saddam regime - to successfully introduce a cohesive and extreme ideology to the public. While many of these groups, like the Mujahideen al-Salafiyah in Balad, have even reached out to members of the former Feda‘yeen Saddam as long as the latter drop their allegiance to Saddam.

Secondly, they have increased the prospects for communal violence by waging a campaign of deliberate and focused attacks against leaders of other Muslim communities, promoters of "moral laxity," and non-Muslims. In the fall of 2003, Islamists were particularly active in Mosul, where they attacked a nunnery, killed a well-known writer, bombed a popular cinema, and torched four liquor stores. The worst atrocity came with the bombings of Christian churches this summer.

Thirdly, they have been responsible for the suicide bombing campaigns in Iraq between early fall 2003 and summer 2004. August 2003 saw three massive car bombings. Some of the most devastating suicide attacks came in mid-November 2003 against Italians in Nasiriyah and in mid-January 2004 outside a Coalition Provisional Authority (CPA) compound in Baghdad. In March 2004, the Shi‘a religious celebration of Ashura witnessed multiple suicide bombings which killed hundreds. [7]

However, as of summer 2004, it is increasingly evident that the different agendas and modus operandi of the nationalist Iraqi insurgency and their ostensible jihadist allies have caused considerable tensions between these groups. While they admire the motivation and skills of the foreigners, many mainstream Islamist and tribal insurgents resent an ideological agenda which has resulted in the killing of Iraqis simply for not adhering to a strict religious line. The foreign fighter's apparent blood lust, which has led to indiscriminate attacks and the beheading of abductees, also angers Iraqi nationalists. [8] In early summer 2004, nationalist insurgents in Fallujah were about to assault a group of foreign jihadists based in the Jolan suburb, led by a Saudi with the nom de guerre of Abu Abdullah. Later, insurgent "authorities" in Fallujah - largely made up of former military personnel and Iraqi police and led by clerics - succeeded in evicting a number of non-Iraqi terrorists from the area.

State Support for the Insurgency

The Bush Administration has accused two of Iraq's neighbors, Syria and Iran, of facilitating or actively encouraging foreign fighters to cross the Iraqi border. The singling out of these two countries, despite the fact that Jordan, Saudi Arabia and Turkey also maintain porous borders with Iraq, reflects the political dynamics at play as the U.S. tries to stabilize the Great Middle East.

Syria and Iran fear the U.S. will succeed in its (unstated) goal of implementing a pro-American "puppet regime" in Baghdad. Such a regime would allow U.S. bases to operate in Iraq, giving U.S. forces the ability to threaten these countries. Both Tehran and Damascus see each other as the next U.S. target for regime change. The logical response is to support anti-US operations in Iraq, thereby ensuring that the hostile Bush Administration remains mired there. However, this is a very risky endeavor on many levels.

Both countries understand that to overtly support anti-US forces in Iraq risks incurring America's wrath. Not long after the end of Operation Iraqi Freedom, warnings from the Bush administration to Syria and Iran not to help the nascent insurgency were issued from a position of strength. Both Syria and Iran bent over backwards to avoid irritating a U.S. that was itching for a fight. Indeed, there were reports that U.S. Special Operations Units undertook actions across the border into Syria and actually clashed with Syrian border guards. Therefore, the growing U.S. problems in Iraq by fall 2003 must have been a source of considerable satisfaction to both Tehran and Damascus.

While neither could overtly support the insurgency, it is not too far-fetched to assume that they did so covertly or turned a blind eye to pro-insurgent activities conducted by elements within their respective countries. Both Syria and Iran have domestic constituencies that are thoroughly hostile to the U.S., and furthermore, alarmed by the belligerent attitude taken by Washington towards their respective countries. Arab nationalists in Syria, for example, are inclined to lend support to the remnants of the Iraqi Ba'ath party. Meanwhile, Iranian groups like the Revolutionary Guards might be inclined to support Shi'a insurgents such as the Mahdist Army led by Muqtada al-Sadr.

But there are attendant risks. Firstly, neither country wants continued instability on their borders. Secondly, neither country is particularly enamored of the leading ideological elements responsible for the violence in Iraq. As a regime dominated by the minority ‘Alawis (thoroughly despised by Sunni extremists), Syria does not want to see the growth of Sunni extremism in Iraq. Nor does secular Damascus wish to see a theocratic Baghdad, despite its sympathy for and traditional alliance with the Shi‘as. For its part, Iran is hardly likely to support Sunni extremists or Arab nationalists. Both are antithetical to Tehran's agenda. Instead, Iranians continue to support Shi'a groups that are not fighting the U.S., in the plausible and logical expectation that these parties will play a leading role in Iraqi politics once the U.S. has left Iraq.

So, while the restraints of Middle Eastern realpolitik keep states from openly supporting foreign insurgents against the coalition in Iraq, there are many other factors and organizations that contribute to this continuing and complex problem.

Notes:
1. Quoted in The Washington Times, April 30, 2004.
2. Islamist groups, on the other hand, recruited from among the growing population of disgruntled Islamists in Jordanian cities such as Ma‘an.
3. Personal interviews with the author.
4. New York Times, January 14, 2004, p.1.
5. Quoted in al-Hayat, December 20, 2003, p.4.
6. Cross-border traffic between Iraq and its neighbors by smugglers and tribes existed even in the best of times when Iraq was able to police its borders. Now even though the borders are not effectively policed, foreign infiltrators are unlikely to come into Iraq on their Sports Utility Vehicles - which outrun the two-wheel drives of the border patrols - laden with large quantities of light weaponry or explosives. Nor do they have to since Iraq is one huge ammunition dump.
7. The Independent (London), March 07, 2004.
8. For more see The Daily Star (Beirut) July 16, 2004.