The following six articles by Nick Davies attempt to investigate the foundations of our criminal justice system. They are posted here as a detailed example of the way in which, as the book puts it: "A Flat Earth story can end up being passed backwards and forwards between government and media, like a mud pie in some children's game, until between them they create something which delights their imagination but which, in reality, is just a mess. And this falsehood and distortion passes into government policy." These articles relate to a passage which begins on page 37 of the book.

System failure: How to lose the fight for law and order

The Guardian June 2003

Right there. That's where they got the Yardie guy. He was in that pub, the Jolly Roger, over on the corner of All Hallows Road and, although it's dark now and our van is racing, we can still catch a glimpse of the lamplit pavement where he lay with his blood pooling over the kerb and onto the tarmac street.

There were people standing all around, but it was one of those times when nobody saw anything. It seems like the men with the knives sent a message into the pub - "Tell Chrissie to come outside" - and for some reason (maybe he was stupid, maybe he was just too cool to be scared) he went out to them, and they slashed him 24 different times, tore open his belly and stabbed him through the skull on the neatly swept pavement of a quiet street in a city in south west England.

The inspector in the van says they never caught the people who did it. They tried. They arrested something like 19 different men including a former police officer - most of them Yardies from East Kingston, which is the original home of the Hype Crew and the Mountain View Posse and the Back to Back Gang and of most of the other Jamaican gangsters who have moved into this part of Bristol. But they couldn't prove anything against anybody. They just didn't have the evidence.

And that's how it is. The inspector knows it. Any police officer of any rank knows it. What see you on The Bill is not what you get in real life. Most of the time, most criminals get away with most of their crimes. The Home Office know it too. They crunched together their best statistics and analysed a hundred typical offences and then they worked out how many of them were actually brought to justice (which means a conviction, a caution, or being 'taking into consideration' for sentencing). And the answer they found was..... three. To put it the other way around: as far as we know, from the best research available, 97% of offences are never brought to book.

And that's the strange thing about this criminal justice system. It has power and, ever since Sir Robert Peel first picked up a truncheon, it has been gathering more. It has money. It is the one and only limb of the public sector which escaped the cuts of the Tory years. Even defence took a dip with the 'peace dividend' after the fall of the Soviet Union. But all through the years of the cuts, the funding of the enforcement of the law grew, from less than £5 billion in the early 1980s until, by the end of the last Major government, we were spending £16.2 billion a year on chasing and punishing criminals, most of whom were not Yardie gangsters but pimply adolescents armed with nothing more sophisticated than the sawn-off top of a Pepsi bottle (they use it to prise open Yale locks). And they were still getting away with 97% of their crime. The criminal justice system bristles with questions. There is only one that really matters. Who's winning?

An hour ago, the inspector addressed the troops. They were in the old police gym, 40 men and women in white shirts and black stab-proof vests, sitting on crooked rows of plastic chairs in amongst the battered punch bags and weight-lifting machines. A rusty bucket collected rainwater at the bottom of the concrete steps. It was very quiet while the inspector spoke, and they sat with their arms folded and their chins on their chests. "This is a really serious situation," he told them.

He described how the Yardies were being attacked by Bristol-born dealers who call themselves the Aggi Crew. The Aggis used to run the St Paul's area in the city centre, which is a magnet for drug deals, but a few years ago, the police busted them, and they all went to prison. Now they're out again and they want their turf back. A few weeks ago, five of them put on balaclavas and picked up guns and made a personal tour of the local bars - the Black Swan, the Malcolm X Centre, Lebeqs and the Caribbean Club - and announced that they were taking over. They offered the Yardies a deal: they could stay and work in St Paul's, but they'd have to pay a tax for the privilege. £50 a day for each Yardie and £100 a day for any business they were running.

The Yardies didn't like that idea. They met in one of the local cafes that has an illegal gambling room in the back, carrying their Mag 9 automatic pistols and their Brocock Magnum air pistols converted to fire live rounds, and they agreed that, first, they would not be paying the tax, and second, they would take on the Aggis. But one of the Yardies changed sides and went over to the Aggis (for a fee). The next night, in Badminton Road, he caught up with one of his former mates and shot him through the back of the knee with a 9mm handgun. The main man in the Aggi Crew repeated the message in the Caribbean Club, where he started jeering at a bunch of Yardies; he attacked one of them, a guy called Dufus, and hit him across the face with his pistol; Dufus went home and got his gun and came back and shot the Aggi (didn't kill him.)

So, naturally, the Aggis attacked the Yardies' street-dealing operation, staged some rip-off robberies, stole their drugs at gun point, and kidnapped one of them - they hauled this guy out of his home, locked him up in the boot of their car and drove him off somewhere quiet where they pistol-whipped him. And just to make the point that they were in charge now, two Aggis walked into the Black and White Cafe on Grosvenor Road and robbed every single Yardie in there at gun point.

If tonight's the night that the Yardies hit back, there is going to be blood on the streets. The inspector tells his officers: "The danger is that there are people out there who have killed before, in other countries." He tells them their priorities: above all, community reassurance; apprehend drug dealers; take firearms off the street; public safety; flexible response to whatever goes off. In one corner of the gym, a group of the officers are carrying handguns. For the first time in the history of Bristol (and for almost the first time in the history of any British city) there will be armed police on the streets - not covertly in plain cars, but overtly where the people of St Paul's can see the protection and where the gangsters can see the threat.

But the rest of them will be unarmed. The inspector tells them how to contact the special intelligence cell, to check car numbers and IDs; how to link up with the 24-hour charging unit who will handle all the paperwork on arrests. He shows them mugshots of the Aggis and warns them to look out for the mixed-race male with the burgundy scarf who shot and pistol-whipped a man on Saturday night. He has a silver revolver, his mate has a seven-inch knife. He tells them not to use their radios - there are scanners out there, so stick to mobile phones. "And don't self-deploy," he tells them. "This is about your safety as well as public safety."

There has never been a human community without crime. There are thousands of specialist academics around the world who have tried to put their finger on why some communities generate so much more than others, why some individuals are so criminal, why some victims are so vulnerable. Some thought that crime was linked to consumption: more goods, so less need to steal. Then they discovered the opposite: more goods, so more opportunity to steal. Then they discovered the opposite again: more goods, so lower prices, so less that is worth stealing. No theory fits. Not poverty, or inequality, or maternal deprivation or paternal absence. They all have some impact - big sources and small sources, all constantly shifting in power and in relationship to each other. It's like trying to map the wind - infinitely complex.

And crime control is just as complicated. Councils put up street lights: the night crime falls; but the day crime falls too. Nobody knows why. New York police clamped down on their first generation of crack dealers, imprisoned masses of them, but gun crime soared: the second generation were younger, more impulsive and they had seen what had happened to their older brothers. People carry credit cards instead of cash; mugging falls. They get mobile phones; mugging rises. The police chase around behind them. Governments hire more police officers to cut crime; the extra officers discover more offences; so recorded crime rises. Crime has risen almost constantly for many decades, and yet the criminal justice system now delivers fewer detections and fewer convictions than it did only 15 years ago.

An hour after the briefing, the inspector in the van prowls St Paul's, his ear-piece crackling with updates from the men on the women on the street. This is the police at maximum force - manpower, fire power, intelligence back-up. Who's winning? At the moment, it's quiet on the streets - on Foster Street where a man had his car stolen from him at gun point, on Denbigh St where they stabbed a man in the street before they robbed him, outside the St Nicholas pub where somebody absent-mindedly left a fistful of shotgun cartridges on the bar. The inspector cruises past the Black and White Cafe and automatically glances up at the roof of the council flats across the road. The Aggis were up there the other day, firing giant fireworks like mortar rounds at the cafe door, to oust the Yardies. They've fired them direct at his van too, from close range.

Tonight is just the latest crisis. This has been building up for two years. The night they murdered Chrissie Hewitt outside the Jolly Roger was more or less the beginning. That was in June 2001 and the police knew then that something strange and terrible was happening in St Paul's. There was crime bubbling out of the ground like swamp water. Some of it was hideous - vicious assaults and kidnappings - and, when they looked back at the end of the year, they found their 'major and serious' crime had shot up by 72%. And the ordinary everyday crime had shot up with it - robberies nearly doubled that year. People breaking into cars, breaking into houses, snatching bags in the street.

That summer, the St Paul's Carnival turned into a shooting match. The Burga Crew came down from Birmingham; one of the Bristol Yardies shot one of them with a nail gun; the Burga Crew pulled out their guns; the Bristol guys got theirs; somebody had a machete. That night, there were at least five separate gun attacks. Nobody even really knew what it was all about - might have been a squabble about a drugs deal, might have been a ripple from some turf war over in Kingston, where the Yardie gangs are locked into alliances with local politicians, controlling whole neighbourhoods with corruption and fear.

The police were ahead of the game. As early as February 2001, they had spotted the Yardies coming in and set up a special team to gather intelligence. They found there were Yardies crawling all over St Paul's, maybe 200 of them, some of them well-known in East Kingston as killers. Within a week of that first Yardie murder, they had pulled in 19 more officers to tackle the open drugs market which was now booming around Grosvenor Road and to target the Yardies who were behind it - who had been giving away crack cocaine and selling 'snowballs' of heroin and crack combined, expanding their client base. On Grosvenor Road, the dealers had started wearing bullet-proof vests.

That autumn, 2001, the police were all over them. They found a local guy was running a phony college on Lower Ashley Road, selling enrolment to non-existent courses so that gangsters and their smugglers could slide through immigration controls - 300 of them in less than a year. They set up a new unit to track their money, busted a greengrocer and a travel agent for money-laundering and followed streams of cash from Bristol back to Kingston - just under £10 million in their first year, some of it buying property in Jamaica, some of it funding new crack consignments to the UK. And they arrested hundreds of street dealers, something like 700 of them in a year.

By now they had set up a special unit, Operation Atrium, which pulled in officers from all over Avon and Somerset. All the divisional commanders lost men and women. They pleaded for replacements and eventually the local police authority agreed to backfill the lost manpower. It would take 18 months to fund, hire and train the new officers. The street dealers they were arresting were being backfilled in less than 24 hours.

This was a police force recognising a threat to one of its communities and defending it with all of the weapons of the criminal justice system, and... it made no difference. It was like trying to walk on water. Each day began with dozens of reported crimes; each day ended with most of them undetected. The police would arrest people; the courts would give them bail. They would deport people; the airports would let them straight back in again. One of the Yardies they arrested for the murder outside the Jolly Roger was a guy known as Mr C. He was here illegally, so they deported him. He came back; they caught him a few months later and deported him again. In October 2002, they found him a third time and deported him yet again. This time, he was slower to come back: he got shot by another gangster in Kingston. And all the time the Yardies were dealing, bringing in more users, who committed more crime.

Even when the courts did lock someone up, it was nothing more than a breathing space. The ones who were worth locking up - the 'prolific offenders' - came straight back out and carried on offending. Down the road, for example, the city centre car parks have just seen a surge in break-ins on cars, simply because one local lad has been released from prison and gone straight back to the life he knows best. (There's another lad down there who's also a specialist in car crime, but he lives on the job: he sleeps in a multi-storey car park, wakes up, robs a few cars, goes out for the day and drifts back to base when he's tired or needs some more cash. He's been arrested more times than they can count, only to pop out of one of the multiple exits in the system and carry on thieving.)

The inspector turns out of St Paul's and stops in a street full of shops on the edge of the City Centre. He wants to check the cells in the Bridewell. At the desk, he ticks charts and signs forms. Down in the cells, he tells the girl who's been done for shoplifting that he'll get her a light for her cigarette, tells the man who is sitting in total darkness in the next cell that he'll be seen by the doctor very soon. This is the 3% in real life, the offenders who have finally been brought into the system. This is where you begin to see that the machine whose neck is so narrow that it misses 97% of the offences it is aiming at, then lacks the equipment to change the behaviour of most of those it does manage to catch.

Sometimes, it makes no difference simply because it is too weak to enforce its will. Most of the offenders who are convicted will end up in magistrates court where the bench may solemnly fine them. And 41% of those fines will never be collected. We know that, because the Lord Chancellor's department, which is responsible for the courts, checked the numbers. They reckon it would take four years just to collect the backlog even if there were not something like 320,000 new fines being doled out each year. In the financial year to April 2001, the courts wrote off £74 million of fines which they had failed to collect. The offenders simply shrugged and walked away.

It's the same with community sentences. The courts handed out 169,000 of them last year (and they are much fiercer than they used to be), but tens of thousands of offenders simply don't turn up to be punished. With the most severe - the Community Punishment and Rehabilitation Order - the Home Office's records show that 47% of offenders who were supposed to serve community punishments simply didn't bother. Of course, their breach was eventually reported to the court which duly issued a warrant for their arrest, but we know from the Audit Commission that 56% of those breach warrants were never executed. They just piled up on a shelf somewhere.

And then there's prison. At least with prison, the sentences are usually enforced; very few inmates now escape. But here is the radical contemporary critique of its impact: "Prison sentences are not succeeding in turning the majority of offenders away from crime. Of those prisoners released in 1997, 58% were convicted of another crime within two years... The system struggles particularly to reform younger offenders. Eighteen-to-twenty-year-old male prisoners were reconvicted at a rate of 72% over the same period.... At a conservative estimate, released prisoners are responsible for at least one million crimes per year... In terms of the cost to the criminal justice system of dealing with the consequences of crime, recorded crime committed by ex-prisoners comes to at least £11 billion a year." That is the verdict not of some maverick commentator but of the Prime Minister's Social Exclusion Unit.

The inspector ticks the last form in the Bridewell and drives back to St Paul's. And we drift through the dark together, the inspector and the reporter, surrounded not so much by the life of crime as by the near-death of the criminal justice system. In the dark shadows of Brunswick Square, a pimp sprawls across the front seat of a Mercedes, smoking and eye-ballling the inspector. Not five yards from him, there is a bright yellow sign, posted by the inspector, warning punters that their car numbers may be recorded by CCTV if they cruise the square. The pimp's girls are working just around the corner, standing there as thin as string and using the old graveyard behind the unitarian meeting house to deal with their customers. The inspector could arrest dozens of working girls every night: he'd tie up multiple hours of manpower to send them to court where they'd just be fined, and then they'd be back again, working even longer hours to pay the magistrates as well as the pimp.

The failure of the system is mirrored by the complacency of many of those who talk about it. The government will tell you it is hitting its key targets, right-wing pundits still cheer the idea that prison works, liberals twist themselves around statistics like beans running up a pole, to prove that all is well. The truth is that the evidence that crime is falling is about as solid as mist (see sidebar). Even if, behind the veil of numbers, the truth is that nationally crime is coming down, communities like St Paul's - always the most impoverished - are nevertheless pockmarked with streets which tell a different story.

There's a family who live not far from here with five adolescent boys. All of them are thieves. They all steal almost every day. One of them was at it on Christmas Day. Another brother recently was arrested and released on bail - he was caught at it again within an hour. There's another lad whose routine is to go out screwing houses each night - at least three of them. Every night. And they steal from here. They don't go off into the middle class areas - that's foreign territory, they'd stand out on the street, and they don't know the houses or the routines of the people. Thieves steal from their neighbours.

Those who raise their claret to the success of law enforcement should talk to the primary school teacher here who has to tell her ten-year-olds that it is not cool to carry guns, that the flash lads who parade on the pavement with the rolls of cash and the chunky gold jewellery are not role-models; to the parents of those children, trying to get to school in the morning without stepping on bloody syringes; to the man who says he can't reach the corner shop to buy a paper without being hassled by some toerag with a hood over his face; to the old woman clutching her handbag like a baby as she takes her pension home from the post office. The council street sweeper here the other day was stopped in his work by a guy who jammed a gun up against his head: it turned out that the dealers had been stashing rocks of crack cocaine in old milk cartons on the pavement, so that they could sell their drugs without actually holding them, and the sweeper was demolishing their business.

Most of all they should talk to the police, who really know who's winning out there. Those on the right might learn that a lot of these officers have zero faith in their zero tolerance gospel. The left might learn that something important has changed. Twenty years ago, this country's police were led by a generation of chief officers who had left school at 15 and learned their lessons in life from the army. Frequently, they were crude and unsophisticated. A lot of them didn't like hippies or lefties or queers. Some of them were racists. But a great deal has changed. Partly, this is about one of the great success stories of state education. In 1976, the Edmund Davis report gave the police better pay and far better pensions than anybody else in the public sector, and so - slowly at first - the police service began to attract working class men and women who were the cream of comprehensive education. They joined a force which was then battered by a series of traumatic scandals - ruthless corruption in London and Birmingham, the miscarriages of justice around IRA bombing trials, urban riots spawned by clumsy and sometime vicious street policing, the Yorkshire Ripper inquiry, Stephen Lawrence. They had to change.

Of course, there are still police officers who abuse their power, who are idle and incompetent and dripping with prejudice, but far more than ever before, there are officers who are neither the lapdogs of the right nor the demons of the left. And they are losing. It is not for want of trying. They work in a system that bristles with power, that is rich in individual effort and (relatively) rich in resources. Every day, they win battles and yet overwhelmingly they are still defeated in the war.

That's not because they are all idle or corrupt or racist. It's not even primarily because of the practical difficulties of their work - that most victims (55% of them) do not even tell the police that they have suffered a crime; that it is simply very difficult to detect an offender whose crime was not witnessed; that more witnesses now are intimidated and unwilling to come to court; that courts which demand proof beyond reasonable doubt necessarily release some of the guilty. The real problem is that these officers are running a system which has a design fault on the scale of the Maginot Line: it confronts a problem of infinite complexity with a response of gross simplicity. It approaches all crime with the same rigid routines - patrolling, detecting, imprisoning. They were born in the distant past; they are strikingly ineffective (see sidebar). Those antique rituals are themselves based on one rigid underlying idea which is equally weak: that all criminals will behave in the same rational and predictable fashion; that they will all be deflected by the prospect of arrest and the threat of punishment.

Conventional law enforcement does not fail completely. Its great achievement is that it regulates the behaviour of law-abiding people, who do pick up the signals it sends and react to them. Its great failure is in dealing with criminals. Think about those Yardies, about the five brothers burgling each day away. Conventional law enforcement rarely catches them and scarcely deters them. It controls them to the precise and limited extent that it can lock them up and incapacitate them - about a tenth of the 3%. But that's more or less it.

The government relentlessly ratchets up public expectation for law enforcement. And yet: "Despite all the changes of recent years, the signals sent out are ones of weakness in critical areas: too few criminals brought to justice; too many defendants who offend on bail; too slow to bring them to trial; too many guilty go unconvicted; too many without the sentence they and society need..... Far too many offenders escape justice, creating the ‘justice gap' between the number of crimes recorded by the police and the number where an offender is brought to justice." That is from the same government's preamble to its most recent criminal justice white paper. They put more money into more officers following the same rigid rituals. And yet: "There is very little correlation between levels or changes in funding and crime rates, or between changes in police numbers and crime rates." That is from the Audit Commission.

We spoke to a civil servant who has spent most of his working life in the criminal justice sytem. He put it this way: "The point is that the old Benthamite theory of a rational system producing results has collapsed. We are left with a different set of objectives which are really to do with pleasing people - the electorate, the civil servants, the prison officers.... The public and the press sing a contant mantra about putting more police back on the streets. It doesn't work. Same with long prison sentences in unpleasant conditions - they don't work. But they do work as ways of making people happy. So that's what we do." And we spoke to people at one level or another of every law-enforcement agency in the country and, over and over again, they came up with a version of the same wry line: "The criminal justice system doesn't catch criminals, it doesn't dispense justice, and it's certainly not a system."

Any government which confronts this reality is faced with a fork in the road: go down one route and increase the power and funding of the existing system in the hope that it may do a little better and in the knowledge that every new power for the police is a potential threat to the public co-operation on which they rely; or take a new route, recognising that a complex problem requires subtle solutions, and break out of the boundaries of conventional law enforcement in search of intelligent alternatives. Since April 1997, generally without public recognition, this government has done both.

It has made far more noise about its journey on law enforcement. It has taken the criminal justice system and shaken it like a Victorian nanny used to shake an insolent child. It has injected the whole system with an almost dizzyingly energetic programme of reform and used criminal justice bills for the hazardous purpose of giving it new muscle. But, behind the almost ceaseless flow of law-enforcement rhetoric, they have also looked outside the system. They have taken the advice of some of the most radical criminologists in the country and embarked simultaneously on a completely new approach, trying to link up all the agencies of social welfare to draw the criminal poison out of impoverished communities. The effect of this twin strategy has been electrifying. The great question is whether, as a result, the police have started winning.

In a small way, the inspector won tonight. The streets of St Paul's are quiet, and they stay that way - until two in the morning, when he has to pull back his officers, and the dealers and the gangsters will come out to work. They are working even now, around the corner on the pavements in St Jude's, and in their cars using mobile phones to find their buyers. The drug-buyers who have been robbing and burgling in St Paul's to find the cash for the dealers, have moved up the hill temporarily to St Lawrence's where they have been mugging the prostitutes and unleashing a mini epidemic of burglary.

Success is hard to find. This wave of crime, which has now become a shooting war, was almost certainly triggered by what looked like a law-enforcement victory: it was only because the police, through diligence and skill, arrested the Aggis a few years ago, that the drugs market in St Paul's had no owner, and the power vacuum sucked in the Yardies with their crack and their guns. The best available tactic for stopping the war now - quicker than arrest and prosecution - is for the police to link up with probation, using minor offences and intelligence to revoke the parole licences of the leading Aggis. That will put them back behind bars. For a little while.

The inspector has stopped the van for a moment in a dark side street. The radio plug in his left ear is babbling with feedback about Yardies and Aggis. There's another radio hooked into his right ear feeding him with reports of ordinary life on his sector. Somebody can't find the charging unit to handle an arrest. There's a 15-year-old girl wandering through the middle of the operation. The mental hospital has lost a patient, and they think she may be headed for the river. There's a battered woman and child at Trinity Road police station, looking for a refuge. The inspector needs his mobile to get the helicopter to look for the hospital woman. Another mobile in his pocket starts to ring; he's eating a sandwich; he can't read his notes; he reaches for the light, accidentally hits the siren and for a moment or two, we just sit there, wailing in the darkness.

Who's winning?

Additional research by Tamsen Courtenay

The futility of punishment

The Guardian, April 2004

There is order in the court. There is chaos on the streets. And they meet in the main hall of Thames magistrates court in the East End of London. It is busting with people - this guy made of muscle yelling at his tiny female lawyer "This is MY case, this is MY life"; the elegant Somali man with the beautiful black suit cruising quite lost through the crowd without a word of English to find his way; the young Bengali lad who just blew a spliff in the toilets; the prosecutor reading "God Knows" by Joseph Heller; the little knot of regular defence lawyers, Charlie and Teresa and Denis and Keith, swapping the gossip and wondering how long it will be before somebody tells them the new security code for the door to their room so they can finally start work for the day.

The courtrooms open. Here come the overnights - the people who've been arrested by Hackney police and held in the cells for this morning's hearing.

Case One: male, white, aged 26, charged with theft of eight packets of bacon from Londis supermarket - tested postive for heroin and cocaine. Case Two: male, Bengali, aged 19, charged with theft of £59.63 of food from Tesco - tested positive for heroin and cocaine. Case Three: male, white, aged 32, charged with harrassing his mother and stepfather by kicking down their door, breaking their window, threatening them in persistent search for money - tested positive for heroin and cocaine.

The cases blur. Amost all male, almost all young, almost all of them trailing a string of other cases that have just been dealt with or are about to be dealt with, almost all of them already serving one or more community punishments, many of them with unpaid fines. And over and over again, the magistrates' mantra, "tested positive for heroin and cocaine". (Finally, an overnight defendant appeared who had not tested positive for drugs. He was an alcoholic.) Most of them plead guilty and are adjourned for reports.

This is the story of a week in court - the story of what happens when the criminal justice system finally deals with the mere 3% of offences which it manages to capture. The courtroom is the crossroads for all the players in the system. Here is where the police send those they have arrested. Here are the prosecutors with the evidence they have collected. Here are the probation officers and the prison escort officers waiting to dispose of the guilty. Here are the offenders "brought to justice". And here is the question: what does it achieve?

It is a question of acute importance in a system which now more than ever relies on court-sanctioned punishment as its mechanism for controlling crime. As this series has already described, the Home Office has quietly compromised its innovative crime-reduction programme, including its support for problem-solving policing; and the National Treatment Agency has taken the promising initiative to give treatment to drug users and smothered it in misconception and mismanagement. In the background, the probation service, which was once the world leader in tackling the causes of crime, has been sliding into a state of demoralised chaos - restructured, mismanaged, starved of funds, overburdened and now on the verge of being restructured yet again. With its best efforts at rehabilitation struggling, this government now presides over a system which is overwhelmingly devoted to punishment, handing out more jail sentences and more community punishments than ever before in the history of this country. What does it achieve?

Look at this man here, standing in the dock, fiddling with his fingers. There is something Victorian about him. It might be the respectable best suit he is wearing - white shirt, black tie and a black jacket that's so big he must have borrowed it. It might be his supplicant posture, his hands neatly folded in front of him, his head hung down on his chest. It might just be the sheer frailty of the man. He is as thin as a whippet, his face is as pale as paper, and he is quivering slightly as the district judge and the clerk of the court assemble the facts about him.

At first, the case of Mathew King sounds simple enough. On January 17 last year, in this same court, he admitted driving whilst disqualified and trying to syphon petrol out of somebody else's car. He tested positive for heroin and cocaine. He was given a 12-month community rehabilitation order which meant he had to keep regular appointments with a probation officer, but in September, he failed to turn up twice, and now he is charged with breaching the order. He admits it, and the judge adjourns the case for three weeks so that probation can prepare a pre-sentence report on him.

Mathew King turns and slips quickly out of the court and, just as the door swings shut behind him, there is a fleeting glimpse of his life - a small woman with long blond hair and a face like a snapshot from another world. It is slightly frightening and immediately haunting and somehow familiar, the way her skin is stretched so tight across her cheeks that it has made her eyes grow large: this woman has the face of a starving child. Again, there is something Victorian here. And she is pulling at Mathew's black-jacket sleeve - no time to lose. Why the urgency? Where are they going with their pale, skinny bodies? What goes on in the private world of Mathew King?

Tracing him was easy enough: he had given his address at the start of the hearing. It turned out to be a tower block in the backwash of a motorway in the heart of what was once the teeming bustle of the London docks. The lift was busted. The stone stairs smelled of piss and garbage. On the fifth floor, just outside his flat, somebody had smeared something brown and nasty on the wall. And when the door finally cracked open on the security chain, there was Mathew and his world, shaped by chaos.

He was just back from Bethnal Green Road. He had gone there with the woman with the hungry face, who turned out to be his girlfriend, Lisa, and with John, who turned out to be the father of Lisa's two oldest children. The three of them live here together and every day they go to Bethnal Green Road to buy their gear. Everybody does. But today it went wrong. John gave £70 to this black lad on a bike. The black lad's mate kept Mathew and Lisa talking round the corner. The black lad rode off and just didn't come back, but Mathew knows him from a year ago and he's got a mate of his coming round - he's well over six foot and built out of bricks - and they're going to go back down there and sort him out. But just now they've got no gear.

Lisa sweeps the carpet. She is 29, although she looks nearly twice that. John fiddles with his fingers, which are mangled and maimed by bad injections. Mathew says he's had a letter from probation about preparing his pre-sentence report, and John gets it for him and - just look at this - it says "Please come and see me at this office. See address below." But there is no address below! And no phone number. So Mathew's going to miss another appointment. At least this letter reached him. The last lot got sent to his mum's house, and he can't go round there because he doesn't get on with his step dad. He's had all this with probation before. The last time he went, he said "What time have I got to be here?" and they're "You haven't got to be here." Then he tells the story of how he missed the appointments in September.

It's all about him and Lisa and the terrible business with the baby. But then again, it's not - it's all about their whole life. Mathew is 29, the oldest of two brothers, born around the corner in Mile End. His father was a drunk, used to smash the house up, and, when Mathew was five, he left. His mum gave him a chance to come back, but he just turned up and stole the stereo. Soon afterwards, his dad's mate came out of prison and came round to see him and found he wasn't there, so he moved in with Mathew's mum, and he's still there. His dad is still in the East End somewhere, still drinking. As far as Mathew is concerned, that man doesn't exist.

He went to Hackney Down school and he was no good at it, he still can't read or write except for his name, and the teachers picked on him, so he swore at them, and they sent him home, so he carried on swearing and didn't do no more school. The first time he was nicked was for breaking into a car when he was 13; he got a caution. He is good at breaking into cars. His crime was just silly stuff, though, until he started using gear when he was about 18. Then he got into nicking cars - he's sold a lot of taxis that way - and he was doing burglaries too (shops, not houses).

Lisa and John both came up the same broken ladder. Lisa's dad left, and she doesn't get on with her stepdad. John's dad died of cancer when he was young and then he got teased for wearing crap clothes because they were poor. Both of them were no good at school. John beat up a teacher and left early. Lisa ran away from home and by the time she was 12, she was selling herself down Aldgate to businessmen on their way home from the City, then she ended up in foster homes and secure units. She was 15 when she met John and got pregnant with their first child, a boy. The social workers didn't like the look of them, but John thought it would be OK: he was 20 then, working as a site supervisor down at Canary Wharf, and they had a room at a bed and breakfast on the Romford Road. But the social workers took the baby away, so that same day Lisa went and found her mate Nicola (she's dead now) and she smoked £50 of heroin in one afternoon. She'd never done it before. She's been doing it ever since. John was soon at it too - he just wanted to know where all his money was going.

Mathew didn't meet them until five years later. By that time, Lisa and John had just had a second son. Lisa was still selling herself, and John was on three kinds of medication for a paranoid obsessive disorder. He was committing crimes every day of the week to fund their heroin habit, so the social workers took this little boy away as well, although in the end Lisa's mum went to court and got custody of both Lisa's kids. Lisa and John sort of drifted apart. Mathew was arguing with his step dad and so he moved out to live with Lisa, and John stayed around.

Now, the three of them spend their days ducking and diving and trying to find gear. Most days, they'll spend £100 down on Bethnal Green Road for the three of them. They can get some of that in benefits, but most of it, they have to hustle, so they have to break the law. Most of the time, nobody knows about it. Then, once in a while, they get caught. Mathew has only been done half a dozen times. The worst one was when he and a mate broke into a car in the West End without noticing there was a police car parked right there in the road, so the cops chased them, and Mathew started driving like a maniac, going through red lights, heading the wrong way up a one-way street until finally the police cornered him on the Embankment. He got 30 days in Wandsworth for that and a driving ban, so he lost his job driving for a haulage firm (Mind you, he never had a real licence in the first place.)

Lisa and John also get away with most of what they do, although Lisa's criminal record fills six pages. She's had fines, community orders, five or six jail sentences. John's been done for theft, burglary, fraud, bits of violence including battering a cop, driving whilst disqualified. He's done 19 prison sentences. Nothing changes. It might have been different if there was real treatment around, if they could get a decent prescription for methadone or diamorphine, instead of the rations and rudeness they get from NHS clincis. Most of the time, no court even tried to help them. So, they still need £100 a day - and, by the time you allow for the prices the fences will pay, that means laying their hands on more like £500 of property. John says it's simple: going without gear is not an option; you do what you have to do; the law doesn't come into it.

It all bent out of shape last summer. Lisa was pregnant. She and Mathew really wanted the baby, but they were afraid the social workers would move in again, so they kept it a bit quiet. Then one day in June, Lisa fainted in the street - she can't afford money for food so she'll often go two or three days at a time without eating - and the hospital told the midwife, and the midwife told the social workers, and so Lisa and Mathew cut a deal with them: they would give up gear, they would sign on at the local NHS drugs clinic for some methadone, and so the social workers would let them keep the baby. They had to wait two weeks to get the methadone. When they got it, they were only allowed 40 mill a day, which wasn't enough. And they were refused injectible methadone, which was bad for Lisa, who is a fixer. But they stuck to the deal more or less. For the sake of their baby. Because they already loved the baby.

He was born in August. They named him after Mathew. They were proud and happy, and the baby was healthy - he wasn't addicted to anything - and they spent just about all day every day in the hospital, like real parents. And after two weeks, the social workers took the baby away from them. It did their heads in. Lisa says it really did Mathew's head in. He thought they had a deal. He just couldn't see the fucking point of doing anything any more. Then, Lisa and John got into a row with the staff at the drugs clinic, and they got banned - except that John was wearing Mathew's hat, so they banned Mathew instead of him. Mathew wasn't that bothered. They just went back down Bethnal Green Road.

And that's how Mathew ended up back in court. Not that they got nicked down there, just that Mathew couldn't see the point of keeping his probation appointments, so he ended up with a letter telling him to turn up at Thames magistrates court. That's a hassle. He's got to find this probation officer with the secret address. But before he does that, he's got to go back down Bethnal Green Road and get their £70 back off the little toe rag that rode off with it this morning.

And then what will this court hearing achieve? At what point in a typical day do Mathew and Lisa and John stop and quietly consider the impact of their behaviour on anybody else or on themselves? At what point do they consider anything at all beyond finding some way to get hold of £100 a day? Born and raised in chaos, they have been punished before by the system, often for good reason, but never changing their behaviour, occasionally becoming much worse. Surviving now in chaos, at what point does the threat of punishment stop them committing a crime?

There are signs of the government changing its track. Following January's Carter report, it is setting up the new National Offender Management System, which is supposed to cut the rise in the prison population. It wants to use new day fines - geared to the income of offenders - and also its Sentencing Guidelines Council to encourage courts to use more fines for petty offenders. The Crown Prosecution Service has plans to extend the use of cautions by adding conditions to them. But these moves are designed to take the pressure off the prisons and the courts. None of them challenges the underlying assumption that punishment is the primary tool of crime reduction.

If they had been born 40 years earlier, Mathew and Lisa and John would have been allowed to get clean drugs from their GP. John would not have lost several fingers and half a lung from blood clots. Lisa would not be suffering from malnutrition in the heart of one of the richest cities in the world. None of them would be involved in stealing something like £500 of property a day. As it is, they can turn to the NHS which will offer them long delays, lots of paperwork, tight rations and lectures about abstinence, or they can come to court to be punished.

The courthouse is busy again when Mathew and Lisa turn up there a week or so later. An ulcerated old man whose boot laces are tied up round the ankles of his trousers, totters around the hall looking for a lawyer. A fog of nicotine rolls out along the ceiling from the smoking area. There are two televisions mounted on the walls: the studio audience are clapping, the caption says "Leave my lover alone". Outside court number three, a baby in a buggy is crying its lungs out.

Mathew has got his act together and found the probation officer who needed to see him, even without an address. And the pre-sentence report is OK. It explains that Mathew missed the two appointments because he was so upset about losing the baby and it recommends another community sentence. Mathew's lawyer is good at his job. Lisa and John sit in the public gallery, while Mathew takes his place in the dock.

The district judge has a dark suit, gold-framed glasses on the bridge of her nose and pearl ear-rings. For two or three minutes, she reads the report from probation. The court is quiet. "Yes," she says finally, in a voice just like the Queen's.

Mathew's lawyer starts: "I think it's worth noting - "

But the judge cuts across him: "He was fortunate, wasn't he, to be given a probation order in the first place?" She seems irritated. The lawyer continues, reciting all the appointments which Mathew did keep before he missed just two. She sits with her eyes shut and then cuts in again: "He was given the opportunity but he doesn't seem to have taken it." The lawyer quotes from the probation report and explains that Mathew has had his baby taken from him. "Because he is a heroin addict," says the judge as though that explained everything. The lawyer keeps making his points, the judge keeps interrupting, Mathew is looking paler than ever. Then suddenly, the judge has heard enough.

"Mr King, will you stand please? You were given the opportunity of doing a community penalty and you chose to lose contact with -"

Now Mathew interupts: "I been in contact with them. I give them my new address. I give them my mobile number. I been down there before this court business started."

"You have had your opportunity. Now you will go to prison." Mathew stares blank at her, as she hits him with it: "Three months on each count."

He turns towards the public gallery, breathless. Lisa is on her feet. She knows in an instant that the prison is not the problem - it's having no gear that is going to hurt him. And she knows he hasn't even got a pack of cigarettes in his pocket to help him get through it: he was never expecting to go to jail. Two security men are closing in on the dock. Lisa leans out of the public gallery, her cigarette pack in her hand, Mathew reaches out to grab them, the security men close in, Mathew stretches quickly, Lisa strains towards him, can't reach, the cigarettes fall on the floor. Lisa goes berserk, barges into the court, shouts that the judge is a scumbag, but the judge has gone. And so has Mathew. Mathew has been brought to justice.

Minutes later, Lisa and John slip away into the streets. They are in a hurry, just like they were when Lisa tugged Mathew away from court a few weeks ago. They have to get down to Bethnal Green Road. On the way, they have to find £100 from somewhere. Justice has been done today. Chaos has encountered order. And what did we achieve?

* At his request, Mathew King's name has been changed.

See below for background statistics

How the courts increased their sentences:
During 2002, all courts gave immediate prison sentences to 111,600 offenders - a record and 53,500 more than in 1992.
During 2002, all courts gave community sentences to 186,500 offenders - a record and 84,100 more than in 1992.

How the magistrates made community sentences more punishing:
In 1992, rehabilitation orders (then known as probation orders) accounted for 41.5% of all community sentences. In 2002, they accounted for only 31% of community sentences.
In 2002, rehabilitation orders accounted for only 2.2% of all magistrates sentences.

How magistrates moved from fines to prisons:
In 1992, magistrates fined 1,074,800 offenders. In 2002, they fined only 894,300.
In 1992, magistrates jailed 10,300 offenders. In 2002, they jailed 26,500.

Outcomes in London magistrates courts:
81.7% of defendants plead guilty;
8.9% are found guilty in their absence;
7% are convicted after summary trial;
2.1% are acquitted after trial;
0.3% are dismissed with no case to answer.

Note: A community rehabilitation order (formerly known as a probation order) requires the offender to stay in regular contact with a probation officer for up to three years. In some cases, the court may also require attendance for treatment or other courses.
A community punishment order (formerly known as a community service order) requires the offender to perform up to 240 hours of unpaid work.

Madness behind bars

The Guardian, December 2004

On the evening of Thursday August 19 this year, a prisoner was locked into his cell in the segregation unit of Wormwood Scrubs prison in west London. This man had been sent to the seg to be held in solitary confinement as a punishment because he had threatened a cellmate. But that was three weeks earlier and, since then, he had settled down and been quite easy to manage. Everything was normal.

On the following morning, August 20, an officer called Dickie Hampson unlocked the door of the the cell, and, without warning, the prisoner pounced on him and stabbed him in the back of the shoulder with a toothbrush which he had sharpened into a rigid blade. Hampson was rushed to hospital where doctors found that the epaulette on his shirt had saved him from the worst of the wound.

Nobody could understand why this prisoner had suddenly turned violent. Immediately after the attack, a group of officers with a riot shield restrained him, handcuffed him and removed him to the 'safe cell', designed to prevent self-harm, while they searched his possessions to make sure he had no other weapon concealed in there. That afternoon, the prisoner was calm again, remorseful and worried about Hampson's condition, and soon he was taken back to his normal cell.

On the next morning, Saturday August 21, a second officer, John Leadley, unlocked the same prisoner to take him to the governor for an adjudication for the assault on Dickie Hampson. As Leadley approached him, the prisoner slipped another sharpened toothbrush out of his shirt sleeve - nobody ever found out where he had got it from - and clawed at Leadley's face: the spike cut down through his eye brow and into his cheek bone, missing his eye ball by less than an inch.

With blood flowing from his face, Leadley too was taken to hospital. The prisoner was moved again to the safe cell. It was several days before the prisoner spoke to the senior officer on the seg, whom he trusted, and explained what had happened: soon after he was locked into his cell on that first night, he said, a black cat had slipped through his window and sat on the bed next to him. This cat had been following him for some time. Now, it handed him a card, the ace of spades, and then dropped down to the floor where it danced for him. There was music. The dance went on for hours, the prisoner watched, and then the cat turned to him to warn him that the next person who came into the cell would be his enemy: the prisoner must kill him. The cat had promised to stay with him to make sure he was all right. The prisoner had started sharpening the toothbrubush.

This man was psychotic, he had already been diagnosed as paranoid schizophrenic. None of the officers who work at Wormwood Scrubs is trained in psychiatry. Yet routinely, they deal with mentally-disordered men. When John Leadley was stabbed in the face, it was the fifth time he had been attacked in three months; every one of his attackers was suffering from a severe mental illness.

There is nothing unusual about Wormwood Scrubs. Every prison in the country now warehouses the mentally disordered: the numbers have been spiralling upwards since the closure of the old asylums. This has reached a point which beggars the imagination: figures from the Office of National Statistics show that, if we diverted to treatment all those prisoners who are mentally disordered and/or addicted to alcohol or drugs, 90% of inmates would no longer be held in jail.

There are now some 75,000 men and women behind bars in this country. The findings of the Office of National Statistics suggest that nearly 50,200 of them have personality disorders; 6,175 are psychotic; and more than 35,000 of them have neurotic disorders. Several tens of thousands of them suffer a combination of disorders. More than 75% of them are intellectually impaired, with IQs below the national average. And these are not figures that the government denies. The prisons minister, Paul Goggins, himself describes these ONS statistics as "our principal source of knowledge about the extent of mental ill health in the prison population." Most mentally disordered people do not end up in prison; but overwhelmingly most prisoners are mentally disordered.

And every one of our prisons suffers from the 'treatment gap' - the gulf between the care that is needed for the mass of mentally disordered men or women within its walls and the care that is actually provided. After years of neglect, when there was no effective mental health care in prisons, the government is finally tackling the crisis, but resources are short, there are real limits to what can be achieved in a prison regime and, worst of all, as the health minister, Dr Stephen Ladyman, told the parliamentary mental health group this year: "It is generally accepted that mental health will deteriorate in prison."

The results are deeply worrying. It is not simply that mentally-disordered prisoners may be violent to staff or other inmates. Indeed, they are far more likely to be violent to themselves. On average: every single week in prisons in England and Wales, two prisoners take their own lives and two or three others are resuscitated after trying; and every day, some forty prisoners cut or gouge themselves in their cells. Beyond this, the fact is that we are herding disordered men, women and children into our prisons, storing them there for months and years, and then processing them back into the community with every prospect that their disorder has not been addressed and so they will offend again. They suffer. Those around them in prison suffer. Everybody suffers.

Every afternoon at Wormwood Scrubs, the white prison vans queue up outside the gate, ferrying men back from the courts - the young and surly, the old and confused, the crackhead who was caught shoplifting yet again, the homeless man who set fire to an empty building, the man who thumped a stranger because he looked at him in a funny way, the old Rasta who screams at himself in the park. The reception officers process them and give them a bedroll, a nurse takes them through a health questionnaire: "Have you ever received treatment from a psychiatrist outside prison? Have you ever received medication for any mental health problems? Have you tried to harm yourself?"

Once, several tens of thousands of mentally-disordered men and women were held behind the high walls of the old asylums, but they failed and, in the late 1980s, they were all closed down. Now, those who would have been their patients rely on a network of community care which is notoriously overstretched and, if they fall through its gaps, they are scooped up by police and sent back behind high walls. Those walls protect a crooked structure.

We now have139 prisons. We are already committed to building five more. By the end of the decade, the Home Office plans to incarcerate at leaste 80,000 men and women. If the Office of National Statistics is right, some 56,000 of them will suffer from at least two types of mental disorder. Yet we have only just over 4,000 secure psychiatric beds, which are already full, and no plans to increase them. The whole structure is crooked, because it is built on a crooked foundation - the emotionally satisfying but deeply ineffective idea that if we hurt these people enough, they will obey the law. Hospitals are built with bricks of care, prisons with the plans of politicians.

The old Rasta who screams in the park was arrested for trying to steal a handbag and for exposing himself in a public place. He is soon processed and taken down to A wing. Nobody realises he has lied about his psychiatric history, nor that he has been arrested, charged and now remanded in custody under a false name. Lots of new prisoners conceal their psychiatric history: they don't want the stigma, they are afraid it will count against them, they have forgotten. On the wing, the old Rasta is soon marking himself out as an odd ball, muttering to himself, eating just about nothing, refusing to wash himself or to clean his cell. It gets so bad that one day the officers force him to take a shower: nobody can stand the smell any more.

By chance, a visiting nurse recognises him from the hospital where she works and remembers his real name. The prison doctors contact the hospital's community mental health team for his notes and persuade the old man to take some medication while they wait for them to arrive. Three weeks later, the notes finally reach Wormwood Scrubs, detailing his long history of schizophrenia, but by that time, he has been taken back to court where he is given bail or a non-custodial sentence - nobody bothers to tell the prison - and he is back out on the streets again.

The number of mentally-disordered men and women who are held in our prisons has increased seven-fold since the old asylums were closed. We have talked to staff at every level of the prison service who are alarmed and depressed to find themselves warehousing the sick. The former director general, Martin Narey, now in charge of probation as well as prisons, has publicly described the strain as 'overwhelming'. A senior manager told us it was 'a bloody awful problem'.

The cutting edge of the government's response has been to create 'inreach teams', to deliver care-in-the-community on prison wings. In Wormwood Scrubs, the team consists of one consultant psychiatrist, one social worker and one community psychiatric nurse. They care for a prison holding some 1,167 men at any one time, with some 3,900 coming and going over an average year - 90% of whom they reckon to be mentally disordered. The simple reality is that the Scrubs team spend so much time assessing new patients that they rarely have time to deliver a care plan.

There is also a new Day Care Centre which borrows money from the education budget to run courses in relaxation, art and acupuncture, but staff are so short that sessions sometimes have to be cancelled. An unpaid counsellor comes in once a week, but there is no psychotherapy at all. There are signs that the centre has helped some of the low-end patients, but the service is limited and helpless to deal with the most severely ill who need beds in outside hospitals, which are extremely hard to find. In a special report on nursing in prisons, the Department of Health itself acknowledged that, while staff may do their best, there is a level of care which "prison health care does not and can not provide".

Even at the lowest end of the scale of mental disorder, this leaves the neurotics with their phobias and anxieties and panic attacks hiding quietly in their cells, not eating and/or not sleeping and/or being punished for low-level disobedience. In the outside world, they might end up on the general ward of a local hospital; here, their disorders frequently pass unnoticed in the muddle of daily life. Sometimes, they are prescribed something to help them; sometimes it is stolen by other prisoners. The parliamentary mental health group has taken evidence on the victimisation of mentally-disordered prisoners who report being robbed, bullied and indecently assaulted.

Others with neurotic disorders are screamingly obvious. A man was shipped into the Scrubs from Highbury magistrates in north London a few months ago. Suffering from depression after the break-up of a relationship, he had slapped a police officer. Within four hours of arriving, he had cut his throat from ear to ear including his jugular and slit both wrists. All the prison could do was to rush him into Hammersmith Hospital to tend his wounds for four days and then beg the Home Office to allow them to transfer him to a local psychiatric ward to treat his anxiety. They refused: the courts had ordered that the man be held in custody awaiting trial and they deemed the security in the local hospital was inadequate.

Some of the psychotics too can be withdrawn - "quietly mad" in the language of doctors - and likely to remain undiagnosed and untreated. To untrained prison officers, they may seem irritating, asking the same question over and over again, or simply weird, like the man who was frightened of water. He would sit on one of the wings at the Scrubs staring at a splash of water on the floor, worrying that it was evaporating too quickly. It was two weeks before he clumped a member of staff, triggering a process that diagnosed his paranoid schizophrenia and eventually transferred him to an outside hospital. (His worry about water was that it was poison and he would die if he drank it.)

Once they are spotted, the psychotics can often be stabilised. There is a young schizophrenic in the Scrubs at the moment: his father lives abroad; his mother has been evicted from her home and is sleeping rough somewhere; he ended up living on the streets, cold, hungry, hallucinating and finally trying to steal a woman's handbag. He is on remand awaiting trial and he is now stable and relatively secure in a single cell with a television. His main worry is that the court may take pity on him and put him out. The big problem with psychotics is that, if they refuse medication, the prison - unlike a hospital - has no right to treat them against their will.

One of the staff at the Scrubs told us about a psychotic man who refused to be treated. The prison wanted to transfer him to an outside hospital, but there was no bed. Without medication, the man's condition started to deteriorate. Soon, he took to standing on the sink in his cell, holding his arms out sideways and swallow-diving head first on to the concrete floor. His face was soon broken and bleeding, and staff were then able to use their limited power under common law to tranquilise him by force for long enough to stitch his wounds. Then the law required them to stand back and let him carry on swallow-diving. At one point, they got special permission from the Home Office to hold him in a padded cell in a strait-jacket (something which officially is no longer done in UK prisons). Finally, an outside hospital bed was found for him.

The most disruptive are those with personality disorders. Technically, they are not suffering from a mental illness, but their behaviour is distorted by traumatic experience, usually in childhood: one out of every three men who is remanded into custody by our courts has been in care as a child; one in three women has been sexually abused; one in ten men has been sexually abused. They may be withdrawn, mistrustful, aggressive, anti-social. At the top end of the scale, they may be grossly callous and cruel. In prison cells, they cut themselves, they bang their heads against the wall, they manipulate, they protest, they swallow pills like sweets. But, from the doctors' point of view, although they may be mentally disordered, they are not 'ill' and are often dismissed as untreatable. The daily strain of the treatment gap, dealing with so much mental disorder without the resources or skills to match the challenge, produces some real tensions in the prison.

On Saturday August 21 - the same day that John Leadley was stabbed in the face - there was another prisoner on the segregation unit who was causing trouble. He had been moved there from B wing, where he had been cutting himself, and now he was standing on the water pipes of his cell with a ligature around his neck, threatening to hang himself. Just a fortnight earlier, a prisoner had succeeded in committing suicide in the seg, so the senior officer there was in no doubt about what he wanted: this man had to be transferred immediately to the prison's health care centre, where there are seventeen cells for acutely ill prisoners. He called the centre, and one of the doctors came down.

To the senior officer's horror, the doctor refused to take the prisoner. He said the man was suffering from a personality disorder, which was not treatable, and that this behaviour was simply a self-harming gesture. The senior officer became agitated, reminding the doctor that two weeks earlier, they had cut down a body which had then laid in the cell all day while the investigators did their work: "I'm not going to engage in semantics with you. Suppose it happens again. You just can't take the chance." The doctor stood his ground. The senior officer reluctantly backed down but told the doctor: "If this goes wrong, I guarantee you I will stand up in coroners court and point the finger at you". Then he recorded all that had happened in the seg's observations book.

In the event, the man did not hang himself. The friction passed. But the problem remains. Just like the officers in the seg, the staff in the health care centre see themselves struggling to deal with an impossible problem. Apart from the shortage of specialist psychiatric help, the prison has12 unfilled vacancies for general nurses, leaving only 28 to cover the whole prison, day and night, for all forms of illness. Sometimes, they try to fill some of the gaps with agency nurses, but they are expensive and can wait up to a month for security clearance. They have only 17 in-patient beds for the whole prison. And, running through all this, they are medical staff working in a prison which has different priorities and understandings and even language.

Are these men prisoners or patients? Who should win the argument if prison officers want to search a man's cell but health staff say it will aggravate his mental disorder? What should happen when health staff want half-a-dozen prisoners out of their cells for some group activity, and security staff say they cannot do that without three officers - who are not available? How can prison officers be expected to understand the mental disorders of the men they are looking after, if generally they have no psychiatric training and are not even allowed to read their medical notes? How can health staff provide regular care when their patients are suddenly snatched from their surgeries and sessions because there has been an incident and prison officers are locking down the wing?

The diagnosis of mental disorder is difficult - and made more difficult by the possibility of prisoners trying to manipulate the system. Certainly, there is some manipulation: a prisoner at the Scrubs this summer repeatedly evaded his court appearances, because he knew that if he got into the prison van and cut himself - even superficially - the security company would refuse to transport him. It was a prison nurse - not an officer - with years of experience who told us that some prisoners manipulate the regime for preventing suicides: "Let them hurt themselves, let them cut themselves, it's their responsibility. I have stood outside a cell and watched an inmate tear up his clothes and make a noose and put it round his neck; And I've said 'Yeah? Go on then, do it.'"

The bottom line is that, with difficult patients, poor resources, and the culture clash between security and care, prison staff can be stretched to breaking point by some of the people in their care. Look at what happened in September with Terry Moreton. He has a history of violent crime, including a serious attack on a teenaged girl at a railway station, but he was in the Scrubs awaiting trial for a fairly minor offence. The Inreach team had spotted him and diagnosed him as suffering from bipolar illness (manic depression) as well as personality disorder and they had given him some medication. But he had stopped taking it. For several days, he had been thumping his head against his cell wall in B wing, and officers had opened an F2052SH form, which is kept for any prisoner who is deemed to be at risk of self-harm. That Friday night, Moreton ran out of tobacco, threatened to kill himself if he was not given some, assaulted his cell mate and barricaded the cell.

During the night, officers managed to get in and took him down to the seg. On Saturday morning, the senior officer came in and said Moreton did not belong there: only 48 hours earlier, the doctors had said his illness was so bad that he was not fit for segregation, and there had been no doctor on duty the previous night to 'fit' him. He called health care and, before long, a doctor came along with the duty governor. By now, Moreton had sharpened the end of a metal flask, which is used to give the prisoners hot water, and was threatening to slash anybody who came into his cell. The duty governor wanted him restrained. The seg officers said they could not do that: if he was not deemed fit for the seg, they would be guilty of assault if they restrained him. The doctor suggested he could now fit him, and so the officers went in.

Moreton was taken to the seg's safe cell and strip-searched. The doctor then suggested that he might not be fit for seg after all, which was bad news for the seg officers if it meant that they had committed an assault, but good news if it meant they got rid of him. Late that Saturday, Moreton was finally taken to the in-patient health care cells where he proceeded to tear the plastic lid off the toilet, break it into shards, wrap one of them in a torn shirt and threaten to slash anybody who came near him. He then smashed his sink, broke the water pipes and flooded the whole health care unit in two inches of water. The health staff now said he was, in fact, fit for seg and sent him back.

The following morning, an officer outside the seg saw him sharpening his breakfast spoon on the window. The senior officer persuaded him to hand it over. That night, Moreton made a dummy out of his bedding, hung it from the window and hid himself under the bed, apparently hoping to ambush any officer who came in. He spent the next day literally gibbering in his cell, while those in neighbouring cells threatened to kill him, because he had now kept them awake for three nights.

Seg officers were fed up with the health care staff. They felt their unit was being used as a dumping ground, that a solitary cell was the worst possible place for somebody who was mentally ill, that it was absurd that the man could be unfit for seg one moment and fit the next. Health staff were just as fed up. They, too, felt that their unit was being used as a dumping ground and that, although Moreton had a history of bipolar illness, he also suffered from a personality disorder - which was not treatable - and that it was the disorder, not the illness, which lay behind his behaviour. They explained that he lurched between being fit and unfit for seg simply according to whether or not he was taking his medication. And both groups felt they suffered from decisions made by night staff and weekend staff who were not equipped to make these decisions. The truth, of course, is that they are both right: they are both being used as a dumping ground; the whole prison system is.

For years nobody really bothered about the mental health of prisoners. Only four years ago, research published by the British Medical Journal found: no doctors who were in charge of prison inpatients had psychiatric training; only 24% of prison nurses had mental health training; patients were locked up for between 13 and 20 hours per day; services for the mentally ill in prisons fell far below standards in the NHS; patients lives were restricted and access to therapy limited.

In the last few years, as the problem has grown into a crisis, the prison service has started moving the mountain. Three years ago, they published a joint report with the department of health, Changing the Outlook, which frankly admitted: "There are too many prisoners in too many prisons who, despite the best efforts of committed prison health care and NHS staff, receive no treatment, or inappropriate treatment for their mental illness, from staff with the wrong mix of skills and in the wrong kind of setting." The prison service adopted the 'principle of equivalence', that prisoners should have the same health care as any other NHS patient, and took the historic step of handing over the commissioning to local Primary Care Trusts.

Since then, they have funded 300 community psychiatric nurses to work on Inreach teams, like the one in Wormwood Scrubs, providing a care-in-the-community service for prisoners; adopted the Care Programme Approach which should link their work with community teams before and after a prison sentence; and set up new screening for prisoners on their first night to do a better job of identifying mental disorder (the old system was found to be missing 75% of cases.) But can a prison pretend to be a hospital?

Some of the culture clash between care and custody can be dealt with. There is a new programme to teach prison officers how to spot the symptoms of mental disorder. In principle, the rules may be changed to allow officers to read some of the medical notes of their inmates, which are currently hidden from them on grounds of confidentiality. There is some hope that the new NHS computer system will cut the number of prisoners who turn up with no medical notes.

But there are other problems which are harder to reduce.The Mental Health Commission last year warned of the limits of community care in a prison: "The prison 'community' cannot offer any real equivalent to the support and care available outside prison, and any assumed equivalence between prison and the community outside greatly under-estimates the isolation and bullying of the mentally ill in prison and the stigma of mental illness in such a situation."

There are limits also to what can be done with medication in prison, because prisons cannot treat inmates against their will, which means that those who reject their medication can deteriorate rapidly. But, with few exceptions, we have found prison health staff are deeply opposed to allowing compulsory treatment in an institution which is not a genuine hospital: the risk of accident and the temptation for disciplinary prescribing are too great. The Mental Health Commission last year said it was "deeply concerned" about the "extremely serious risk" of unrecognised or unchallenged coercive treatment in prisons.

Beyond that, it is hard to see how prisons will find the cash to reverse years of neglect, and there is a real fear among governors that the Treasury have stored up trouble by failing to ring-fence future funding for prison health, leaving Primary Care Trusts free to divert the money to patients who are more popular than offenders. The treatment gap will not be closed, and yet the problem will persist.

Prisons are the wastepipe down which other institutions send their rejects. The flow of mentally-disordered men and women into custody is being pumped not simply by the failure of the Department of Health to organise effective care in the community. Within the criminal justice system, the police and the courts have special systems to divert the mentally-disordered. They don't work.

Police can remove them to a place of safety for 72 hours for assessment - but only if they are in a public place and not if they are being charged in the police station, and only if their illness is recognised by the officers and not if it involves drug abuse, and only if they are in immediate need of care and control and not if they are merely ill, and only if the police have time to go through the procedure rather than simply processing them for court.

The courts, too, have powers to send people to prison or hospital to be assessed - but only if they are told that a bed is available. In 1990/1, all the courts in England and Wales between them used this power only 412 times. Since then, despite the massive increase in the charging of mentally-disordered people, they have used it even less: in 2000/1, they sent only 168 defendants to be checked for mental illness. Two years ago, the Home Office studied its court diversion schemes and found that "many schemes are currently ineffective" - patchy in geographical availability, peripheral to local psychiatric services, poorly designed, "inadequately supported" by local hospitals and "unpopular with local psychiatrists." Juliet Lyon, director of the Prison Reform Trust, earlier this year said the network of 136 schemes had "virtually fallen apart".

So, at a prison like Wormwood Scrubs, they can find themselves dealing with a man like Manuel York who was agitated, unpredictable and hallucinating sounds, visions and physical sensations. A visiting psychiatrist at the prison immediately recognised that he was profoundly disturbed and diagnosed hebephrenic schizophrenia, a particularly disorganised version of the illness. And yet that man, who had been arrested for making unwanted phone calls to a woman, had passed straight through the police and the courts without anybody stopping to ask if he was ill, let alone diverting him for treatment.

And the failure is repeated when it comes to their release. There was a man at the Scrubs recently who tried to hang himself three times in a week; he also bit, punched and spat at staff. Then he left the prison - and nobody knows where he has gone. Research two years ago found that 96% of mentally-disordered prisoners were put back into the community without supported housing, including 80% of those who had committed the most serious offences; more than 75% had been given no appointment with outside carers. And yet Department of Health policy requires all mentally-disordered prisoners to be given a care plan on release.

When the Home Office researched mentally-disordered defendants in court, they found that: "The majority were not career criminals who had become mentally ill. Most appear to have offended in the context of mental illness and social exclusion, having fallen through gaps in community care." When the Home Office recently reviewed the care of those men and women in prison, the researchers concluded by quoting an earlier survey: "It seems that mentally ill offenders will be as much at risk from society, as they will be a risk to society." The white prison vans still roll up outside the prison gates.

* To protect medical confidentiality, names and some identifying details of prisoners have been changed

Additional research by Roxanne Escobales

See below statistics of mental disorder in prison

Mental disorder in prisons -

  Male Remand Male Sentenced Female All UK All
78% 64% 50% 4.4%
Psychosis 10% 7% 14% 0.5%
59% 40% 70% 17%

Source: Office of National Statistics

Suicide and self-harm among prisoners -

  Male Remand Male Sentenced Female Remand Female Sentenced
Self-harm attempts
(current sentence)
5% 7% 9% 10%
Suicidal thoughts
(previous week)
12% 4% 23% 8%
Suicide attempts
(previous year)
15% 7% 27% 16%

Source: Office of National Statistics

Mental disorder includes

* mental illness such as psychosis or severe depression, often occurring as an episode in an otherwise healthy person and liable to respond to treatment;
* personality disorder, such as antisocial or paranoid, occuring as a continuing pattern of abnormal behaviour, sometimes the result of childhood experiences, generally difficult to reverse;
* neurotic disorder, such as anxiety and phobias, occurring at a level likely to interfere with normal activity, generally amenable to treatment;
* learning disabilities, usually involving significantly impaired intellectual functioning.

The most dangerous prisoner in Britain

The Guardian, December 2004

In the second part of his investigation into mentally-disordered prisoners, Nick Davies tells the story of a prisoner who has become a living nightmare, one of the several thousand severely ill held behind bars and denied a hospital bed.

We do not need to know exactly what happened to Glenn Wright when he was a child. It is enough to say that he suffered most painfully at the hands of an adult who forced him to live through nightmares. Before that, he was just an ordinary boy, growing up in Northampton in the 1970s, his father a supervisor at the Carlsberg brewery, his mother looking after the five children, of whom he was the youngest. Afterwards, he was himself a nightmare in the making.

Glenn Wright is not at all well-known to the outside world, but inside the prison system, he is notorious, and he is sometimes described by governors as the most difficult prisoner in the country. Even from a maximum security cell, he exerts enormous power over those around him. He does this most often by attacking his own body as a form of blackmail. He swallows razor blades and screws. He eats batteries. For years, he has kept open a savage wound in his leg; sometimes he puts his own shit into it. He pushes taps and broken porcelain into his rectum. He went through a phase of keeping a cup of blood in his cell to throw at anybody who tried to come too close.

Occasionally, in his campaign to control those he comes into contact with, he embroils others in his theatre of horror. He told a prison pyschiatrist that he would like to kill her, slice her, fry her and eat her. He has talked about murdering people and feeding them to dogs or to their own families. He has killed other prisoners without touching them: he talked three of them into hanging themselves; one recovered, and two of them died, and he was given life for murder and aiding suicide.

For the psychiatrists who have examined him, Glenn Paul Wright, now aged 32, is suffering from a severe personality disorder, the legacy of the relentless torment which he suffered as a child. The disorder pitched him into a life of offending, he ended up in prison, where he remains trapped in a regime which cannot treat him effectively. He is locked up alone for 23 hours a day. If he leaves his cell, he has both hands ratchet-cuffed to a prison officer with four others in escort; he has the option of exercise in a fenced cage. There have been times when, as a deliberate policy, he has been denied all access to psychiatric care. Routinely, he has been shifted from one high-security prison to another, even though prison psychiatrists have warned that this aggravates his condition.

The one solution to the problem - both for Wright and for the prison service - is that he might be tranferred to a high-security hospital for treatment. Five different forensic psychiatrists have assessed Wright and recommended that he be moved to a secure psychiatric bed. And yet, he remains in prison.

It is government policy that all patients be given "timely access to an appropriate bed". And that specifically includes prisoners. That policy is wrecked. There are thousands of severely mentally-disordered prisoners who are being denied transfer to outside hospitals. When the chief inspector of prisons' medical expert studied a sample of inmates in local prison hospital wings two years ago, he found 40% of them were not just mentally ill, but so ill that they should have been sectioned under the Mental Health Act.

The problem is two-fold: first, that the Department of Health has failed consistently to provide anything like enough secure psychiatric beds; and second, that many NHS psychiatrists simply do not want disruptive patients on their wards. Even the new units which the government has created for the most dangerous prisoners with severe personality disorders have refused to take Glenn Wright.

Dr Adrian Grounds, a forensic psychiatrist who works in prisons and who is also a senior lecturer at the Institute of Criminology at Cambridge University, estimates that there may be up to 3,700 prisoners who are so severely mentally disordered that they should be in a psychiatric hospital but for whom there is no bed. And where beds are available, delays are commonplace, and prisons are replete with stories of hospital psychiatrists ducking and weaving to avoid taking the most difficult prisoners.

One of the most senior forensic psychiatrists in the country told us: "Psychiatric teams come into prisons and make the most amazing judgments. I had a man in custody who was grossly manic, naked, occasionally smearing himself in his faeces and spending 22 hours a day in seclusion. He needed to be in a safe environment. But the visiting psychiatric team said: 'No, no, you're doing very well with him, he's fine here.'"

Glenn Wright drifted easily from a boyhood of pain to an adolescence of crime. By the time he was eleven, he was regularly truanting from primary school. By the time he was 13, he was expelled from secondary school and appearing in court for burglary and theft. He was recommended for a special school; it never happened. By the time he was 15, he was sniffing glue, smoking dope, swallowing blackmarket pills and serving his first sentence at Glen Parva Young Offenders Institute. He told psychiatrists he was hearing voices; nothing was done. When he was 19, he served his third sentence at Glen Parva. Since then - a period of 13 years - he has spent all but seven months in jail and has little prospect of release.

At the beginning of those 13 years, he was just another mentally-disordered offender. In Glen Parva, he was beaten up by other prisoners and swallowed some razor blades. He was transferred to Feltham, where he cut himself and was put on the hospital wing. The doctors said he had an anti-social personality disorder with impulsive acts of deliberate self-harm in the context of abnormal personality traits. He took to slashing his arms, cutting his wrists and swallowing more razor blades.

A couple of times, he was released, utterly unchanged in his behaviour. Rapidly, each time, he went back inside. The last time was in June 1996 when, aged 24, he was given three years for burglary. At the police station, he slashed his wrists. A few months later, he reported being assaulted and seriously harmed by other prisoners at Woodhill. Now, his self-harm started to take off: he swallowed screws, smashed up his cell, strangled himself, removed the tap from his cell sink and buried it in his rectum, tore a deep wound in his leg, ripped bits of fixtures and fittings from the cell walls and inserted them into his leg wound. Then his behaviour spilled over onto those around him.

On November 16 1996, a few weeks after he reported being seriously assaulted, he persuaded his cell mate at Woodhill, William Scott, to hang himself. On February 8 1997, still in Woodhill, he persuaded another cell mate, Peter Karelius Smith, to do the same. On January 27 1998, in Pentonville Prison, he persuaded Kenneth Cross to swallow an overdose of tablets and hang himself with a torn sheet; the noose broke, and Cross survived. As a result of what Cross told the authorities, Wright was prosecuted for the incidents with all three cell mates.

Thus far, there was no sign at any time of the prison service providing any effective treatment for his disorder, which was clearly becoming worse. That is not unusual: as we described yesterday, the mental disorders of prisoners were ignored for years; and even now, when the prison service has started to tackle the problem, treatment in prison remains deeply inadequate. And even those with the most serious disorders - those who would be sectioned and sent to secure hospitals immediately if they were in the community - are trapped in a system that cannot help them.

The chief inspector of prisons, Anne Owers, personally came across a man who was strapped into a body belt with his wrists clamped to his waist. When she asked about him, she was horrified to be told that he was kept like that permanently: it was the only thing the prison could do to stop him castrating himelf. One of her inspectors witnessed a prisoner gouging out his own eye ball - he had started on the other eye before officers managed to restrain him.

With their move to an outside hospital either blocked or delayed, the condition of many severely disordered prisoners deteriorates. The Prison Service is currently being sued by the family of a man who was stored on a hospital wing, awaiting a long slow transfer, and who tried to kill himself by hanging. He failed and, a week later, he tried again. Once more he failed. Two weeks later, he tried a third time and, by the time he was discovered and saved, he had been hanging long enough to suffer apparently permanent brain damage.

When he was charged with murdering his two cell-mates and attempting to murder a third, Glenn Wright finally forced the system to confront his condition. In the run-up to his trial, he was visited in prison and assessed by two independent forensic psychiatrists. Both agreed that he now had a serious psychopathic disorder and that he met the criteria for admission to a special hospital. On April 30 1999, Wright was given a life sentence at the Old Bailey and waited to be transferred to a secure psychiatric bed.

A year later, he was still in prison, untreated, still attacking his body and now covering himself and his cell in his own dirt. He waited like that for two years before finally, on November 7 2000, he was moved to Broadmoor.

The government says it has introduced "quicker and more effective arrangements for transferring the most seriously ill prisoners to appropriate NHS facilities" and claims that, on average, at any one time there are only 40 prisoners waiting longer than the official target of three months to be transferred to a secure mental hospital. The truth is more complicated than that. First, any patient in the community who was considered so mentally disordered that they had to be sectioned would be moved to a psychiatric bed within hours. A delay of a week would be extraordinary, and so a target of three months is scarcely effective.

Second, the government is dealing in averages, which disguise the long delay experienced by some prisoners. And the disguise appears to be deliberate. The Department of Health are refusing to release the true figures even though they have them and even though they relate to a government target for which they are accountable to the public. We spent more than two months asking for them: the department finally replied by claiming that the figures are "management information, not routinely published".

Third, prison staff report that outside doctors will fiddle the system by stalling their assessment until they have a bed available so that the true length of the delay is disguised. Finally, psychiatrists who work in prisons tell us that the process of asking for a transfer is so time-consuming and so unlikely to succeed that they begin it only for the most extreme cases, leaving a long queue of severely mentally-disordered prisoners who ought to be transferred but who do not even get to the starting line (or show up in the government's statistics).

There is a man in a Midlands prison now, for example, who is below average intelligence, diagnosed psychotic and hearing voices, frequently cutting himself, his condition exacerbated by being held in prison. The consultant psychiatrist who works part-time at the prison says he should be in hospital, but he is not dangerous enough for a high-security bed, not sick enough for a medium-security bed and too likely to abscond for a bed on a local locked ward.

The Department of Health's refusal to provide enough secure pyschiatric beds is like a boulder in the doorway that should lead the very ill from prison to hospital (just as their failure to provide effective community treatment diverts so many of them to prison in the first place). This summer, Dr Adrian Grounds of the Institute of Criminology told a Cambridge conference that the shortage of secure beds was so grave that: "Overall, the picture was of a system that was under such pressure that it was almost seized up."

Dr Grounds referred back to research in the early 1990s by professors John Gunn and Tony Maden, who analysed samples of prisoners to discover how many were in clinical need of transfer to an NHS psychiatric bed. Acknowledging that this was "fraught with difficulties and uncertainties", Dr Grounds applied their findings to today's much larger prison population and concluded that there may now be up to 3,660 men and woman who are so severely ill that they should be in a psychiatric ward but who are trapped in prison for want of a bed. Most of these men and women will not even have been assesed for transfer. Of those who are, we are told that at the moment there are some 300 men and women who have actually been sectioned but are still stuck in prison cells. Dr Grounds said: "The policy aim of diverting the mentally disordered from prison to the care of health and social services has chronically failed."

The Department of Health refused to comment on Dr Grounds' findings even though we gave them three months to do so. They said it was no longer their job to provide secure beds - it was a matter for the local health trusts. If Dr Grounds is right, up to 3,700 prisoners are blocked from appropriate beds - and yet, according to the latest available figures, in 2002/3 we transferred only 219 sentenced prisoners to psychiatric hospitals - and 45% of them (99) were sent back again. Which is what happened to Glenn Wright after he finally made it to Broadmoor in November 2000.

He lasted only four months. During that time, he threatened staff, incited other patients to refuse their medication and was put into seclusion where he tore panels from the wall and covered himself in faeces. The doctors at Broadmoor did not dispute the diagnosis which had brought him there but told the Home Office that he could not be managed safely in their high-security hospital. In March 2001, he was taken back to prison.

Over the following twelve months, he was shuttled from one high-security unit to another. This is a strategy which is known among prison governors as 'sale or return': when a prisoner is difficult, the governor ships him off to another prison on the understanding that, if he acts up there too, the second prison can send him back. The inspector of prisons, Anne Owers, earlier this year told the parliamentary joint committee on human rights: "You have a whole category of prisoners who are basically on a merry-go-round. That is likely, I would have thought, to exacerbate their mental condition." She said she knew of one prisoner who had been transferred 30 times in two years and who was now dead. After he was removed from Broadmoor, Glenn Wright was moved seven times in twelve months from one maximum-security unit to another, finally bouncing from Long Lartin to Woodhill, back to Long Lartin, then back to Woodhill again.

His attacks on himself became frantic. In Full Sutton, he swallowed three batteries and was taken to general hospital to have them removed. In Whitemoor, he cut his arms and covered himself in shit. In Long Lartin, he kept a razor blade under his foreskin and put a tap up his backside. In Woodhill, he buried a pen and more shit in the open wound in his leg and put another tap up inside him. En route to one of his stays in Long Lartin, he produced a razor blade, cut up his clothes to make a noose and started strangling himself. When he finally arrived at the prison, he produced another blade, made deep cuts in his arms and legs and refused to have them treated. While this was going on, the Home Office and Rampton high-security hospital refused even to assess him and said he could have hospital treatment only if he improved his behaviour.

It is not just the shortage of beds which keeps mentally-disordered prisoners out of secure hospitals. There is clear evidence that the NHS discriminates against them. In the ten years between 1992 and 2002, the number of men and women sectioned to secure beds increased by 24% - but the number sectioned from the courts and the prisons fell by 24% (and this was while the prison population was rocketing). As the Department of Health itself admitted in a policy paper three years ago: "In some cases, it seems that the NHS does not always give a prisoner patient the same level of priority as they would have in the community."

At its worst, this sees NHS psychiatrists occasionally resorting to apparently cynical misdiagnosis to justify rejecting difficult patients. The mental health group MIND told the parliamentary joint committee on human rights in January of a mentally-ill woman who succeeded in being transferred from prison: "She got into the ward and was apparently more trouble than the ward anticipated, so she was hastily reclassified as having an untreatable personality disorder and booted out that same night. One wonders whether this was on medical or management grounds."

There is clear evidence that, under pressure of a gross shortage of places, hospital doctors may cherrypick the easiest patients. Researchers in 1998 studied 44 women from Holloway prison who had been referred to secure hospitals. Half were turned down, and the researchers found that, compared to those who were allowed to have beds, the rejected women were more likely to have harmed themselves, to have suffered childhood abuse, to have committed serious offences and to be seen as violent or dangerous. The research concluded that their rejection was the result not simply of difficulties in treating them but also of "inadequate service provision".

Further evidence of distorted judgements by NHS psychiatrists was uncovered by Home Office researchers two years ago who studied what actually happened when mentally-disordered offenders were transferred by courts to secure hospitals. They found that in reality, they were less likely than other patients to be violent on the ward or to re-offend after treatment: "The perception amongst some psychiatrists that admissions from court gain little benefit from admission, are more disruptive in hospital, fail to engage in follow-up, and rapidly offend upon discharge is almost entirely false."

NHS psychiatrists who turn their back on prisoners are aided by the fact that their hospitals have the right to refuse admission: prisons have no such power to refuse to hold an inmate, no matter how inappropriate. A senior forensic psychiatrist told us: "Putting mentally disordered offenders into prison is hugely ineffective. There is no argument against the moral point of view that they should be transferred. That is very strong. It is not just that the faciliities are not there. The whole ethos of psychiatry is against it."

Since Glenn Wright was returned to prison from Broadmoor, in March 2001, three more consultant psychiatrists have assessed him and deemed him suitable for transfer to hospital. Yet, he not only remains in prison but has, at times, been deliberately denied psychiatric care. He was held at Long Lartin for two months in early 2002, when the prison said explicitly that his regime would be one of containment and not treatment. Two years later, he was back in Long Lartin, where staff were instructed that he must be held in solitary confinement, exercised in isolation and denied access to a psychologist.

His self-harm now has become an explicit protest at the lack of treatment
for his mental disorder. Ironically, he has then lacked treatment for the physical damage he has caused himself. In March 2002, the general hospital at Milton Keynes, which serves Woodhill Prison, refused to remove jagged porcelain and batteries from his rectum, having already performed the operation twice before. In September 2002 and again in March 2004, the wound in his leg was untreated and said to be producing an unbearable smell. In some prisons, he is allowed to see a doctor only in the presence of six prison officers, leading him to refuse to have objects removed from his penis and backside. On other occasions, as part of his protest, he himself has formally refused to be treated.

The prison service reacted by continuing to shuttle him between prisons, despite the evidence that this was making his condition worse. In the second half of 2002, they moved him six times in seven months. Finally, in January 2003, he came to rest at Woodhill, where, for the first time in all his years in prison, he was given a full assessment and a care plan. He was given weekly meetings with a psychologist, and sessions in dealing with stress and self-harm. By April, he was showing signs of improvement and his meetings with the psychologist, which had originally been conducted through the hatch in his cell door, were held at a table.

Yet, despite this improvement, the prison service started to shuttle him again. In one prison, he claimed that officers in riot gear punched him and slammed his head against the floor. Another prisoner who was on a segregation unit with him, reported that officers were banging on Wright's cell door during the night to keep him awake and telling him: "As long as you are as you are, you're getting fuck all." The shuttling climaxed in a move to Long Lartin in March of this year, when the Woodhill psychiatrists complained bitterly that they had not been consulted and that the movement was causing his mental condition to deteriorate. Wright reacted in Long Lartin by putting shards of broken glass, porcelain and wood into his body, re-opening his leg wound and covering his body and his cell in his own dirt.

During this time, the Woodhill psychiatrists have tried repeatedly to transfer him to Rampton special hospital. In February 2002, Rampton agreed to assess him but kept him for only 36 hours before saying they could not help him. In November 2003, after his improvement at Woodhill, Rampton were sent three reports by senior forensic psychiatrists who agreed that he would respond to hospital treatment, but Rampton turned him down again without even seeing him.

Earlier this year, Woodhill applied to transfer him to Rampton's new unit for dangerous and severe personality disorders, one of four such units which are being set up by the government apparently to treat prisoners of exactly Wright's kind. This time, the Rampton admissions panel agreed that he had a personality disorder, noted that he had expressed motivation to be treated, acknowledged that he had been treated effectively at Woodhill but concluded that it would be "not appropriate" to give him a bed in the new unit.

Part of the difficulty here is that Wright is suffering from a personality disorder, ie behaviour distorted by earlier experience, and this is not regarded as an illness like schizophrenia. NHS psychiatrists often argue that personality disorder is not treatable and that it would be wrong to fill a scarce bed with somebody who cannot profit from it. However, we have spoken to senior forensic psychiatrists who say that this is too narrow an interpretation of treatment, that hospitals will take in a man who has been paralysed in a car crash, even though his condition cannot be cured, in order to help him manage his condition. The same logic, they say, should be applied to those who are mentally disabled and so a man like Glenn Wright should be given the systems and therapies that will help him to manage his condition, even if he cannot finally be freed from it.

Wright's disorder may be extreme, but the trap of 'untreatability' catches most of the estimated 50,200 men and women who have a personality disorder and who have ended up in prison cells. A survey of health trusts two years ago, revealed that only 17% of them were providing a dedicated service for personality-disordered patients, and there is only one secure residential unit in the whole of the NHS - Arnold Lodge in Leicester - which provides forensic beds reserved for them. Some units reject them as a matter of policy; others will admit them only in very small numbers; many also reject patients who have drug problems, which is commonly part of life for offenders with personality disorders.

Now, the government is proposing to change the law so that those who have disordered personalities can no longer be rejected as untreatable. Clearly, there is a benign argument for this, but there is also a real danger if the new law is not backed up by a regime of effective treatment. Without that, those who suffer from these disorders may be sectioned to secure hospitals and simply stored indefinitely even though they have not been convicted of any crime. The legal right to treatment makes sense only if there really is treatment on offer.

But the underlying problem for prisoners like Glenn Wright is that they are being managed by a system which insists on seeing them primarily as offenders who need punishment rather than as patients who need care. There was a poignant example of this in the case of a nineteen-year-old woman called Petra Blanksby. She was charged last year with arson with intent to endanger life and sent to Newhall Prison in Wakefield. But the life she had intended to endanger was her own: she had set fire to herself, and the flames had spread to her room. She had a long history of trying to hurt herself on more than 90 different occasions. Yet the system treated her as an offender and held her on remand in custody for 130 days. She hanged herself in her cell.

There are senior psychiatrists who are urging the government to mount a double-barrelled attack on the problem, by embarking on a major programme of investment in supported housing, employment and care in the community for mentally-disordered men and women; and by supplying the secure NHS facilities which would make sense of their claim to want to provide an appropriate bed for all mentally-disordered prisoners. These psychiatrists say the investment would yield a clear cost-benefit, particularly in crime-reduction.

In the meantime, tens of thousands of mentally-disordered men and women remain in prison, often deprived of the treatment they need. And several thousand of them, like Glenn Wright, are so severely disordered that they need hospital treatment. Once, in their struggle to contain him, officers locked Wright into a body belt in his cell. Within minutes, he had dislocated his own hip and wriggled free. The escape to hospital is much harder.

Additional research by Roxanne Escobales

See below for statistics and other case histories

NHS secure psychiatric beds -

  Current Estimated shortfall*
High Security 1046 420 - 780
Medium Security 2557 830 -1440
Local locked wards 514 830 -1440
Total 4117 2080 - 3660

* Institute of Criminology, Cambridge

Patrick Morley

Aged: 39.

Convicted: of rape, sentenced to life.

Diagnosis: psycopathic disorder.

In January 2000, after several years in prison, Morley was transferred to Rampton high-security hospital. There, he took part in a sex offenders programme, and, in January 2002, an independent psychiatrist reported that he was responding well and warned that he should not be returned to prison: "I have little doubt that if Patrick is returned to a purely custodial setting in which there was no understanding of his deficits and no capacity to work with him, the risk of repetition of his former anti-social behaviour on returning to the community, would be considerable."

Nevertheless, three months later, Morley was sent back to prison. A new doctor had taken over his case and told the Home Office that Morley had a bad attitude, was not responding to treatment and that he presented 'significant management problems'. Morley had been accused of sexually assaulting another patient, although no charge was brought. On the same day that they received this report, the Home Office agreed to move him.

In the background, two consultant psychologists and four trainees who had worked with Morley at Rampton objected to the move, warning that without this work, he was likely to commit more serious sexual offences. Another independent psychiatrist then reviewed the case and concluded: "I do not understand the rationale for prematurely curtailing Mr Morley's treatment at Rampton only for him to commence a similar programme in prison, but one in which the skills normally found in a hospital multidisciplinary team will be lacking." He added that evidence suggested that premature departure from sex offenders treatment programme actually increases risk of reoffending.

But the Home Office stuck to their decision. Morley's lawyers, Bindman and Co, went to court to complain, but the judges ruled that the decision of the doctor in charge was final, as long as it was fair and rational, and that the disagreement of others made no difference. Morley remains in a high-security prison unit.

Gary Frank*

Aged: 32.

Convicted: of rape, sentenced to seven years.

Diagnosis: severe mental impairment.

Gary Frank drove a car in which another man abducted and then raped a young woman. He was accused of knowingly assisting the attack. He was arrested and interviewed at length by police, prosecuted and tried, jailed and held in three different prisons without any of the police, probation officers, prison officers or prison nurses and doctors recognising that he was so mentally impaired that his IQ was in the lowest 0.1% of the population.

He could not remember his own birthdate nor count money nor tell the time. In prison, they recorded his condition: "Very distressed, not coping... Crying a lot... Not eating... Sitting on a chair in the dark... Says he wants to use a razor on his arms... Missing mum... A broken spirit, tearful." At Christmas, he asked if he could go home. The prison officers could see there was something wrong with him and took the highly unusual step of allowing him to sleep with his cell door open, and yet nobody diagnosed his severe mental impairment and nobody offered him any effective treatment for his depression, insomnia and anxiety.

His lawyers sent in a clinical psychologist who reported in some detail that he had "a very low level of intellectual functioning" and added that he was very distressed and not coping. The lawyers sent the report straight to the prison. Still he was offered no treatment, no move to the health care centre and no assessment to be transferred to an outside hospital.

Action was finally taken when a visiting consultant psychiatrist saw him, recognised his extraordinarily low IQ and severe depression and concluded that he was "one of the most ill-placed people I have seen in prison in 20 years of forensic psychiatric practice". Even then, the nearest secure hospital refused to take him, and the visiting psychiatrist had to spend nearly four months making phone calls and writing begging letters before somebody finally provided a bed in a secure unit. The court of appeal subsequently ruled that his statements to police and to his original trial were wholly unreliable, and overturned his conviction. Gary Frank said: "No wonder people commit suicide."

* Name changed at the request of the prisoner

Chris Edwards

Aged: 30

Arrested for: breach of the peace.

Diagnosis: mild psychosis

Chris Edwards was arrested for making inappropriate remarks to young women in the street. He had a history of psychiatric problems - some hallucinations, religious delusions - and he had been prescribed an anti-psychotic drug in the past. The police called a social worker to the cells where he was beating on the walls and removing his clothes and yet, despite all that, instead of using the 'place of safety' law to send him to hospital to be assessed, they charged him with breach of the peace and, for this minor offence, he was remanded to Chelmsford Prison.

That same evening, the prison was holding a man named Richard Linford, who was charged with serious assaults on a woman and her neighbour. He was being held in a single cell, because he had a history of far more severe psychosis than Chris Edwards as well as a record of violence which included an assault on a cellmate. On his previous sentence, he had been transferred to Rampton secure hospital. During the course of the evening, as more and more prisoners crowded into the jail, Linford was moved, to share a cell with Chris Edwards.

During the next three or four hours, prison officers realised that the alarm bell from the two men's cell was not working but did not investigate. At one point, they heard banging from a cell door on their landing. When an officer went to look, he found Chris Edwards had been beaten and kicked to death. One of his ears was missing. Linford had blood on his mouth and was talking about evil spirits. Chris Edwards had been in the prison for only nine hours.

An inquiry by police and prison services found that neither man should have been in prison and that Chris Edwards had been failed by "a systemic collapse of the protective mechanisms that ought to have operated". After the event, Linford was diagnosed as paranoid schizophrenic and belatedly transferred to Rampton high-security hospital.

Chris Edwards' parents subsequently sued and took their case all the way to the European Court of Human Rights. Two years ago, the court found that a series of errors had denied Chris Edwards his right to life. His parents continue to campaign for the government to make good its obligation to the court to erase such errors from the system.


The jailing of mentally disordered children

The Guardian, December 2004

Concluding his investigation into mentally-disordered prisoners, Nick Davies looks at the soaring number of children who have been locked up in prisons which cannot deal with their often alarming mental health problems.

We jail more children than almost any other country in the European Union. While politicians have been wringing their hands over the relentless increase in the number of adult prisoners, they have stayed almost silent on the fact that the number of children behind bars has been rising even faster.

During the 1990s, it just about doubled, with the sharpest increases among girls and children under 15. And now, while the adult prison population has fallen in the last six months, the number of children behind bars has risen again. We have obtained figures which show a dramatic surge in the number of children being jailed for breaching the government's Anti Social Behaviour Orders even though most of them have not been convicted of any criminal offence.

And since some 80% of children in custody suffer from at least two mental disorders - an even higher rate than among adult prisoners - the result is that young children with depression, anxiety, psychosis and severe personality disorders are now exposed to the same profound lack of effective treatment which afflicts adults behind bars. They are cut off from all psychotherapy and highly likely to be denied access to specialist child and adolescent psychiatrists.

And, just like adult prisoners, the most severely disordered children also suffer from the blockages which obstruct their transfer to secure hospitals. We have been told by psychiatrists who work in the system that there are at least a hundred children who are trapped in custody, often in punishment cells, even though their mental condition is so bad that they should be sectioned and sent to secure psychiatric beds.

These problems are particularly acute for mentally-disordered 15 and 16 year old boys. The surge of younger children into custody has pushed them out of the local authority secure homes and secure training centres, where the law allows them to be held in conditions designed for children, and into Young Offender Institutions, which are simply prisons for young adults up to 20 years old and which run a far tougher regime. There are up to 450 boys who are officially regarded as vulnerable being held in these prisons whose staff can only struggle to cope with the intensity of their problems.

Kevin Jacobs was 16 when he was sent to a Young Offenders Institution. By that time, he had been let down by the state for more than 14 years. He was less than 18 months old when he was first taken into care by Lambeth Social Services because his parents, who had learning disabilities, were failing to cope, yet he was not finally put on the register of children at risk for two-and-a-half more years; he was not presented to the fostering panel until he was eleven; even then, he was never placed with a foster family; and there is no evidence he was ever considered for adoption. Instead, he was shuttled between his family and various children's homes until, at the age of six, he was sent to a home outside London which was already being investigated for alleged abuse. Within two months of arriving, he complained that he had been sexually assaulted by an older boy and by a member of staff. Yet, he was left there for months still reporting more abuse and, when the police finally interviewed him six months later, they took no action.

When he was only four, Lambeth social workers asked for him to be assessed for special educational needs. It took nearly seven years for the education authority to do so and, in the meantime, he had been permanently excluded from three mainstream schools. Also when he was four, a paediatrician asked for him to be given a full psychiatric assessment. That was not done. When he was 15, another paediatrician made the same request - the community mental health team said they were too busy. His severe disorders and depression were never effectively treated.

From this childhood of institutional failure, abuse and confusion, Kevin stumbled into a criminal justice system which was geared to punish him but reluctant to help. He was eleven when he was first arrested, for setting fire to a barn. The police cautioned him and let him go. A Lambeth social worker who had been trying to help him and his four siblings, noted in his file: "We have wasted the best years of these children's lives already." When he was 12, the courts locked him up for two years for burglary, assault and more arson. Early in the sentence, a doctor recommended he be transferred from the secure children's home to a psychiatric unit, but they could not find him a bed. Later in the same sentence, another doctor found Kevin was a threat to himself and to others and urged that he be moved to the secure unit for adolescents at St Andrew's Hospital in Northampton, but St Andrew's said he was not ill enough and rejected him. During this two-year sentence, Kevin cut himself on some 40 occasions.

By the time he turned 16, he was locked into a pattern of crime and self-harm, often at the same time. He burgled a doctor's office to try to inject himself with insulin; stole a knife to cut himself; stole another knife to cut himself and some paracetemol for an overdose. Twice, he was taken to hospital and resuscitated. The police arrested him for trying to rob an old lady in front of his own social worker. In the cells, he cut his arms with cutlery and swallowed an overdose of ibuprofen. At court, a doctor and a worker from the local Youth Offending Team produced reports on him. Both agreed he was at serious risk of harming himself. Both said he needed a full psychiatric assessment, but it did not happen. Both said he needed secure accommodation, not prison; the magistrates agreed; but the secure accommodation was all full. The Youth Justice Board, to whom the local YOT worker reports, might have pushed and got him onto a waiting list, but vital paperwork which might have persuaded them to do so never reached them. So Kevin was sent to await trial at the Young Offenders Institution at Feltham, in Middlesex. That was when the trouble really started.

In his first two weeks in Feltham, Kevin was interviewed by a prison officer on reception, a prison GP, a prison psychiatrist and a psychiatrist preparing a report for the court. None of them was a specialist in child psychiatry, and all of them were working blind: the Youth Justice Board report on his history, which included his self-harm, was misfiled and never made it to the prison; nor did his medical records and other history. Kevin himself, like many prisoners, chose to conceal his history of disorder, claiming that he had not harmed himself recently nor ever tried to kill himself. Nevertheless, it was clear he was at risk. He was placed in the health care centre but, after three days, he was moved to a standard cell on the juvenile wing. The prison opened a F2052SH form to ensure he was checked regularly to stop him harming himself, but, after ten days, the form was closed.

When he went back to court to be sentenced a couple of weeks later, he was given a six-month detention order, three months to be served in custody, three in the community. There is no evidence that the court made any effort to ensure that he served the three months in a secure childrens home. He went back to Feltham, where he embarked on an erratic journey of self-harm and suicide attempts while the prison struggled and failed first to treat him and then to transfer him to a secure hospital bed. The sixteen-year-old boy began to slide into despair.

He was bullied, smashed the light bulb in his cell and was found lying on his back bleeding from both arms. The prison re-opened the F2052SH form on him and sent him to the health care centre. Twenty-four hours later, he was back in a standard cell where he was spotted with equipment to make a noose. So he went back to the Health Care Centre for another 24 hours.

The pattern was clear. Kevin would harm himself. Prison officers, one of whom described him as "a timebomb waiting to explode", would try to move him to health care, who would house him for a short time or turn him away because they were short of staff and/or had no suitable cell available. Sometimes, he was held in solitary confinement in the segregation unit; often he was held in a standard cell whose bars and fittings are not made safe against self-harm and suicide. The prison psychiatrist reviewed him four times and each time, still working blind without Kevin's missing paperwork, he accepted Kevin's assurances and concluded that he was at no immediate risk of suicide or self-harm: he advised that Kevin needed "no further psychiatric input" and referred him on a non-urgent basis to be seen by a trainee psychologist who was working in the prison.

Ten weeks into his sentence, two prison officers were leaving Feltham at the end of their shift on a day when Kevin had cut his arm with a broken cup and been told there was no room for him in health care. The officers were so anxious about him that they turned around and went back to tell the governor that Kevin must have a nurse to tend to his injury and special clothing which could not be turned into a noose. When they reached the wing, they asked another officer to check on him: Kevin was squatting beneath his window with a sheet tied around his neck, pulled tight against the bars above him, drifting into unconsciousness. They saved his life, walked him over to health care, insisted he be given a bed and finally left for the night. The suicide attempt was not recorded in his medical notes. Nor were his family told. The incident was not investigated.

During his first week on remand in Feltham, the psychiatrist who prepared his court report had recommended that he be transferred to a secure bed at the Bethlem Royal Hospital in Beckenham, Kent. This psychiatrist warned that, if he remained untreated, Kevin's behaviour would remain criminal and destructive. But the Bethlem wanted a consultant's opinion. The prison provided it. The Bethlem then said they would have to assess him themselves and said it could be four weeks before they did so. Four weeks later, they offered to assess him after a further four weeks - on the very day Kevin was due to be released from prison. In the background, the Youth Justice Board made no move to transfer him to a secure children's home.

As the prison struggled to protect the boy from himself, Lambeth Social Services held a series of meetings about his future and decided that it was too expensive to keep him at the children's home in Guildford, Surrey where he had been living for 20 months before his arrest. Since this was one of the few places where he had ever settled, Kevin was distressed by the news. Since Lambeth were then unable to come up with any other plan for him, and with his release date approaching, he became morose and anxious.

An outreach worker from the health centre recorded that Kevin was frightened and angry about the lack of a release plan and contacted Lambeth social services, the YOT worker and two prison governors to alert them to the crisis. The social worker and the YOT worker were both urging the prison to organise a full psychiatric assessment because of the risk that he would harm himself. A week later, Kevin was caught trying to break his cell light and tearing up sheets to make a noose. He said he had no real intention of killing himself. The psychiatrist judged he was at no immediate risk of self-harm. The health centre said their unit was too noisy for him. He was put in an ordinary cell, where officers checked on him every hour. Kevin told staff he was having flashbacks to being sexually assaulted as a child in care.

Two days later, his social worker came in to see him and told him there was still no sign of anywhere for him to live after his release, which was now just three weeks away. That afternoon and evening, Kevin sat in his cell watching television: there was a shortage of staff, and so he was not unlocked and, for five hours nobody spoke to him at all. The trainee psychologist visited him, he asked her to stay, she said she would come back the next day. By 10.45 that night he was asleep. At 2.15 in the morning, an officer looked through his hatch and found him still asleep. At 2.35, he was found hanging from his window bars. At three o'clock, the ambulance crew instructed staff to stop attempts to resuscitate him: Kevin Jacobs was dead.

It is three years now since the prison service and the department of health declared: "The government is very clear that the needs of juveniles and young offenders have to be a priority. It is therefore important to ensure that services for young offenders take account of the special and often very complex needs of this group." Those words have not been translated into reality.

An inquest into Kevin Jacobs' death found his death was "suicide to which neglect contributed". Most of those who dealt with him in prison were clearly worried about his welfare and often went beyond the call of duty in trying to protect him. The neglect to which the inquest jury referred was, in their own words, the result of "gross deficiencies within the system."

This systemic failure has a familiar shape. On the one hand, just about everybody who deals with young offenders knows that the best way to stop them committing crimes is to keep them out of custody and to attack the problems which are pitching them into crime in the first place. The government knows it too. Four years ago, they created the Youth Justice Board with a network of local Youth Offending Teams with the precise aim of pulling together different agencies to tackle the individual roots of any child's crime. And yet, the same government has sabotaged the YJB's work.

Relentlessly, this government has joined its immediate predecessors in driving more and more children into custody. Dr Barry Goldson, senior lecturer at the University of Liverpool, analysed crime figures and found that between 1992 and 2001, the number of children being jailed each year soared by 90%, from 4,000 to 7,600, while those held on remand jumped by 142%. The number of children under 15 sent to custody increased by 800%, the number of girls by 400%. The average length of their sentences also increased, by 16%, for example, for boys aged 15 to 17. And yet, he found, there had been no significant change in the number or seriousness of the offences children were committing; and no impact on re-offending. This was all about a politically-led drive to jail more children.

This government came to power in 1997 with a prominent pledge to "fast-track the punishment of persistent young offenders". Since then they have kept up the rhetorical pressure on courts and reinforced it with: the 1998 Crime and Disorder Act which specifically extended the powers of the courts to impose longer sentences on children aged 15 to 17 and made it easier to impose custody on children aged 12 to 14; and the 2001 Criminal Justice and Police Act which gave magistrates power to remand children as young as 12 for minor repeat offences whereas previously that had been done only for the most serious crimes.

They launched their new Detention and Training Orders for juveniles, which were seized on by magistrates, causing another spike in the jailing of children. In 2002, they launched their Street Crime Iniatitive. The inquest into the death of Joseph Scholes, who hanged himself, aged 16, in Stoke Heath YOI two years ago, heard evidence that the government put direct and arguably improper pressure on the Youth Justice Board to make sure that children picked up under the initiative were sent to prison.

More recently, the government have campaigned noisily to persuade police and courts to make more use of Anti-Social Behaviour Orders, which are civil measures which usually restrain their subjects from visiting named places or people but which can yield prison sentences of up to five years for those who breach them, even if they have not been convicted of a crime.

We have obtained figures which show a sharp surge in the jailing of children for breaching ASBOs: in the first four years of their use, some 255 children were sent to custody for breaching their ASBOs. The government were disappointed. Downing Street and the Home Office demanded they be used more often. The result was that in only four months this summer, 195 more children were jailed in ASBO hearings. (see panel). Some are only 12 years old. Often, these children had not even been charged with breaking the law. And 80% of them suffer from at least two mental disorders.

The result of all this incarceration is that the Youth Justice Board has been left struggling to fulfill its purpose. It now finds itself having to spend 74% of its budget buying places in custody with the prospect of spending another 1.6% just to keep pace with the new rate of ASBO jailings, all of which cuts the money which the board has left to spend on crime prevention. And these children cannot fit into the institutions which are designed to house them.

All of the local authority secure childrens' homes and all of the new private secure training centres between them hold fewer than 500 child offenders. Almost all 15 and 16 year old boys now end up in Young Offender Institutions, which have much lower staffing, much higher rates of self harm and suicide, far less education and far more chance of being locked up for 22 hours a day. They are prisons. And it is not just that these are children, but that significant numbers of them are officially regarded as 'vulnerable', like Kevin Jacobs.

When the YJB started work in 2000, it told the Home Secretary: "Young people should be placed in accommodation which most effectively meets their needs and the risk of harm that they pose to themselves and others. The accommodation should be appropriate for their age, emotional maturity and level of vulnerability." But YJB officials told the Joseph Scholes inquest in April this year that each day they can handle a hundred booking forms for children who have been sentenced to custody of whom 30 or 40 will be officially described as vulnerable and, if they are boys over 14, almost all of them will end up in a YOI. At any one time, they said, there are some 450 vulnerable children in YOIs.

Earlier this year, the board's new head, Prof Rod Morgan, opened a new building at a YOI and told his audience frankly that he hoped he could return soon with a wrecking hammer to knock the place down. But, inundated with child prisoners and short of funds, the board is being forced into reverse gear. Places in local authority secure homes are expensive, and the board has been cutting the number of beds it buys in them. As a result, two local authority security homes closed last year and two more are about to close. This means that the new private secure training centre which opened this year and two others which are due to follow in the next few years will end up largely replacing the homes which have closed instead of diverting children from prison.

For the mass of jailed children like Kevin Jacobs who suffer from mental disorder, this is a disaster. The Youth Justice Board, like the Prison Service, has genuinely tried to improve health care behind bars, but neither of them pretends that they have been able to do enough. In particular, there is a dire shortage of child and adolescent psychiatrists for juvenile prisoners. "What they get is other general psychiatrists or nothing," a senior forensic psychiatrist told us. A survey of psychiatrists working with young offenders in Scotland three years ago found that only three of them had forensic training and few even knew of a specialist child psychiatrist they could consult.

The process of finding a psychiatric bed for the most severely disordered children is even worse than that for adult prisoners, which we described yesterday. In the whole of England and Wales, there are only 44 secure psychiatric beds for children. They cannot cope with the surge of jailed children with severe mental illness. As a senior executive from the YJB told the Joseph Scholes inquest in April: "It's the view of the Board that about a hundred of these children should be sectioned and they should be kept in secure mental health accommodation, but I am afraid that does not exist." Even those who do get a bed can experience long delays - "three, six, nine months" according to a senior source - as first the specialist hospitals and then the local Primary Care Trust conduct their assessments and hold meetings.

The Joseph Scholes inquest heard from a former medical officer at the YOI where he hanged himself, who had personal experience of trying to transfer severely disordered children to secure hospital beds. Dr Jane Bright said: "My experience was that when we did have inmates who had gross psychiatric disturbance, and I'm talking about a psychosis rather than a non-psychotic patient, it was very difficult to get a forensic psychiatrist to see them, very difficult to get an appropriate bed for them. We had to wait weeks and weeks, spend hours and hours and hours making phone calls, very, very difficult. The resources just weren't there to back us up."

Kevin Jacobs' family are now suing the London borough of Lambeth, the Home Office and the Youth Justice Board. Their lawyer, Fiona Murphy of Bhatt Murphy, has told Treasury solicitors: "Imprisoning such a vulnerable and damaged boy was humiliating, debasing and degrading and had the effect of undermining his will to cope with his feelings of helplessness. His will to resist the suicidal ideation from which he suffered, was cumulatively undermined by the failure to plan for his release and address his medical needs."

Additional research by Roxanne Escobales

See below for statistics and case histories

Anti Social Behaviour Orders -

Average number of children jailed each month for breaches of ASBOs

April 00 to Dec 02 2.3
Jan 03 to Mar 04 6.8 (estimate)
May 04 to Aug 04 48.75

Source: Home Office and Youth Justice Board

Children in custody -

  Secure homes/centres Prison
Oct 02 471 2704
Jan 04 473 2194
April 04 459 2345

Source: Youth Justice Board

Mental disorder among children in custody -

  Male Remand Male Sentenced Female All
84% 88% N/A
Psychosis 8% 10% 9%
52% 41% 67%
(life time)
20% N/A 33%

Source: Office of National Statistics

Case histories -

Toni aged 17 assault YOI prison

Toni used to watch her mother trying to hang herself and taking overdoses and using drugs. Her mother used to batter her with her fists or a hammer or anything else that came to hand. Eventually, she threw Toni downstairs, so Toni put herself into care. Her father was never around much. In fact, her mother told her he was not her father, and so she is not sure who he is. She remembers once he said there were dead people buried under the floor, so she and her mother pulled up all the floorboards and found a dog's head. It still had its collar on. Toni says she understands life: "My view of the world is that we are living in hell and we are all ghosts."

She has thoughts she can't control and voices in her head. She has been in trouble with the police since she was eleven - stealing, terrorising the neighbours, setting fire to things. Recently, she cut a girl: "Sometimes I feel I have to cut people up if I don't like them. Sometimes it only takes them to do one thing and, if I don't like it, then I'll have to have a go at them." So now, she is in prison, in a Young Offenders Institution. She is 17.

She wants help for her mental problems but, as far as she remembers, nobody has ever treated her. It's the same in the prison. Sometimes, doctors come to the cell. "Everyone comes to your door at once and they just stare at you." So she doesn't talk to them. When she was first in prison, they put her on self-harm watch, but then she told somebody she'd like to attack another prisoner, so the next thing she knew she was hauled out of her cell and taken down to segregation.

She spent 20 days there, locked up alone for 23 hours a day day. She could hear people screaming and smashing up their cells. One night, two people tried to hang themselves. Toni thought she could hear people crying, but she wasn't sure if she was imagining it. A 60-year-old woman in the next cell had got hold of some glass and was cutting her arm. Being on the seg made her feel low, her thoughts started to get the better of her, and so she took to banging her head against the wall. She felt paranoid, she was sure there were ghosts in the cell and then, one time, all the books fell off her shelf, so she knew the ghost was there.

Jink 17 assault and threats to kill YOI prison

A lawyer from the Howard League for Penal Reform found Jink in a prison cell - a 17-year-old boy who weighed 22 stone and who was suffering from a congenital heart disease as well as mental impairment. Almost as soon as Jink arrived in prison, the prison doctors had said he needed to be sent to a secure psychiatric unit, but seven months later, he was still in his cell. When the lawyer found him, Jink was eating faeces and urine.

When the lawyer asked why this deeply disturbed boy had not been moved immediately to a psychiatric hospital, he found that Jink had simply got lost in the system. The prison were extremely worried about him but they could not give him the care he needed. One day, Jink used his clothes bag to hang himself from the bars of his cell. They cut him down, he was still alive and one of the governors went and had a cup of tea with him, and Jink told him how his dad had started buggering him when he was six years old and used to keep him locked up in the bedroom. The governor said he seemed very low.

Straight after that, the prison called together all the agencies who had an interest in the boy - social services, youth offending team, Welsh health commissioner, prison doctors, hospital psychiatrists, defence lawyer. But they still could not sort out where to send him and how to pay for it. And all the time he waited, Jink was being bullied mercilessly. If ever he left his cell, he clung to a prison officer for protection.

Jink had assaulted his mother and threatened to kill her. From time to time, they would take him to court for a remand hearing. It was the first time he had been in trouble. He hated going to court: the cubicle in the van was too small for him; he suffers from claustrophobia; once, the door got jammed and they couldn't get him out; he had a panic attack in the van.

The lawyer from the Howard League said it was simply unacceptable that Jink had been wating seven months for a hospital bed and so he got onto his computer and Googled psychiatric units and rapidly found one which was suitable. He called the unit, who agreed to assess Jink. Two weeks later, he was moved. He is much better now.

Henry 16 arson with intent to endanger life Secure Training Centre

Henry was severely sexually abused as a young boy. He has had mental health problems since he was nine. He hears voices, has blackouts and flashbacks, sets fires, harms himself, drinks to the point of hazard. The government says no prisoner with a severe mental illness should wait more than three months to be transferred to a secure psychiatric bed. Henry waited fourteen months.

The first six months were spent simply waiting for a local community psychiatrist to assess him. Finally, the psychiatrist declared that there was no suitable placement in the Primary Care Trust's area, so he had no remit to do the job. In the background, the local youth offending team had asked another psychiatrist to asses him, but that never happened and the YOT lost interest. Henry's lawyers appealed to the local director of public health, but he was off sick.

In the seventh month, the judge who was waiting to preside over Henry's trial said he was 'extremely concerned' by the delay. The lawyers wrote to the head of the local YOT and threatened to summons him as a witness before the judge if he did not take responsibility within two weeks. The YOT replied that it was really up to the PCT. The PCT said they would consider a non-local placement, but it was up to the community psychiatrist to arrange an assessment. The community psychiatrist said he would refer Henry to the Gardener unit in Manchester. The doctor who was advising the lawyers said it was the wrong sort of unit: Henry needed to be in St Andrew's in Northampton. The community psychiatrist said St Andrew's was private and too expensive and he could not refer him there. The lawyers thought the best thing would be for both units to assess Henry. It was the ninth month before they discovered that nobody had actually referred Henry to either place.

The lawyers then got an independent psychiatrist to assess Henry, and he agreed that the Gardener unit was wrong for him but suggested the Proudhoe in Northumberland. But the Proudhoe had no beds and said they could not even assess him for two more months. The lawyers wrote to the local MP who wrote to the PCT who said they still wanted the Gardener unit because it was cheaper. The Gardener unit also had no beds. Then Henry got lucky. The Proudhoe brought forward their assessment and agreed to take him - but not until the twelfth month, when a bed was due to come free. In the meantime, St Andrew's also assessed him, agreed to take him but failed to tell the lawyers. And they also had no beds.

The MP wrote more letters. The lawyers made a formal complaint to local social services, who started a dispute about how to proceed. They also lodged a complaint with the local PCT, who failed to reply. By now, it was the twelfth month and the Proudhoe bed was due to come free. It didn't - the patient they were expecting to leave had needed to stay. Then the lawyers finally heard from St Andrew's that they had been willing to take Henry, that they had had two free beds in the past couple of months but had said nothing because they thought he was going to the Proudhoe, and now both beds were filled again. They agreed to put Henry back on their waiting list, but only if they had a new referral from the community psychiatrist, who was still saying he should go to the Gardener unit. In the fourteenth month, after four assessments involving three units and seven doctors, Henry finally was given a bed at the Proudhoe. During all that time, he received no effective treatment for his illness.

Babs 17 affray and arson with intent YOI prison

As a little girl, Babs was abused sexually, physically and emotionally and spent years going in and out of care. In one foster family, she was sexually abused again. She also saw her young brother die. As an adolescent, she tried to get help but was told she was just being manipulative. She often committed offences.

When she was 17, she set fire to her bed, was taken to court and pleaded to be sent to a psychiatric hospital. When they refused, she ran from the court, met up with two friends, who were both schizophrenics, got drunk and swallowed all their medication with them, was indecently assaulted by one of them, trashed the house, set fire to the sofa, was subdued by police using CS gas and still not sent to a hospital. Instead, she was remanded in custody to await trial.

There, a prison doctor learned about her two personalities (Andy who she liked and Andrea who she didn't) and diagnosed personality disorder, mood disorder and post traumatic stress disorder. The doctor said she must be transferred to a secure hospital. But that took eleven months, during which time her condition careered downhill and the prison alternated between desperately trying to protect her and punishing her.

She made her her most serious suicide attempt when she hanged herself with a cable from her cell window. Officers cut her down and left her in the cell under constant observation. When she recovered, she wanted to leave and tried to shove her way past the officer in the doorway. Other officers were called. She was handcuffed, stripped, put into a strip gown and placed in the punishment block where one of her friends had hanged herself.

During her eleven-month wait for a hospital bed, Babs tried to hang herself using cables, laces, torn shirts and sheets. She cut and burned her legs, hair, face, hands and tried to cut off her thumb. She also tried to throw herself off the wing landing. On several occasions, she dug a wound in her arm and buried a pen in it. The pens had to be removed at a local hospital and she was warned that if she continued to damage her arms or hands, they might have to be removed.

Margie 17 breaking and entering and assault YOI prison

Margie has had trouble with mental illness all her adolescence. She has been sectioned twice and held in hospital for psychiatric care and she has been been prescribed anti-psychotic medication. She hears voices. Given the chance, she will drink eight cans of beer a day and a bottle of vodka a week.

The reason is not hard to find: she spent the first eight years of her life being beaten up by her father and watching him do the same to her mother; when she was eight, her mother fled to a refuge taking Margie and her four siblings with her; but, one night, the father found them and broke into the room and Margie hid and watched while he stabbed her mother 17 times and then kidnapped her brothers and sisters, leaving her alone with the corpse.

At first when she was arrested - for breaking into another girl's home and forcibly shaving her head - it looked as though she would be moved from prison to hospital without much trouble. The judge said her transfer was 'extremely urgent' and, within two months of her arrest, a psychiatrist assessed her and diagnosed schizoaffective disorder with post-traumatic stress and borderline personality traits. He not only recommended that she be transferred to a secure hospital but pointed out that she wanted to kill babies and added: "Formal therapeutic interventions are vital. It would be impossible to contain her behaviours and provide such intensive interventions, if she were not to be managed in a secure psychiatric unit.... Failure to implement such provisions at this opportune time can potentially result in far more serious consequences to property and people in the future."

But the primary care trust, who were going to have to pick up the bill, were not so sure. They sent another psychiatrist to assess her, and she concluded that the first psychiatrist's diagnosis was wrong, that Margie would not respond to treatment and would not mix well with other patients in the unit and might be malingering. The PCT refused to fund the transfer.

Margie was given a long sentence and no visible prospect of a hospital place. The last time she was heard of, she was extremely anxious: she had just heard her father was due to be released from the sentence he was serving for murdering her mother. She said: "My brain thinks that if I throw a stone in a pot then I won't die, and I don't throw a stone in a pot."

* the names of all children have been changed

Dying for a break

The Guardian, May 2005

Maybe nothing really changes. Several hundred years ago when red-faced judges and pot-bellied politicians were happy to procure power by ordering men to be hanged by the neck and left dangling to rot by the wayside, there was a popular rhyme: "Little villains oft submit to fate, so great ones may enjoy the world in state."

Think of Andrew O'Connor. When Michael Howard tells you that prison works, when the Prime Minister celebrates his tough line on crime, when you hear once again that this country jails more of its men and women than Burma or China, think of Andrew O'Connor - petty thief, pain in the neck, prolific offender, habitual prisoner, dead.

There is nothing so unusual about his being dead. People who work with offenders will tell you that, as things stand, there really are only three ways that most repeat offenders finally stop breaking the law - they just grow out of it; they kill somebody and get locked up for life; or they die. Some of those who die commit suicide, some of them accidentally overdose, most of them just die from general neglect.

As offenders go, there never was anything very unusual about Andrew O'Connor. Any social worker or police officer or prison officer reading this story could stop now and, with their eyes shut, they could predict just about all of his background: his family were poor, he grew up on crap estates, he never knew his father, he was knocked about by his step father, he was a nightmare at school and a pest on the streets, he was put into care, he was sexually abused. By the time he was ten, he was being arrested. By the time he was 15, he was doing custody. By the time he was 30, he was dead. Nothing unusual. There are thousands of Andrew O'Connors shuffling through the dark tunnels of our criminal justice system.

Now, watch how the judges and the politicians play their hand. And look back, in particular, at what happened after the Prime Minister, in the autumn of 2001, decided to make good on his pledge to be tough on the causes of crime by personally commissioning the social exclusion unit in the Cabinet Office, to find out what he had to do to stop the endless reoffending of people like Andrew O'Connor.

The SEU set off on a mighty investigation, using five civil servants to work full time for the following nine months, trawling through statistics and research, visiting prisons, interviewing experts, talking to officials in government departments, all in search of an answer. And they found it. In July 2002, they published their report, "Reducing re-offending by ex-prisoners" - 218 pages of analysis, packed with facts and figures, supported by more than four hundred footnotes. Its conclusions were clear.

Essentially, they boiled down to three big themes. First: "Prison sentences are not succeeding in turning the majority of offenders away from crime." Second: "A prison sentence can - and frequently does - make things worse." Finally: the real key to reducing offending was to attack its causes (just as the Prime Minister had been saying). Homelessness, unemployment, drug and alcohol problems, mental health problems, physical health problems, educational problems - these were the seeds from which crime grew, seeds which were fertilised by the impact of imprisonment.

In many ways, there was nothing new about these conclusions. Any front-line worker in the criminal justice system would have recognised them. The importance of the SEU report was that these conclusions were backed by irresistible evidence, supported by seven different government departments who had been consulted by the SEU, and delivered direct to the most powerful figure in government, who duly welcomed them as "a significant contribution to our understanding of what works in combatting crime." This was a chance finally for the criminal justice system to start to make a difference.

It could make a difference to the communities who suffered from the crimes of prolific offenders: the SEU found that released prisoners were streaming unchanged out of custody and committing something like a million offences a year and that these offenders alone were costing at least £11 billion each year (that did not count the value of what they were stealing or damaging: £11 billion was simply the cost to tax payers of pushing them through this ineffective process of punishment.)

And it could make a difference to the offenders themselves. By the time the SEU report was published, in July 2002, Andrew O'Connor, then aged 29, had been convicted in the courts of Devon, where he lived, on 24 different occasions. Never once had any one of those court hearings done anything which was conceivably likely to stop him offending - because none of them did anything to tackle the causes of his crime. They treated him as though he really were the Daily Mail's caricature of an offender, breaking the law because he was 'a bad person', simply in need of stiff punishment to deter him. Andrew O'Connor was not this caricature: he was a young man whose life had collapsed under him like rotten floorboards.

His father left before he was born and never bothered to come back for him. His stepfather thrashed him with a belt, so the boy used to run away and sleep in the woods for several days at a time. His mother reluctantly agreed to put him into care for a while - never knowing that her much-loved nine-year-old boy would be slippered, caned, bullied and casually used as a sex object by an older boy and one or two members of staff. Then he started going out with other kids, nicking stuff from shops, setting fire to an empty building, stealing cars, taking pills, screwing houses, smoking dope.

By the time he was 12, he was the walking, talking, non-stop-offending embodiment of what the SEU report was describing. Prisoners are 13 times as likely as anybody else to have been in care. They are also 13 times as likely to be jobless, ten times as likely to have played truant at school, six times as likely to be very young parents. Eighty per cent of them cannot write with the skills of an eleven-year-old. More than 60% of them have drug problems. More than 70% of them suffer at least two mental disorders. Andrew O'Connor had all this. The courts, however, just carried on hurting him, as though they had some moral right to pile torment on a ten-year-old, as though they really thought that by doing this, they were protecting the community from his offences.

He was convicted of stealing cars in Plymouth. So they endorsed his licence. That made no difference at all. He didn't have a licence: he was only 12 at the time. He carried on stealing cars. So the courts endorsed his non-existent licence again - when he was 13 and once again when he was 14 and yet again when he was 15. On the last two occasions, they also disqualified him from driving, never minding the fact that he was still too young to qualify to drive in the first place, never wondering why this small boy was earning a living by stealing expensive cars for adults in Plymouth and taking his pay in cannabis and amphetamine.

From the age of ten, he came up in court for burgling. The bench solemnly imposed a care order on him. But he was already in care. That was part of the problem. They fined him too, even though they knew very well that his only source of income to pay the fines was more burgling. When he was 14, they started locking him up - that year they sentenced him to three bouts of custody, amounting to ten out of the ensuing 13 months, even though it was plain that he was coming straight out of custody each time to carrying on with his offending. When he was 18, they solemnly gave him three years probation, never minding the fact that they had already given him two years probation - not once but twice in the previous ten months. He just went ahead and caried on offending.

After the courts failed, the prisons failed too. By the time the SEU report was published, Andrew O'Connor had been released from custody eleven times and he had never received any effective help for any of the problems which made him an offender. Over and over again, he would come out homeless and jobless, bingeing on drugs and alcohol, haunted by personality disorders from his abusive childhood, unregistered with a GP, still unskilled, still unable to read or write like an adult. Each time, he stumbled straight back into crime - thefts, burglaries, assaults. On one release, he was given £10,000 in compensation for the sex abuse he had suffered in care. He blew the lot in weeks, on drugs and generosity to strangers. Sometimes, it took him only a day to get back into trouble; never more than a few months.

Mr Blair's government already knew quite a lot about this and had started to talk more about rehabilitation. Andrew O'Connor had heard the talk and he had believed it. On his last big release before the SEU report, in March 2000, he had written to his legal adviser, Nicki Rensten of the Prisoners' Advice Service: "I've got a good personal officer. He's set up meetings for my release and is in touch with my probation officer. He's guaranteed me I won't just be dumped on the streets like last time. They're working to set up a package for me with the best support I can have.... Between him and another officer, they have phoned the council housing and DSS on my behalf. He says he'll get probation to pull some strings to get me in my own council flat on relase with help with clothing, furniture etc, and help to set myself up working.... I want to live in this lovely place in Devon called Teignmouth.... I think I'll give prison a rest in future. I think this is going to be a good year."

It never happened. Andrew O'Connor was pushed out of the gate of Long Lartin Prison one morning with no address to go to, no job to go to, no GP to look after him and only £46.75 of discharge grant to last him the several weeks it would take him to sign on. None of the help he had been promised materialised. The housing office in Teignmouth said they did not have to house him, because he was not vulnerable enough to qualify. Without an address, he had even less chance of finding a job. The DSS gave him no help with clothing or any of the other basics which he lacked after a four-year prison sentence; his discharge grant soon ran out and, when he eventually received his benefit, he found the minimal weekly payment had been cut even further to make him repay a crisis loan he had taken out six years earlier. His former GP refused to take him back even when he broke his foot. He ended up sleeping in a friend's car where he was arrested and sent back to jail - for failing to tell probation where he was living, even though they themselves had allowed him to be released without an address to go to.

And what the SEU report captured so vividly from its reservoir of research was that prison was not merely failing to stop the offending of people like Andrew O'Connor: it was actively stimulating it. Homelessness was a cause of crime - but 30% of prisoners lost their homes as a direct result of being jailed. Unemployment was a cause of crime - but two thirds of prisoners who were in work, lost their jobs as a result of being jailed. Debt was a cause of crime - but a third of all prisoners saw their debts get worse as a result of being jailed. Broken family links were a cause of crime - but 43% of prisoners lost these links as a result of being jailed. The failure of the system to deliver its objective was stunning. This failure - this routine, ritualised, systemic failure - was why the SEU report mattered so much. But look how the politicians played their hand.

The warning signs were there from the beginning. This was the first and only report from the SEU which had no action plan at the end of it: the government departments who were responsible for housing, employment, education, benefits etc had agreed to support it only if there was no timetable committing them to change. All they would accept were 'recommendations' without any specific policy commitment. The SEU went along with this in the hope that these departments would have to deliver a real commitment once the report was published if only to allow the government to produce its response, which was due six months later at the end of 2002. What happened was very different.

There was no government response after six months. There was to be no response for two long years, because the government departments whose co-operation was vital, were determined not give extra money or priority to a bunch of politically unpopular offenders, no matter how much crime was generated as a result. As one direct witness told us: "The SEU were shafted. These department officials don't really see the 'social dregs' at the bottom of the pool. They have no sense of the link between the decisions they make and the crime that will be committed as a result. There was two years of acrimonious meetings, with these departments sitting on their hands, digging in their heels, blaming each other. It was really disgraceful."

The Home Office set up a team to implement the report. It was considerably smaller and far less senior in its leadership than teams which had been set up for other comparable issues. The Treasury was interested in saving money by turning people away from prison, but it was not prepared to take any political risks, so Treasury officials backed the SEU's plans for more education and job advice for prisoners - they were politically easy to sell - but they would give nothing for the other causes of crime. Overwhelmingly, the Treasury's investment stayed with prisons, even though the SEU had found so much evidence of their failure - even though the Home Office team came up with their own cost-benefit analysis to show how, in the long term, investment would save public money.

Other government departments were even worse. The small Home Office team spent months trying to secure housing for released prisoners, waving the SEU evidence that a stable home cut re-offending by 20%. All they wanted was for the Office of the Deputy Prime Minister to tell local authorities to treat more prisoners as 'vulnerable' and, therefore, entitled to housing. The ODPM would not have it. They said prisoners' housing was a job for the prison service. The prison service said that was fine but they didn't have any housing. The ODPM said it was no good coming back to them, because the law would not allow them to treat prisoners as 'vulnerable'. One of those involved said this was "nonsense - they just didn't want to do anything". The ODPM got their way: even if it did mean more crime, there was no effective priority on housing for released prisoners.

And there was no help for the thousands of prisoners each year who rented homes and then lost them when they were jailed because they could not get housing benefit to pay the rent. A relatively new rule meant that those who did manage to get housing benefit lost it immediately if it looked as though they were going to spend more than 13 weeks behind bars. Before the report was published, the SEU had persuaded the Department for Work and Pensions to extend this to 26 weeks, but then Andrew Smith had become Secretary of State for Work and Pensions and ruled that single men were not a priority for his department and stopped the whole move. Now, the Home Office team tried to tackle some local authorities who were refusing to pay prisoners any housing benefit at all on the grounds that if they were jailed, it was their own fault, so in the eyes of the law they were intentionally homeless and barred from benefit. The ODPM variously refused to admit that the problem existed and/or declined to solve it.

One after another, the recommendations of the SEU report were killed off: the prison service refused to increase the discharge grant; the Department for Work and Pensions refused to pay prisoners' benefits quicker. At one point the DWP agreed to grant released prisoners immediate crisis loans to stop them running out of money while they waited for benefit, but then they reneged on the deal. There was nothing to make sure that released prisoners had GPs to go to. Nothing at all to help prisoners with alcohol problems, just a strategy document without any resources (a decision which was disguised with classic Whitehall elegance as "building on good practice within existing resources").

The Home Office team tried to re-unite imprisoned mothers with their children who had been taken into care. Nothing. They tried to help the 125,000 children a year who lose a parent to prison - children who subsequently prove to be three times more likely than their peers to have mental health problems. Nobody was interested. Two Whitehall officials sat through a meeting where they refused to make prisoners' children a higher priority for the Sure Start programme and walked away declaring quite audibly: "Well, that was a load of old bollocks, wasn't it?"

The Prison Service itself was asked repeatedly to require all governors to assess the needs of all new prisoners so that their problems were at least known. "That was thrown out with real scorn," according to one witness. "This was a key thing - to know what problems they were dealing with, so they had a chance to solve them - and they just refused point blank."

The government did not reject everything. They agreed to set up Jobcentre Plus advisors in prisons and to fund new programmes to link drug treatment inside prison to agencies on the outside. They transferred prisoners' healthcare to the Department of Health with Inreach teams to work with the most mentally disordered on prison wings. But these moves were politically unavoidable. The fact remained that when the government had the copper-bottomed evidence they needed to attack re-offending, they had chosen to be dishonest about crime and dishonest about the causes of crime.

The civil servant who had headed the Home Office team, Julian Corner, subsequently resigned and wrote a little-noticed article in Safer Society, the magazine of the prison rehabilitation group Nacro. Corner recalled the strength of the evidence in the SEU's report, which he had helped to write: "The premise was that seven government departments would go on to forge a united front against reoffending, and prison would only be used as a last resort. What actually happened was that they cherry-picked the most politically acceptable and convenient actions, and rubbished the rest - namely, the social inclusion measures."

But all this left the government with a problem: they still had to publish a response to the SEU's report. How could they justify rejecting so much evidence of what really stopped reoffending? On the face of it, the Prime Minister who personally had commissioned and welcomed the report should himself hold a press conference to respond to it. Or perhaps one or two of the relevant ministers would take on the job. But they did nothing like that. The response, when it came, was brilliant.

On Monday July 19 last year, the Prime Minister went to a community centre in north London and made a speech about crime in which he blamed our problems with law and order on what he called the 1960s liberal consensus. It is fair to say that this was an idea which was unsupported by any criminological research anywhere on the planet.

That speech was leaked in advance to half-a-dozen newspapers, including The Guardian, all of whom ran it prominently on that Monday morning. During the day, the Home Office put out details of future plans for attacking crime - on-the-spot fines for children, electronic tagging, satellite tracking. This ensured that the speech enjoyed another round of reports on the Tuesday morning. The Daily Mail's headline said: "Blair the crimefighter (and about time too)." Then the columnists and the leaderwriters joined in. The alleged link between the historic rise in crime and 1960s liberalism was still being reported and discussed in the Sunday papers at the end of the week.

And on that Monday afternoon, while all the crime specialists and all the leader writers and the columnists were chasing the decoy of a speech which was unsupported by a single grain of criminological evidence, in the midst of that, without any press conference or press briefing or statement in the House, the government quietly placed their response to the SEU report on the Home Office website. And while the Prime Minister's speech received tens of thousands of words of coverage, spread over a whole week, their response to the SEU's vital research work received.... not one single word.

In the same Safer Society article, Julian Corner reviewed the response which was presented as 'a national action plan' with 60 action points: "There's a commitment to ‘mapping and analysing existing housing and service provision'. Has it actually taken all that time to build the resolve to map provision? How about ‘developing a strategy for a more coherent information and advice service'? Isn't this the work needed to produce the action point? Then there's ‘developing a timetable to introduce a measure' - a new definition of action, surely. And ‘work will examine the possibility of working with a particular region to establish best practice'. Beyond parody. These are, in reality, 60 inaction points."

The 60 points did not even mention the particular problems of prisoners on remand, or on short sentences, or of women prisoners or young prisoners or those from ethnic minorities. Corner continued: "We had, after the SEU report, the near ideal circumstances for putting into place the social care framework that might, conceivably, have made imprisonment compatible with the reducing reoffending agenda. But ministers and officials lacked the resolve to follow through on the report's recommendations." And nobody noticed. The government had hacked the SEU's work to bits and now they had quietly buried its remains.

By now, of course, Andrew O'Connor was dead. While the government had been sitting on the evidence of the failure of prisons, the numbers being jailed had carried on climbing and Andrew O'Connor had been behind bars again. During the two years when government departments were busy blocking most of the SEU's findings, he had managed three months at liberty, ended up with a crack habit and a bed in the Salvation Army hostel in Devonport, Plymouth. One night, he walked into a snooker club and asked for a drink. They told him he would have to pay £10 for membership. He told them he would pay £5 this week and £5 next. The bar staff told him to get lost. He told them to give him a drink or he would take everybody in the bar hostage. He was unarmed, but the club called the police and he ended up doing 16 months for affray. When he came out of that sentence, he lasted one day before being sent back for breaching his licence: they said he had stumbled drunk into a probation hostel; he said he was having an epileptic fit.

He spent another four months in prison, in Exeter. He did a course in enhanced thinking. When they asked him for his objectives, he wrote "No objectives - you won't get anywhere." He told them: "I never had a proper childhood so am unsure a lot of the time how I should respond or even if what I am thinking or feeling is appropriate for the situation." He passed the course.

He filled in a form at the prison: "Paramount, I need structured care plan to make sure of good recovery. I would like to achieve outside counselling for the root causes of my problems, so the right support and right environment, ie around my family, with intense counselling, this would be the best plan for me." He had a big reconcilation with his mother, who said he could come home to live with her. He wrote to a friend: "My mum is taking three months off work to love and support me, take me to meetings and appointments, help me fill out forms, take me places and re-integrate me into society generally."

As usual, it never happened. There was a meeting in the prison to discuss his release. His probation officer failed to turn up. The prison workers agreed that he should go to his mother's with help from a local agency to deal with his drug habit. But the probation officer then over-ruled this and said he must go to a hostel miles away from her, in Cornwall. The hostel housed sex offenders and, recalling his past abuse, Andrew O'Connor refused to go there and, in protest, then refused to work with the local drug agency. His mother spent the three weeks before his release phoning the police and probation and the prison, warning them that Andrew would not go to the hostel and must come to her. Most of her calls went unreturned.

On Thursday October 23 2003, he walked out of prison. On Wednesday October 29 at 7.30 in the morning, at the Peppermint Park caravan site in Dawlish, south Devon, he was found dead. The pathologist said he had had an epileptic fit in the night. If he had been at home, if he had had a GP, if he had had a life, maybe it would have been different. As it was, he was living a half life. He was fully clothed. His mother said he must have slept that way so he could run away if the police came to get him. She wrote afterwards: "Andrew was a victim before he was born and continued to be a victim. He wanted to come home. In the last twenty years, he never had his own bed. It breaks my heart and fills me with so much pain to know one of my children that I loved, suffered so much for the biggest part of his life, because of my mistakes, to let him go into care, and the mistakes of a system that was supposed to care for him. Why was it so hard to get any real help? Is it easier to just lock someone up to keep them off the streets?"

The SEU's work was not entirely wasted. Its research has become a reference point. There is now a Reducing Reoffending Committee in Whitehall where senior officials carry forward the national action plan which eventually emerged. There are signs of new life - some new money for housing advice, a pilot project in the South West to help released prisoners find homes, "invest to save" money for working on family links. And yet the central thrust remains diverted, away from an effective attack on the causes of crime, towards the acquisition of political power through the infliction of hardship.

The Guardian recently re-published a letter from a former nurse who had started to work with young offenders in Manchester: "It is a complete non-sequitur that because a boy stole your watch, he should be supported on your rates in jail, perhaps for life... The punishment of jail is not a deterrent.. 'Punishment' is perhaps not a word in God's vocabulary at all and, if so, ought not to be in ours." The letter was written in 1890, by Florence Nightingale. Maybe nothing really changes.

Additional research by Roxanne Escobales