To start thinking about a solution to this difficult problem, it seems to me we have to break it down a little. This problem seems to have a number of different aspects:

1. There are too many people going into the prison system. As I wrote in the topic above, a staggering percentage of people in the U.S. get sent to prison. One way to address the overall problem would be to cut down on the number of people initially going into the prison system. This might happen through the kinds of things JillGStephens mentions -- "strengthening our education system, the family and the community." Another strategy might be to repeal the sentencing guidelines for some "victimless" activities -- such as drug use -- or even decriminalize them altogether, so that the prison system was reserved for people who represent a direct threat to others.
2. The prison system environment does not rehabilitate those within it. When 80% of those leaving are going to be sent back, often for more serious offenses, there is obviously something broken. Both JillGStephens and SRogers2007 point out some of the corrupting influences that inmates are subjected to. So one way to address the prison problem would be to reform the way they are run, in order to make the environment conducive to rehabilitation.
3. Neither the prison system nor the society at large provide ex-convicts with enough help to get on a good track once they are released. In addition to being woefully prepared for a post-prison life, there are too few resources available for released prisoners to rely on in trying to establish productive law-abiding lives. Solutions that target this aspect of the problem might call for programs designed to help ex-convicts start new lives, perhaps in a new environment, with job placement help, etc.

Of course, some solutions might address several of these aspects. For example, what if undereducated prisoners were helped to get their high school equivalency while locked up, including college preparatory work? The government could sponsor college scholarships for prisoners who show some commitment, so that when they come out they are subsidized to get an education. Think of it like a GI Bill for ex-cons. The cost of such a program would very likely be offset by the corresponding decrease in recidivism -- it costs an average of $12,796 per year for tuition and room and board at a public college (College Board), while it costs an average of $24,440 per year to incarcerate someone in a federal prison (Cryptome). So every year that the government sponsors an ex-con to go to college instead of to prison, we the taxpayers save a bundle. And of course the ex-con we sponsor to get a degree is very likely to become a productive contributor to society, while the ex-con we don't is likely to keep going back to prison, costing us huge sums forever.

Maybe I should post this as a solution? What do you think?