Wellingborough Prison will close by the end of this year with the loss of almost 600 prison places, Kenneth Clarke has said.
The Justice Secretary said closing the category C jail in Northamptonshire will save £10 million per year.
The prison population in England and Wales was 86,652 last Friday, with enough capacity for a further 3,500 inmates, the Prison Service said.
Along with the annual saving, closing the jail will also avoid costs of up to £50 million which would have been necessary to maintain the longer term viability of the prison, the Prison Service said.
It is expected staff will move to other jails and compulsory redundancies are not expected.
Mr Clarke said: "The public has the right to expect continuing improvement in the quality and efficiency of public services, without compromising public safety.
"Closing outdated and expensive prisons is an important step in our strategy to deliver a fit-for-purpose, modern custodial estate that can provide high quality, cost-effective and secure regimes that protect the public and reform prisoners. Closing this one prison alone will save millions of pounds for the taxpayer."
He went on: "The changes will reduce our current capacity by 588 places and I am confident that they can be safely managed within existing headroom, whilst maintaining our ability to cope with any potential increase in population."
The Prison Service added that jails will only close when capacity allows, saying: "We will always ensure that there are sufficient places for those offenders sentenced to custody by the courts, including a margin to manage fluctuations in the prison population."
Two new prisons - the G4S-run Oakwood jail near Wolverhampton and the Serco-run Thameside prison in south-east London - opened earlier this year, creating about 2,500 extra places.