If the prison population continues to spiral, the government will need to build a new prison every two or three years, the Justice Minister says.
Andrew Little has initiated a review of bail, sentencing and parole laws, and better rehabilitation of prisoners.
Mr Little told Newshub Nation that the policy followed over the past 30 years could not continue.
"On the current trajectory on prison populational growth, if we did nothing we would be building an extra prison every two to three years. That's how bad it is."
Currently 60 percent of prison inmates re-offend within two years of being released from jail.
Mr Little said he wanted to start a public conversation about ways to reduce re-offending.
"That 60 percent re-offending figure - that is a mark of failure of 30 years of criminal justice policy that says 'we'll lock more people up and we'll lock them up for longer'."
Mr Little said the government wanted to reduce re-offending and thereby reduce the number of victims of crime.
Meanwhile, the Prime Minister said she expected New Zealand First would sign up to reform of the justice system.
Last week New Zealand First refused to support abolishing the three strikes law, which requires judges to sentence repeat offenders to long prison sentences.
Ms Ardern said the government as a whole supported reform.
But she told Newshub Nation it also had to get public support for change.
"We need to bring people with us - that's the whole point. If you end up being a one term government as a consequence of changes you have made, you probably haven't brought people on that journey.
"The pitch that we're making is when we have a static crime rate...but an ever increasing prison population, is that the kind of country we want to be?"