Social Development Minister Paula Bennett says she is mortified that private information was accessed through public computer kiosks at Work and Income offices.
The Ministry of Social Development is calling in an independent security expert to investigate the breach.
Information about children in the care of Child, Youth and Family, beneficiaries, and contractors employed by the ministry has been freely accessible through the kiosks.
Freelance journalist and blogger Keith Ng has revealed he was able to download 7000 documents from the ministry's network through kiosks set up for job-seekers in two offices.
Mr Ng said the documents were just a fraction of those available. They included invoices detailing the medical conditions of children in state care, the names of people being investigated for benefit fraud and pay rates for individual ministry contractors.
He said he did not have to hack into the system and no passwords or special commands were needed. He then informed the Office of the Privacy Commissioner.
The ministry has begun an investigation after Radio New Zealand informed it of the breach on Sunday night.
Paula Bennett said on Monday the breach should not have happened and the ministry now has to make sure that it can assure the public of the integrity of the system.
She said she would make sure a database of vulnerable children that is being set up will have every check and balance put in around it.
"I apologise to everyone now - I'm mortified that they had that level of trust in the ministry and at some level we've let them down. We have to now take a step forward, really make sure that the review is robust, learn from it and move forward."
The Social Development Ministry said it hoped to find out if anyone else has exploited a weakness in the security of the kiosks.
Chief executive Brendan Boyle told Radio New Zealand's Checkpoint programme on Monday the ministry is not yet sure if anyone else has exploited the method used by Keith Ng, but has hired an independent expert to find out.
Mr Boyle said it is investigating whether computer security company Dimension Data should have discovered the weakness when it was hired to check the security of the kiosks in April 2011. The company reported no problems at the time.
He said the ministry received a phone call last week referring to security concerns and it immediately began testing its website for security flaws, but did not check the kiosks until it was specifically told by Mr Ng where the breach was.
Kiosks throughout the country have been closed and the ministry said they would not be reopened until it gets clearance from the security expert.
The terms of reference for the inquiry would be released by Wednesday and Mr Boyle expected an interim report in a fortnight.
The Green Party said the privacy breach is symptomatic of a ministry with low regard for client privacy. New Zealand First said there is a risk that people will stop trusting government agencies.
Ministry told of problem a year ago - federation
Privacy law specialist Kathryn Dalziel says the Social Development Ministry needs to undergo a thorough investigation from the top down.
Ms Dalziel told Radio New Zealand's Morning Report programme on Monday that privacy needs to become part of its culture.
The Beneficiary Advocacy Federation says the ministry was told a year ago that there was a security problem with public computer kiosks at Work and Income offices.
Spokesperson Kay Brereton told Radio New Zealand's Nine to Noon programme that she discovered what information could be accessed and told officials.