The Minister responsible for Ministerial Services, Chris Hipkins, says overpayments to the Prime Minister and her deputy should have been picked up earlier.
Jacinda Ardern and the Winston Peters have been overpaid just over $12,000 and $9000 respectively by the Department of Internal Affairs.
The payments were 'out-of-town' allowances MPs receive for accommodation in Wellington, but as ministers, Ms Ardern and Mr Peters are not entitled to it.
The money was paid between late October and early February, and both have since reimbursed it and have received apologies from the ministry.
Mr Hipkins, told Morning Report it was unacceptable and should have been noticed earlier.
"I've expressed my disappointment to [officials]. I think they have let the ministers in question down.
"I think people should be able to trust that the paycheck that they receive is the paycheck that they're entitled."
Ministerial Services knew where they were living and should have adjusted the salaries accordingly, he said.
Mr Hipkins said Ms Ardern and Mr Peters may not have noticed the increase in their salary because they both got pay rises after taking the one and two spots in the government.
"I don't expect them to pull out the calculator and manually calculate whether they're getting the correct amount, I expect that that's the job of Ministerial Services and they clearly didn't do that properly."
He said Ministerial Services told him there were no other payment issues for other ministers.
A review would be undertaken, Mr Hipkins said.
But he has also asked for a more general inquiry into the way Ministerial Services handled the change of government, with delays in recruiting staff, arranging emails and issues with shifting offices.
'What happened to their responsibility' - Seymour
ACT Party leader David Seymour said he wondered how Ms Ardern and Mr Peters did not notice they were receiving allowances they were not entitled to.
He told Morning Report it was a large amount going into their personal accounts.
"It's a little bit too cute that the Ministerial Services has apologised to them, what happened to their responsibility."
People make mistakes but Mr Peters had a similar situation with superannuation payments that came to light last year, Mr Seymour said.