Prime Minister John Key says claims the head of the Serious Fraud Office took a bottle of wine that did not belong to him is a storm in a champagne flute.
Comments by Adam Feeley in a leaked email have been referred to the State Services Commission by the minister responsible for the office.
It is claimed SFO staff celebrated progress in a case against Bridgecorp Finance by drinking a bottle of the company's champagne.
Mr Key says he does not think it is a major issue.
"It's probably a little unfortunate but, as (Finance Minister) Bill English said there's been a lot on their plate and they've been working through a lot and it's probably a boss trying to reward the staff a little bit.
"If they had stolen it, I think that would be a different issue, and that's a matter for the State Services Commission ... but my understanding is it was left behind."
Labour Party leader Phil Goff says it would be extraordinarily unwise for Mr Feeley to take a bottle of wine that did not belong to him and if the allegations are true it is a worry.
Mr Goff says it would be inappropriate for anybody in a law enforcement position to take possession of something they did not own.