Police are investigating an allegation against National Party MP Richard Worth, who announced his resignation as a minister on Wednesday.
Prime Minister John Key says the police investigation is a criminal one and is related to a relatively recent incident in Dr Worth's capacity as a private individual.
Neither Mr Key nor Dr Worth would comment on the specifics of the allegation on Wednesday.
However, Mr Key confirmed that he has looked into other allegations of a "similar type of nature" as the one now facing Dr Worth, who was Minister of Internal Affairs and held other portfolios.
The Prime Minister was asked if he had heard about Dr Worth making a "nuisance of himself" towards women.
Mr Key said he had investigated previous allegations about inappropriate behaviour and was satisfied with Dr Worth's responses, but that the current circumstances were different.
Mr Key told a media conference on Wednesday morning he was made aware of a situation by a third party almost a week ago, after which he approached Dr Worth, who tendered his resignation on Tuesday night.
If he had not offered to resign, Mr Key said he would have sacked him.
"What I can say is, irrelevant of what may occur, I no longer have confidence in him," Mr Key said. "I just no longer have confidence in him for a number of reasons."
Mr Key defended the speed with which he has dealt with the allegation, saying he acted as fast as he could, under the circumstances.
Police have released a statement saying an allegation has been made against a sitting Member of Parliament and Wellington police are conducting a preliminary investigation into the matter.
Police confirmed on Wednesday a complaint was made in Auckland two weeks ago regarding a recent alleged incident involving Dr Worth.
In a statement, Dr Worth said he was resigning for personal reasons.
Dr Worth has been granted two weeks' leave of absence from the House, which Mr Key said he should use to consider whether he wants to stay on as an MP.
"All I can tell you is his conduct does not befit a minister and I will not have him in my Cabinet. That is a different test to whether he ... sets the standard to be a Member of Parliament."
Controversy over India trip
In April, Dr Worth came under scrutiny over perceived conflicts of interest, which earned a reprimand from Mr Key and sparked calls from the Labour Party for his sacking.
Dr Worth had travelled to India as a private citizen, but spoke in his role as a Government minister, outlining the advantages of aviation training in New Zealand.
At the time, he was a shareholder and director of the company New Zealand Aviation.
Dr Worth was also the minister for the National Library and for Land Information. His portfolios are temporarily being handled by Minister for Building and Construction, Maurice Williamson.
Labour wants answers
Labour Party leader Phil Goff says Dr Worth should have been sacked at the time of his business trip to India, but says the Prime Minister lacked the necessary leadership to do so.
Mr Goff says the nature of the allegation against Dr Worth should be revealed.
He says if someone is in public office, they cannot hide behind secrecy and non-disclosure of reasons for their resignation.