The National Party says its confidence in the Speaker of the House has been "severely shaken" because of recent events.
Its deputy leader, Paula Bennett, stormed out of the debating chamber in Parliament this afternoon in frustration over the Speaker's behaviour.
Since becoming Speaker, Labour's Trevor Mallard has introduced a new system of discipline - deducting questions from MPs who interrupt proceedings.
Shadow Leader of the House Gerry Brownlee has written to him expressing National's concerns.
Stuff has reported that Mr Mallard as confirming he heard a National MP call the Prime Minister a "stupid little girl" in Parliament earlier this month, although the comment cannot be heard on the recording of question time.
Mr Brownlee said "promoting stories about events which cannot be verified" is unacceptable for the Speaker, whose job it is to "take a neutral role" and make sure Parliament runs effectively.
And he took umbrage with Mr Mallard calling an MP a "naughty child" and allowing another to be called a "fool" without intervening.
National also had concerns about the "arbitrary way" Mr Mallard was running Parliament saying it was "seriously limiting" its ability to hold the government to account, Mr Brownlee's letter said.
This afternoon Mr Mallard rejected that accusation, telling National MPs that under his "disciplinary" system their party had received an extra 22 supplementary questions so far this term.