Foreign affairs Minister Winston Peters says migrant workers shouldn't get extra support at the tax payers' expense.
Unemployed migrant workers caught up amid the Covid-19 crisis have reported they're still not getting enough help from authorities here or in their home countries.
However, Peters has said even though the government had already had asked foreign nationals to leave, some decided not to.
He said the government helped arrange for 50,000 people to return home.
"We're talking about people who having lost their job, their legal status to be here expired and they had to get on a plane," he said.
When Peters was questioned whether migrant workers were indeed tax payers, he said that was an extraordinary statement.
"Are they paying for the full responsibilities of education or dare I say it the media bail out of $50 million or all those other things? It just doesn't wash," Peters said.
"With all due respect why don't you tell the taxpayers they are going to be paying for half the world as well."
Everyone was required to pay tax, but that did not mean people got greater rights than they did on the day they arrived in New Zealand, Peters said.
He said many foreign nationals had already been given substantial support through Civil Defence and charities.
Prime Minister Jacinda Ardern said a compassionate approach needed to be given to those who found themselves in a circumstance where they were unable to make any alternative arrangement.
"Be it a tourist who may have found themselves here, be it someone on a working holiday visa, be it someone who had a work visa, that is why we have sought to support them," she said.
She said the government had offered both food parcels and accommodation support.
"We did use that in the initial response because it was fast, it got support out the door and it had flexibility, I won't deny we will need a longer term framework, but for now it has been a very important part of our response."
However, Ardern would not say whether there was any potential benefit on the horizon.
Read more about the Covid-19 coronavirus:
- See all RNZ Covid-19 news
- A timeline: How the coronavirus started, spread and stalled life in New Zealand
- Your Covid-19 questions answered - from health and employment to managing anxiety
- Covid-19 symptoms: What they are and how they make you feel
- Touching your Face: Why do we do it and how to stop
- Scientific hand-washing advice to avoid infection
- Coronavirus: A glossary of terms
- Sign up for the Covid-19 newsletter for the big news - and some things to make you smile
- The Coronavirus Podcast