Sources at Waikeria have told RNZ the remaining 17 men are protesting about dilapidated conditions, cramped cells, and a lack of basic supplies.
One source said there were still men atop the roof at the old part of the prison known as the top jail.
They said the men are making a stand against the conditions they say have been experienced at Waikeria over the last year.
"Our water comes out brown, but we still have to drink it," one source communicated to RNZ.
A source claims there is a lack of basic supplies, and health and well-being treatment, at the prison and it has taken a toll on mental health of those imprisoned.
"We sit and eat in the same room we sleep in, we get towel and clothes changes once a week, clothes that don't even fit."
"We don't get blanket changes, we don't get blanket washes, I've been in here a month and I haven't had one, my uncle's been in here six months and he hasn't had a blanket/sheet change."
"The only time they ever want to move us is when they are going to lock us in a cell or take us to the yards."
For those inside Waikeria, the conditions had become too much, the source said.
Negotiations have been ongoing and the source said those still remaining had asked for water and were told to come down from the building one at a time.
The source said the men did not want to surrender at gunpoint and had expressed wanting a peaceful outcome.
The individuals - according to sources - are instead asking that political leaders and some media organisations intervene and help mediate the evolving situation.
'There is no programme'
One source communicated to RNZ about how there had been continuous systemic abuse within the justice system.
"We hear about this Hokai Rangi strategy but we don't see it. There is no programme. There is no rehabilitation. It's 'lock us up, put you in a yard full of gang members and then let you out'. And they expect us to change."
Whānau have also spoken out about why some had reached breaking point.
A partner of one of those incarcerated said after receiving an alarming call late on Tuesday night she learnt that those inside Waikeria were "quite afraid" despite the evacuations.
The family said "everything was burning except for the concrete".
The family said their relatives inside Waikeria had told them sometimes they waited for up to four days for toilet paper and restrictions around confinement had meant inmates had spent up to 23 hours per day in a cage-like cell known as "the hole" with limited natural light.
One source said inmates were double-bunking as well as sharing a toilet within the same cell where they ate and slept.
A family member said those involved in the standoff were unarmed.
Fears armed offenders squad is on standby
She was particularly worried about the well-being of her partner, who is a father, and also the other 16 who were still non-compliant, after hearing that the armed offenders squad were reportedly "getting ready to roll in".
The family said the mental well-being and health of those inside has been deteriorating due to the strict conditions and the age-old facilities, which had already been criticised and parts of the complex deemed "no longer fit for purpose" after a report by the Office of the Ombudsman.
Corrections unaware of complaints
Corrections, in a statement, said the department was not aware of any complaints relating to prisoners not having access to basic toiletry items such as toilet paper, and "we would be concerned if that was the case".
A spokesperson said: "All prisoners are entitled to receive a standard of health care reasonably equivalent to that found in the community.
"Medical Officers working in prisons assess prisoners to determine whether primary, secondary or tertiary level intervention or treatment is appropriate. Primary health care is delivered by Corrections, and includes services such as general practice, prescriptions, and nursing services. Primary mental health care is also included.
"Corrections is responsible for ensuring the secure and safe operation of our facilities at all times.
"National policy is that all units must operate an unlock regime that reflects the working and rehabilitation activities of the unit. Unlock hours vary between different units within a prison and each prison site, and can be reflective of an individual prisoner's management plan. They are also subject to change in order to meet operational requirements."
The spokesperson said prisoners may be denied association with other inmates if their behaviour presented a risk to the security of the prison, the safety of others, or themselves.
In a further response to RNZ's questions, a Corrections spokesperson said in a statement: "This incident remains ongoing. The focus of our staff is on ensuring that it is resolved safely.
"We have highly trained staff with specialist skills who are continuing to negotiate with the prisoners involved. As our Chief Executive stated earlier, we are not going to speculate on matters that are relevant to those negotiations.
"A review into the incident has been commissioned and the reasons for the prisoners' actions will be considered - various things have been mentioned in media reports."
Earlier today the Corrections Department said there had been significant damage done to the top jail and it was unlikely inmates would be housed there again.