Papua New Guinea police have taken four men from one of the refugee facilities on Manus Island this morning with what is thought to be the intention to deport them.
The journalist and refugee Behrouz Boochani said the men from Bangladesh were not granted refugee status.
The men had been given the option to be repatriated voluntarily but remained in Australian funded detention for about five years.
The refugee determination process was flawed, Mr Boochani said.
"They processed people under too much pressure and very harsh conditions," he said.
"Why they didn't deport them five years ago? Why they kept them in prison and tortured them for years and years and now they want to deport them?"
Five other asylum seekers were taken by police last week and were now being held in Port Moresby's Bomana prison, he said.
Manus Island police could not be reached for comment.
Amnesty International researcher Kate Schuetze said there might be an attempt to deport the nine men, "even though they haven't had a chance to have their asylum claims fairly processed".
"We know that there are gaps in PNG law in that people aren't allowed the opportunity for judicial review once a decision has been made and that's the kind of process you'd ordinarily expect to see in Australia or New Zealand if they had a fair claim," Ms Schuetze said.
"You don't spend five years locked up in prison for no reason... They need to be given the opportunity to appeal or review those [refugee status] decisions."
By deporting the men both Australia and PNG risked breaching their international obligations on non-refoulement, or not returning people to danger, she said.
The UN refugee agency, UNHCR, knew refugee status had not been fairly determined on Manus and that deportations would be illegal, Mr Boochani said.
In January, the UNHCR said 500 people required a review of their refugee status determination process by authorities in Papua New Guinea and Nauru.
About 2000 people remain in exile in both countries where they were detained by Australia for traveling there by boat to seek asylum.