The aid agency Doctors Without Borders says it is still being denied access to transit centres on Manus Island to treat sick and injured refugees.
The agency's Australian president, Stewart Condon, said he was concerned for the health of men forcibly removed from the island's decommissioned detention centre last week.
He said the Papua New Guinea government granted his team permission to enter the centres but security contractors would not let them in.
"We're very keen to see where the men are living so we can understand their health as well as what's impacting on their health," said Dr Condon.
His initial concern for refugees suffering from chronic diseases was compounded, he said, by reports of men being beaten during the removal.
"We know that some of them are injured. When I left on the weekend we were hearing about a number of them who are significantly injured. So we are still talking to all the authorities we can, trying to get inside."
Dr Condon said he had posted two doctors in the island's hospital to see men from the transit centres, but he said some refugees were too scared to come out.
"We are worried that there are men that we are not able to reach and men we are not able to treat."
The Kurdish journalist and refugee Behrouz Boochani said many disturbed and institutionalised men were too afraid to leave the centres.
He said about 20 refugees each day had been going to the hospital.
"There are so many people who have been injured and also so many people traumatised by the incident," he said.
"There are only doctors here from the MSF (Doctors Without Borders) and it's not enough. We need some psychologists, more medicine and a specialist."
A doctor from a private health company works at one of the transit centres, but Mr Boochani said the men did not trust the company after the death of refugees in its care.
Australia's reputation damaged
Prominent Australian doctors say their country's reputation has been damaged by inadequate healthcare for the refugees.
In an open letter to the Australian government, 18 senior physicians called for assistance to go to Papua New Guinea to review the health of Manus Island refugees.
The group's spokesperson, Laureate Professor Nick Talley, said the doctors believed harm had been done to Australia's international standing.
"The government may have a very different view about this but certainly one could argue that Australia's international reputation has been damaged by what appears to be inadequate healthcare being delivered," he said.
Professor Talley said for years the Australian government had ignored similar calls from medical bodies to improve healthcare for refugees detained on Manus and Nauru.
"We have an obligation as a nation to ensure appropriate and best health," he said.