The city council suggested the $10 entry fee as a way to bring in cash to develop the gardens, but many people opposed the idea or said the fee was too steep.
Friends of Hamilton Gardens president Virginia Graham told Morning Report that charging out-of-towners could be expensive to set up, and a risk to the decades of loyalty and support from locals.
"The development of the gardens has come mainly from huge support from the community," Ms Graham said. "There's a great sense of ownership within this Waikato region for the gardens ... we don't want to lose that."
"[The community is] saying 'Look, we've worked really hard over the last 30 years to build these gardens and now you're going to charge us. If you're not going to charge us, you're certainly going to charge people who live on the fringes of Hamilton, who are not ratepayers but still identify as Hamiltonians'.
"They would have to pay and they're feeling really resentful about this."
Ms Graham said it was a catch-22 situation, because if it was then decided that more locals from nearby regions would be exempted from the proposed fee then it would have a knock-on effect on the income.
Alongside the concern of differentiating the status of visitors from locals was the amount of investment needed to implement the proposed fee, because of the garden's unique entrance.
"It's not easy. There's talk of expanding the information centre at the cost of around $900,000 and yet independent reviews have questioned whether that sort of investment would bring the returns from the charges that is expected," Ms Graham said.
She said she looked forward to more consultation around the process and its implementation with the council.
Hamilton Mayor Andrew King said the gardens were one of the treasures of the city, but they came at a cost.
"The question is whether the significant costs to further develop the Gardens and around $2 million-plus a year to maintain them, should continue to be largely met by our ratepayers, or whether some of this investment should come from the thousands of visitors to the city."