Grey Power want the SuperGold Card to be upgraded to a smartcard and an official form of identification, a move it believes will prevent fraud.
But the government is hesitant to make such a swift move because it could make it harder to access for senior citizens.
Gold cards are being stolen, borrowed or counterfeited in order for people to access free public transport and other benefits, such as cheaper petrol.
The Ministry of Social Development (MSD) has been alerted to three cases since 2012, but RNZ knows of four people in the Wellington region who have used free public transport in 2018 alone, using stolen or counterfeit cards.
The card has not received any form of face lift since it was introduced in 2006.
It comes with a magnetic strip and the user's name, but no other form of identification. Adding a photograph is optional.
People have been taking advantage because of the way the card is set up, and the ministry has no way of tracking the cards.
Grey Power national treasurer Roy Reid said the advocacy group had concerns about gold card fraud, and believed it was overdue that it was made an official ID.
"Grey Power has been pushing for quite a number of years to have a person's photograph on the gold card so that then becomes a legal form of identification."
As of June 2018, 733,088 gold cards were in circulation. That number is expected to rise past one million in the next decade.
Just 3.5 percent of those people have a photo on their card.
According to MSD figures, 20 percent of gold card holders have no driver license and fewer still have a passport.
While the government is aware of the numbers, Minister for Seniors Tracey Martin said it would be quite a leap to make the gold card an official ID.
"That would be quite a significant shift in the card itself.
"What's coming back to me is that they really value having the card, top of their list would be around transport discounts.
"If and when we smarten if up, there would have to be, because of levels of security, we'd have to change legislation for the first instance, we would have to deal with a lot of levels around security and privacy."
The Labour Party and New Zealand First have committed to introducing a new generation of the SuperGold Card as part of their coalition agreement.
They have talked about introducing a smartcard, which could feature a microchip and the user's photograph, and MSD said that would cut cases of fraud.
Ms Martin said she had heard stories of gold card fraud, but said she had no concrete evidence of it happening.
"I think fraud is a strong word to use. I would probably term it misuse.
"Anecdotally, up in my area, it's been around somebody that people have believed to be getting on a bus when they shouldn't have been getting on a bus."
A 2016 report into gold card fraud prevention estimated the roll out of a smartcard to users would cost up to $6 million.