The X Factor: Grok deepfakes and why NZ is still using Elon Musk's X

Explainer - Elon Musk's X has been in the line of fire over Grok AI deepfakes - with Prime Minister Christopher Luxon's office calling them "concerning."

What's going on? And is there any role New Zealand should play in how it uses X to disseminate information?

Recent non-consensual adult content created by X's AI chatbot Grok has led some to call for a boycott of using X.

X announced on Thursday it would make changes to its AI chatbot after the UK joined many countries in planning to investigate the company.

However, the reputational damage X has been accumulating during Musk's takeover of the company may be lasting.

It's still a popular forum for many New Zealand politicians and public agencies to use, but some left-leaning political parties have mostly abandoned it. A human rights group has called for greater regulation of X in New Zealand, while the Free Speech Union is calling that censorship.

Here's what's got people concerned about X and Grok, and where New Zealand politicians stand on it.

What is Grok AI doing exactly?

Formerly Twitter, the social media platform X was bought by the world's richest man Musk in 2022.

X launched its AI chatbot Grok in late 2023, and recently it came out that users could edit an image to create deepfake nude or partially clothed images.

At some points earlier in January, hundreds - perhaps thousands - of requests were coming in to change photos of women to put them in bikinis and other suggestive poses every hour, separate analyses by Bloomberg News, Reuters and The Guardian all found.

The company soon said it limited the image function to paid subscribers, although NBC News and others have reported that the standalone app and the Grok section of X is still able to generate the deepfake images.

Hundreds of examples of non-consensual Grok AI creations have since come out, and the chatbot has been blocked by Indonesia and Malaysia and under investigation in the UK.

The company now appears to be altering course after the backlash.

"I have been informed this morning that X is acting to ensure full compliance with UK law," UK Prime Minister Sir Keir Starmer told parliament today, adding that the government would take further measures if needed.

Musk posted on X today that Grok will always comply with the law of the countries in which it operates, and also said that he was "not aware" of any naked underage images created by Grok: "literally zero".

"When asked to generate images, [Grok] will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state," he said.

Musk has not yet made further statements about how exactly Grok will be changed.

New Zealand has been described as "lagging behind" on policing deepfake content and needs enforceable standards, an organisation working to prevent child sexual exploitation told RNZ earlier this week.

Auckland media commentator and journalist Russell Brown said the recent "undressing" posts on X crossed a line.

"It strikes me it's only a couple of months ago that these 'nudify' apps were regarded as totally marginal and now it's a front and centre service available from X's flagship product, which is Grok.

"The fact that the company could entirely withdraw that part of the service and say, 'no undressing app,' and they've made it clear that they won't do that, it really should raise some questions."

ACT MP Laura McClure's member's bill to criminalise non-consensual sexually explicit deepfakes was pulled from the ballot last year and will be considered by Parliament in the future.

McClure said while she was horrified by deepfakes, she did not feel X should be targeted.

"I'm alarmed by deepfake abuse because I've heard firsthand from victims, including under-18s, who have been placed in sexual images without their consent. The harm is real, it's happening right now, and it's not limited to X.

"Singling out one company might feel satisfying, but it doesn't actually stop the abuse. Hundreds of platforms provide access to this technology, and I think rather than playing whack-a-mole with tech companies we should directly target the abusers who are exploiting the technology. That's the approach I've taken with my member's bill."

There have been calls to remove Grok from app stores, while Musk himself claimed it was the number one app in New Zealand in a recent post.

Grok hits #1 in New Zealand https://t.co/RPoQilXMnU

— Elon Musk (@elonmusk) January 10, 2026

Are NZ politicians still using X?

World politicians often use X to make announcements and statements, and New Zealand is no different.

Prime Minister Christopher Luxon regularly communicates on X, as well as many other social media platforms such as Facebook, Instagram and TikTok.

In a statement to RNZ on Wednesday, a spokesperson for the Prime Minister's office said Luxon was aware of concerns about Grok.

"The use of Grok in this manner is concerning. We are actively monitoring developments."

Many official agencies such as MetService, NZTA and NZ Defence Force still use X.

Labour Party leader Chris Hipkins does post on X, although the official Labour X account hasn't posted since May 2025.

"Political parties and MPs use a range of social media platforms," said Reuben Davidson, Labour's spokesperson for media and digital economy.

"The Labour Party has chosen to prioritise channels where we can engage constructively with New Zealanders and reach the audiences we are focused on."

The ACT party is also a frequent user of X, but in a statement posted Wednesday condemned deepfakes, while also saying regulating companies or the technology was a step too far.

"The harms are very real, and the problem is not limited to X," the party's official account posted on X. "New Zealand law should target the individuals who are engaging in abusive behaviour."

"We should target abusers directly, rather than shutting down tools used legitimately for news or satire."

Explicit deepfake abuse is rife, the harms are very real, and the problem is not limited to X.
New Zealand law should target the individuals who are engaging in abusive behaviour.

— ACT New Zealand (@actparty) January 13, 2026

NZ First leader Winston Peters is also a regular user of X, as is the party's official account.

The Green Party's official account hasn't posted since 2024, while Te Pāti Māori last posted in June.

Many other politicians post announcements or statements regularly on X, including Wellington Mayor Andrew Little and Auckland Mayor Wayne Brown.

X-slash-Twitter has wielded a heavy influence on social media over the years, despite its relatively smaller user base than other social networks.

It's difficult to find precise figures, but research by Meltwater/We Are Social for 2025 showed Twitter was only the 11th most used social platform, with just 23.8 percent of internet users on it, while DataReportal said it estimated 15.2 of eligible audience (those over 13) used it.

In InternetNZ's 2024 Internet Insights report, X was among sites like Discord and Mastodon that it lumped in with the description "most New Zealanders have never used these social media platforms". It found 73 percent of New Zealanders never used X.

"Most New Zealanders would agree that the creation and distribution of non-consensual or abusive AI-generated images is utterly unacceptable and raises serious questions about the responsibility of platforms to create a safe online environment," Labour's Davidson said.

He said he has proposed a member's bill that would hold platforms more accountable for harmful content, although it has not been pulled from the ballot.

In addition to McClure's deepfakes bill, Education Minister Erica Stanford has also promised regulatory changes over social media harm in response to calls for a minimum age of 16 to access social media similar to what Australia has recently instituted.

Has X gone too far?

Brown said that X has lost its usefulness to many people and that some New Zealand accounts have been leaving the platform.

"That's the tragedy of it. The old Twitter had come to fill a kind of important niche for those kind of institutional accounts and it was a way to reach the public quickly with information. And now you're asking the public to wade through a sewer to get there. It's broken now."

"I think the discussions are being had" about leaving X, he said. "I think there are public groups who are on X who are having the discussion, I think."

Free speech and human rights groups are also staking out sides on whether to rein in X.

Rights Aotearoa put out an open letter to Justice Minister Paul Goldsmith calling for regulation of X, while the Free Speech Union blasted the rise of "censorship infrastructure."

Rights Aotearoa said it was "deeply concerned" about X's Grok tool.

"This represents a serious and immediate threat to human dignity, privacy, and safety," its chief executive Paul Thistoll said. He said Grok's image generation raised concerns about image-based sexual abuse, child safety and gendered and identity-based harm.

"The non-consensual creation and potential distribution of intimate imagery constitutes a form of sexual violence with severe psychological and reputational harm to victims."

However, Free Speech Union chief executive Jillaine Heather condemned calls to block X by the UK and warned New Zealand against doing the same.

"Today it's X for failing to police Grok. Tomorrow it's any platform a government decides is insufficiently compliant. Australia already banned everyone under 16 from social media last month. Not from illegal content. From platforms entirely. Safety rhetoric, expanded control.

"Attempts to use child safety as a pretext to ban, restrict, or de-platform X would represent a serious breach of free expression and would be met with determined resistance. If the New Zealand government joins this international suppression effort, it will have a fight on its hands.

"Free speech is not a privilege granted by governments when it suits them," Heather said. "It is a fundamental right and it will be defended."

But Rights Aotearoa's Thistoll said before today's announcement that the government should act.

"The technology exists. The harm is occurring," Thistoll said. "The government has both the power and the responsibility to act."

ACT's McClure said she also felt there could be free speech issues if X was banned or blocked in New Zealand.

"A crackdown on X, or any social media platform, would have free speech implications. I don't personally use X, but I know many New Zealanders use X legitimately as a public square, and for news or satire.

"The ACT Party posts on X to reach these New Zealanders, and I don't think it's fair to deny these people a news source - whether by banning X or boycotting it - just because of the actions of a small minority.

"If X was totally flooded with repugnant images, I expect decent-minded Kiwis would stop using it. In that case there would be no point in ACT posting on it.

"I think like many social media companies, X is grappling with how to handle its AI tools. Tech companies can move faster than regulators, they can listen to their users, and in X's case it appears there are already changes to the way X handles requests for the bikini images."

Brown said what has happened to X is also part of a wider shift away from platforms like Facebook and broad concerns about "enshittification" of the internet and the polarisation of politics sweeping across all platforms.

"I think a lot of people are moving away from social media altogether, and exactly what solution they've found for keeping in touch with people I think varies from person to person.

"With what's happened with Grok and these non-consensual undressing apps, I think we actually have reached the point where decisions need to be made."

Sign up for Ngā Pitopito Kōrero, a daily newsletter curated by our editors and delivered straight to your inbox every weekday.