I hate to admit it, but I've spent a lot of money online this holiday shopping season. And it's not surprising that some of those purchases didn't meet my expectations. A photo book I bought was damaged in transit, so I took a few photos, emailed them to the seller and got my money back. Online shopping platforms have long relied on photos submitted by customers to confirm that refund requests are legitimate. But generative AI is now starting to break that system.
A pinch too suspicious
On Chinese social media app RedNote, WIRED found at least a dozen posts from e-commerce sellers and customer service representatives complaining about AI-generated refund claims they had received. In one case, a customer complained that the sheet he purchased was torn to pieces, but the Chinese characters on the shipping label looked like gibberish. In another, the buyer sent a photo of a coffee mug with cracks that looked like paper tears. “This is a ceramic cup, not a paper cup. Who could tear a ceramic cup into such layers?” wrote the seller.
The merchants reported that there are a few product categories where AI-generated damage photos are most commonly misused: fresh groceries, cheap beauty products, and fragile items like ceramic cups. Sellers often do not ask customers to return these goods before issuing a refund, making them more susceptible to return scams.
In November, a trader selling live crabs on Douyin, China's version of TikTok, received a photo from a customer in which it appeared most of the crabs she bought had already arrived dead, while two others had escaped. The buyer even sent videos showing the dead crabs being poked by a human finger. But something wasn't right.
“My family has been breeding crabs for more than 30 years. We have never seen a dead crab with its legs pointing upwards,” Gao Jing, the seller, said in a video she later posted on Douyin. But what ultimately gave away the scam was the sexes of the crabs. There were two men and four women in the first video, while the second clip featured three men and three women. One of them also had nine legs instead of eight.
Gao later reported the fraud to police, who determined that the videos were indeed fabricated and detained the buyer for eight days, according to a police message Gao shared online. The case attracted widespread attention on Chinese social media, in part because it was the first known AI refund scam of its kind to prompt a response from regulators.
Lowering barriers
This problem is not unique to China. Forter, a New York-based fraud detection company, estimates that AI-manipulated images used in reimbursement claims have increased more than 15 percent since the beginning of the year and continue to rise globally.
“This trend started in mid-2024, but has accelerated over the past year as image generation tools have become widely accessible and incredibly easy to use.” says Michael Reitblat, CEO and co-founder of Forter. He adds that the AI doesn't have to get everything right, as frontline store associates and refund teams may not have the time to scrutinize every photo.
