At the end of April, China’s top internet control body, the Cyberspace Administration of China (CAC), launched one of its regular “Clear and Bright” purification campaigns, this time against AI abuses online. The campaign, the administration said, would target gaps in AI safety, including vulnerabilities in data security, unregistered AI services and tutorials teaching internet users how to use AI illegally. These tutorials cover “one click undress” (一键脱衣) apps and “face-swapping tools” (换脸工具) to create deepfakes or commit fraud.
The results of the campaign are now in. The Cyberspace Administration of China announced late last month that provincial branches, together with internal policing by social media apps such as RedNote, Douyin and WeChat, “handled” 3,500 mini apps and cleaned up millions of items of illegal information. While administration branches did not list specific examples of the content they had rooted out, private companies were a bit more forthcoming.
Baidu identified one social media account that taught internet users how to generate AI “Chinese-style beauties” to boost their social media following. Tencent, the company behind WeChat, has been taking down “one-click undress” apps and AI-generated videos that have not been labeled accordingly. But there is evidence the administration’s campaign against AI was also about ideological security: Tencent also appears to have removed videos that tell Chinese internet users how to access ChatGPT, currently unavailable within the Great Firewall.
But bad actors can still use AI if they know where to look. One mini-app, called “Many of You”, takes any video or voice recording uploaded by users and performs voice cloning and lip-synching. The company says it helps busy e-commerce livestreamers, but one user went viral on WeChat for a video she posted, demonstrating how the software could create convincing deepfakes of the voice and appearance of a passerby on the street using less than two minutes of material. “If some unscrupulous people used this technology to send [videos like this] to our loved ones and friends, think how serious it would be,” the user said.
A quick look through Telegram, an app outside the Great Firewall that has consequently become a Chinese equivalent of the dark web, shows it is still very easy to access “one-click undress” and “face-changing” software in Chinese. Telegram is used as a base for all sorts of illegal operations that still impact the safety of Chinese internet users, such as “box opening”, where an individual’s private information is shared for a fee.
With the quality of AI-generated content ever improving, the dark side of AI, from deepfakes to misinformation, is becoming a more urgent issue across the world. While China now has some of the strictest policies against this in the world, they are still far from airtight.
