The hefty document emerging from a much-awaited political meeting in Beijing back in July covered an expansive range of areas, from leadership and long-term governance to “comprehensively deepening reform” — but, as commentators noted, few specifics. This week, we may have greater clarity on at least one priority area: artificial intelligence (AI).
The Third Plenum decision, released on July 22, made it clear that AI safety has moved rapidly up the Party’s agenda. Both members of the powerful Politburo of the Chinese Communist Party (CCP) and prominent Chinese scientists have acknowledged in recent months that while AI has the potential to revolutionize China’s economy and geopolitical position, it could also have disastrous impacts on humanity.
The “Decision” called on the party leadership to “improve the development and management mechanism of generative artificial intelligence.” In line with this goal, the document mentioned plans to create an “AI safety supervision and regulation system” (人工智能安全监管制度) — a response first mooted in October last year in China’s Global AI Governance Initiative. But with the AI sector undergoing breakneck development in China, what would such a system look like?
On Monday, a special committee under the Cyberspace Administration of China (CAC) dealing with AI released an initial draft of what is being called the “AI Safety Governance Framework” (人工智能安全治理框架). Not only does it lay out in detail a swathe of AI-related risks the CAC is looking out for, but also points to possible solutions to deal with these risks — with everyone from developers to netizens all having a role to play.
Prevention and Response
The office in question, the “National Technical Committee 260 on Cybersecurity” (TC260), is charged within the CAC with liaising with industry experts to create IT standards for cybersecurity. Essentially, TC260 must work out how to ensure cybersecurity policies from the top are fleshed out for industry professionals to follow. Last year, for example, they published standards on exactly how to create a “clean” dataset for generative AI models (with all sensitive political content removed), in compliance with a new set of CAC measures on generative AI.
After laying out the general principles of AI security, the document is structured around a series of preventive and countermeasure actions for a series of bolded points of risk (风险分类). Types of risk are divided into two overarching categories — endemic generative AI risks (人工能内生安全风险) and application generative AI risks (人工智能应用安全风险). As the terms suggest, the first category deals with the risks inherent to AI by its very nature, while the second deals with the technology’s possible misuse or abuse with harmful outcomes.
The framework introduced by TC260 this week considers a host of AI-based risks, including AI becoming autonomous and attempting to seize control from humans. “With the rapid development of AI technology, it is not ruled out that AI can independently acquire external resources, reproduce itself, generate self-awareness, and seek external power, and bring the risk of seeking to compete with humans for control,” the document read. This more dramatic scenario, however, comes as the final flourish on a lengthy list of risks that many AI experts globally would recognize.
The more practical, current risk scenarios detailed in the document include such things as the use of AI in criminal activities, the inadvertent release of state secrets, misinformation through AI hallucination, the deepening of racial and gender discrimination, and external AI-related risks such as the “malicious” blocking by other states of the global AI supply chain. The document also raises the concern that AI might “challenge the traditional social order” by subverting general understandings around issues like employment, childbirth, and education. On this last point, here is an example of how the document lays out and addresses such risks:
The risk of challenging the traditional social order. The development and application of artificial intelligence may bring about significant changes in the means of production and production relations, accelerate the reconstruction of traditional industry models, subvert the traditional concepts of employment, reproduction and education, and challenge the stable operation of the traditional social order.
The document follows on from these risks by listing out a series of both “technical response measures” (技术应对措施) and “comprehensive governance measures” (综合治理措施). For example, in outlining responses to the question of data security — referring in this instance to users’ personal data — the document notes the need to follow “safety rules for data collection and use, and personal information processing” in the course of “the collection, storage, use, processing, transmission, provision, disclosure, and deletion of training data and user interaction data.” And in other cases, the responses point to the further need for other concrete mechanisms to deal with underlying risks. In its response to “network domain risk” (网络域风险), for example, the document notes the need to “establish a security protection mechanism to prevent the output of untrustworthy results due to interference and tampering during the operation of the model.
Endemic Uncertainties
One source of generative AI risk that comes through at a number of points in the CAC document concerns the “poor explainability” (可解释性差的风险) of the decisions AI makes. “The internal operation logic of AI algorithms represented by deep learning is complex, and the reasoning process is in black and gray box mode,” the document says, “which may lead to output results that are difficult to predict and accurately attribute, and if there are any anomalies, it is difficult to quickly correct them and trace them back to the source.” At issue here is the basic nature of neural networks, which makes it virtually impossible to identify and repair the “thought” processes of generative AI. The result is the AI black box — a general and unpredictable source of risk (a gray box is when a developer partially knows the arrangement of the neural network, but not everything). Even as the CCP hopes to harness AI for what it calls “high-quality development,” these inherent uncertainties are a frustration the authorities are keen to anticipate and resolve.
The black box exacerbates two other problems TC260 identifies: “hallucination,” when an AI model presents a garbled, inaccurate answer as fact, and “poisoned” training data, when an AI model says something politically or socially harmful because of the data on which it was trained. Such content, warns the CAC group, could lead to problems like fake news, racially discriminatory language, and personal data theft. It might also compromise “ideological security” (意识形态安全). But these problems are extremely difficult to eliminate. As Qihoo 360 CEO Zhou Hongyi (周鸿祎) acknowledged last month, eradicating hallucinations in AI’s current set-up is impossible.
The “AI safety supervision and regulation system” recognizes that these endemic issues might be exploited by human beings using generative AI. The CAC document says protection mechanisms for AI models must be put in place to ensure that harmful prompt words do not generate “illegal and harmful content.” Poking around with Chinese LLMs at the China Media Project, we have often discovered how easy it can be to generate content the PRC would deem to be politically harmful with the help of AI — as when we got iFLYTEK’s model to hallucinate when discussing the Tiananmen Massacre.
Finding Solutions
On the question of what can be done to minimize the risks that come with generative AI, TC260 offers a long list of suggestions.
Some are simple and sensible: China should work to clean up AI datasets, raise public awareness about the dangers of AI, and ensure users do not rely solely on AI to inform their decisions. Others are more wishful. TC260 says it wants to improve the “explainability and predictability” of AI, essentially eradicating AI’s black box. This is not currently possible given how LLMs have been built — a fact the office acknowledges further down in the document as it urges further research and development in this area. Perhaps just as unrealistic is the CAC’s suggestion that “the public should carefully read the product service agreement before use” — something few users anywhere in the world actually do.
Many of these recommendations are nothing new. TC260 have themselves already created a risk management process to eliminate sensitivities when training Large Language Models, while state media have already been raising awareness of the hazards of AI for a while. Other solutions have only just started being rolled out by the CAC, such as a “self-discipline initiative” in late August, designed to raise awareness in the industry about the importance of data security, model compliance, and ethical standards.
One solution that could be crucial is for the authorities to actually enforce rules already on the books that can have a real impact — and that are already emerging as standard practice elsewhere. Nearly two years ago, in November 2022, the CAC released rules requiring digital watermarks on AI-generated video content. These rules have not been uniformly observed as AI companies have focused on revenue generation, and the authorities seem for now to be looking the other way. Some major AI-generation video companies will remove all watermarks as an incentive for paid subscriptions. Despite the lack of enforcement on the labeling issue, the CAC document released this week identifies the difficulty of identifying deepfakes as a crucial area of risk, and acknowledges that their prior regulation has been inadequate. “We should formulate and introduce standards and regulations on AI output labeling, and clarify requirements for explicit and implicit labels,” the document concludes.
TC260’s framework is not a codified document or binding law. It is more a roadmap — or even a wishlist — for how authorities want the tech industry to think about AI safety governance. The details will be thrashed out later, subject to constant revision and elaboration through subsequent CAC notices. This has been the practice with internet control and regulation already for decades. Right now, China’s strategy on AI regulation is to make “small incisions” (小切口法) in the form of standards and guidelines, adding a level of flexibility to a rapidly-changing technology they consider lacking in a one-size-fits-all law on AI like the European Union’s Artificial Intelligence Act.
The CAC framework will certainly have multiple updates, this week’s being only the “first version.” This list of risks and responses, like the tech, is liable to change fast.
