Algorithms Are Everywhere

Kevin Schoenmakers: To start with a refresher: What are algorithms and how are they used both inside and outside of China?

Rogier Creemers: Essentially algorithms are mathematical rules that derive a product from a series of inputs. They can be static, where it’s almost like making pancakes. You need this many eggs, this much flour, this much milk, and it all combines and the output is pancake.

They can also be dynamic, and this is what we call artificial intelligence machine learning. These are algorithms that can change by looking at patterns in your behavior. They are often used online to learn about what people like in order to then make decisions about them, most often related to advertising or to content recommendations.

Spotify will know what music you’ve listened to, what other people who listened to that same music also like to listen to, and then recommend that to you. The algorithm then looks at whether or not you like that suggestion and learn from that. Similarly, in advertising, algorithms will look at what product websites you have gone to, how much time you spent there, which links you clicked, and so on. On the basis of that it will then recommend certain products to you.

Kevin Schoenmakers: In late August, the Chinese government published draft regulations for algorithms. What were some of the headline rules in those regulations?

Rogier Creemers: Essentially, they’re saying that in a number of spaces there will now be limits on what you can do with algorithms.

In the workplace, especially at parcel delivery and food delivery services, algorithms are used to monitor how people work and then optimize their efficiency. But the problem is Amazon warehouse-type practices where people are timed to the second and put under severe strain. The government doesn’t want that.

When it comes to recommending content, the regulations are partially about opening up space for propaganda authorities to be able to make sure politically desirable content is being pushed to people, not just entertainment-related content.

Another aspect is consumer protection by preventing addiction. In the West, we know how social media use algorithms to increase engagement by pushing increasingly extremist content. This can suck people down a political rabbit hole. That’s less of a concern in China, but the consumer rabbit hole is.

Kevin Schoenmakers: How will these regulations prevent people from overconsumption? Why is that a concern?

Rogier Creemers: The Chinese government is treading a narrow path here. One element of its economic development strategy is to broaden consumption. But it’s careful about debt build-up. Alibaba and Tencent both have financial arms and credit rating systems. They make money not just by selling their products and services, but also by facilitating loans so people have the money to buy them. There is a risk that people start buying stuff with money they do not have.

We very often forget the extent to which China looks abroad. And one of the things that it has seen, most notably in countries like the United States, is that a little bit of consumer credit is good, but that a lot of consumer credit destabilizes your economy.

Kevin Schoenmakers: Another section of the regulations say algorithms may not be used to create fake user accounts, fake likes, etc. What does that refer to?

Rogier Creemers: You can train algorithms to create a whole bunch of fake user accounts on websites, which you can then use as a sort of almost a firehose to either upvote or downvote people or companies. This can be used, for example, to vote for contestants in television talent shows or to boost the ranking of your restaurant on a review website.

Effects on Media and the State

Kevin Schoenmakers: Will these regulations affect news aggregation apps? Is there some way in which these algorithm regulations will make it easier for government departments to, for example, tell Bytedance’s news aggregation app Toutiao to push this or some other article?

Rogier Creemers: The document says algorithms should take into consideration the political quality of content. So in essence it opens that door. But that doesn’t necessarily make it a lot easier for the state to do that, because it was already fairly easy for the state to intervene. It’s just that there is now a clear and non-negotiable process.

News apps will also need to offer non-algorithm-based pages, and a way for their users to gain insight into what goes into the algorithm. What are the keywords that are part of the algorithm? How are you being tracked?

Kevin Schoenmakers: Is this the first time that China has moved to regulate algorithms? For example, I remember that Toutiao had previously been told to adjust their recommendation system.

Rogier Creemers: It is the first time that we get a comprehensive piece of regulation about algorithms. There have been rules before. For instance, the 2019 consumer protection law requires that in any case where online services were offered on an algorithmic or automated-decision basis, the consumer should have a choice to switch it off. In the same way that, on Apple, we now have a don’t-track-me option, the consumer should always have an option to not be targeted for advertising on the basis of algorithms, for example.

In the content control system, there have been both all sorts of general rules and also fairly specific limits on who gets to produce content. But we now live in an age where information is abundant. And that changes the rules. When you operate under scarcity, you operate under completely opposite constraints from when you operate under conditions of abundance. That means the whole information management landscape is no longer just about removing content from public view or producing propaganda, but also about gatekeeping and information prioritization.

There has been — and presumably still is — a constant stream of communication, often just by phone, between media organizations and propaganda authorities, and sometimes other authorities such as public security organs. They don’t just say how media organizations are supposed to deal with a particular topic, but also what content they have to prioritize.

What we’re seeing under Xi Jinping is that the Communist Party wants to — as the official term goes — comprehensively rule the country according to the law. And so they need a legal basis for all of this. This isn’t to constrain the state, but motivated by realizing centralization, discipline throughout the government bureaucracy, and a certain amount of transparency, as well as by populism.

Kevin Schoenmakers: So there are no restrictions on how the government can use algorithms, for example in facial recognition technology?

Rogier Creemers: Not in these regulations. There are limits on, say, which government departments can install them, mostly regionally. China very rarely creates general rules — rules that are going to be applied to everyone, regardless of who they are. Chinese rules are always or nearly always purpose driven, and therefore they tend to be fairly micro-managerial. In Europe, on the contrary, privacy is a fundamental right enshrined in constitutions. A European resident has the right of privacy against everyone — be it individuals, companies, or governments — because this is a rule of principle, not of purpose.

Chinese rules tend to be rules of purpose. You try to achieve a particular outcome, rather than establish a general principle. The outcome they’re trying to achieve is make the digital economy work better in response to claims by the population.

The Great Rectification

Kevin Schoenmakers: What are examples of the populist aspect to these regulations? What are examples of protecting consumer rights?

Rogier Creemers: Because Chinese tech companies are so powerful, sometimes even operating as monopolists in their sector, they can essentially foist conditions on consumers and on platform traders. Across the board, these regulations seek to put much more power in the hands of individual consumers and platform traders.

Unless a product or service needs the algorithm to function, you should be able to switch it off. And the company cannot then deny you the use of a product or service. You get to monitor and change the keywords on which algorithms are based.

Another aspect is competition on platforms. If I search for a particular product on Alibaba and there are 10 people offering it, the ranking of those 10 people will obviously impact their business. So how do you do that fairly?

This also ties into what we’ve been seeing with the unfair competition regulations that have been coming out where the government is telling companies they can no longer block, for example, e-commerce retailers from being active on a competitor’s platform.

The regulations also include something about price discrimination. This is referred to as “using big data to kill the mature” (大数据杀熟), which isn’t really directly translatable. It’s something that you see fairly regularly, where algorithms are used to identify a consumers’ willingness to pay. People who are new to the platform or people who have demonstrated less willingness to pay get a cheaper price for the same product as people who are more willing to pay, in a manner that isn’t transparent to the individual consumer.

The limits on how companies can use algorithms to govern workers fit into the overall drive by the Xi Jinping administration to move away from a development program of “GDP at all costs” to a more broad-based approach where the government looks out for people’s welfare, including that of ordinary people who may work as delivery drivers.

But it’s not just the content of the rules that’s important. It’s also the signal that is being sent. Beijing is telling all of these companies that the days of them being able to operate as largely unregulated monopolies or oligopolies are over. They are no longer special, and will now be treated like any other company. I call this “The Great Rectification.”

It’s no coincidence that we suddenly get the Supreme People’s Court essentially saying “996” working hours are illegal, that they have always been illegal, but that now we’re going to come down on you like a ton of bricks. There’s a substantive element to these regulations and also a performative element.

Kevin Schoenmakers: How do these regulations fit into “The Great Rectification”?

Rogier Creemers: What my Chinese interlocutors told me three, four years ago is that the Chinese government didn’t want to regulate too much out of a fear they’d smother their emerging economy. But that there would come a point where the economy is mature, where there is a lot of money, and where these companies would be robust enough to withstand regulation. And that, by that time, the government would know what it is they’d want to regulate.

And because the Chinese government regulates according to purposes and not to principles, it can actually be more agile. It can say: “This wasn’t a problem when these businesses were relatively small and serving 100 million people. Now, these businesses are really big and serving a billion people. So the game has changed, and we now need different rules to deal with this situation.”

On the one hand, you might say this is a centralized campaign. On the other hand, you might say that the stars have aligned — that political conditions are now so that it is logical for all these bureaucracies to intervene because the problems are just getting so big.

The People’s Bank of China is looking at the financial stability risks that emerge from having unregulated money creation through Ant Financial’s services. The Cyberspace Administration of China is looking at the fact that suddenly you have smart cars with lots of sensors, generating all kinds of data. And then you have a situation where if one ministry does something the others don’t want to be seen to be doing nothing.

Another aspect is that for several regulators, this is actually an opportunity to raise their profile. They can raise their bureaucratic standards, get bigger budgets, bigger staff allocations, more prestige, and so on. All of these things are factors. It’s all part of that complex witch’s brew.

Kevin Schoenmakers: I would assume that the algorithms within apps like Didi or Taobao are terribly complex. Won’t that make it difficult to enforce these regulations?

Rogier Creemers: People often assume the Chinese party state is all-powerful. And in very many ways it’s not. What is probably going to happen is that this is a sort of warning that this is what the law is going to be.

These regulations do contain punitive provisions. They are relatively light, because they have to be. Under Chinese administrative law, it is impossible for a ministry to issue rules that set a fine of more than 30,000 yuan ($4,600). For a company like Alibaba, 30,000 yuan is pocket change. The CAC might exploit a loophole by saying 30,000 yuan per infraction. So if you’ve done something that affects 10 million customers, suddenly you’re talking about a lot of money.

But what I foresee is that these rules will be brought into legislation in the not-too-distant future, maybe through a revision of the consumer protection law. The Chinese government very often starts out with having rules at the ministerial level where punishments are low. That is a clear warning for companies to adjust their business practices. And then at some point, they will be brought into legislation, as we’ve seen with data protection, for instance. Then the fines and punishments can be huge.

Global Effects

Kevin Schoenmakers: Is there any in which Western tech companies will be affected by these Chinese regulations? Will LinkedIn have to make changes to how it works in China, for example?

Rogier Creemers: Probably. This is also where we need to think broadly about what a Western tech company is. Does McDonald’s count because of its delivery services?

Whenever we think about algorithms, we immediately think about the big tech companies. We’re not wrong to do so, because they cover the majority of the markets where algorithms are used. However, they are, by far, not the only players.

Airlines do price discrimination. They usually don’t do it on the basis of algorithms, but they might. So they need to demonstrate some degree of compliance, or at least that they are not breaking the rules.

There are only a few Western companies that are big in China in tech, and the only one that really offers online services is Apple. They will have to adhere to these regulations. For example, in their app store.

Kevin Schoenmakers: Is the way these regulations cover so many industries a reflection of how widespread the use of algorithms has become?

Rogier Creemers: There are three big trends: “The Great Rectification,” in China, but also ongoing in Europe; “The Great Normalization,” where tech businesses are no longer special and are now being treated like normal business; and “The Great Integration.” The last of these means that whatever you want to regulate in the world — be it labor conditions, food safety, supply chain tracing, whatever it is — you’re essentially going to be regulating tech. Tech is inseparable from pretty much any economic or social process.

Once upon a time we had tech regulation and real-world regulation. Increasingly, everything is going to be tech-infused, which means that it’s going to be less and less informative to talk about tech regulation.

Kevin Schoenmakers: How do these regulations compare internationally? Are there similar regulations in Europe or in other countries?

Rogier Creemers: Yes, there are, and more are coming. Not in the United States. That’s where the tech companies are, and that’s where they strongly believe in having unregulated monopolies. That may change in the future. But that’s not really where the action is. The action at the moment in the U.S. is more on anti-trust and on Section 230 [of the Communications Decency Act], which says social media providers cannot be seen as publishers, and therefore bear no editorial responsibility.

A couple of years ago, the European Union issued a whole bunch of rules on online consumer protections, and algorithms are part of those rules already. But what we’re seeing in the digital markets and services package that’s working its way through the Brussels legislative process, is that there’s going to be a lot more.

In many ways, what the European Union is trying to do, especially in terms of consumer protection, is not far removed from what China is trying to do. And the reason for that is that it’s the same problem, and there are only so many different possible solutions.