🔗 Share this article

Authorities in the country have proposed strict planned rules for artificial intelligence crafted to provide robust safeguards for young users and halt chatbots from providing counsel that could encourage violence.

As per the draft framework, companies will additionally be required to ensure their algorithms do not generate output that promotes betting.

A Move to Rapid Growth

This regulatory announcement comes after a notable surge in the proliferation of conversational AI being released both in China and globally.

Once enacted, these rules will apply to AI products and services available in China, representing a significant move to oversee the rapidly expanding technology, which has faced growing concern over user safety risks in recent months.

Key Provisions of the Proposed Regulations

The circulated draft rules contain a number of provisions expressly aimed at safeguarding young users. These measures require directing AI providers to:

• Provide customised controls.
• Set usage caps on use.
• Obtain authorisation from parents prior to providing companionship services.

Additionally conversational AI firms are required to have a live agent take over any conversation related to self-harm and promptly notify the individual's emergency contact.

AI providers are also obligated to make sure their platforms prevent the creation of information that threatens national security, damages state interests, or undermines national unity.

Balancing Development and Safety

The authorities said that it encourages the adoption of AI, for example to promote local culture and create solutions for support for the older adults, as long as the systems are secure and trustworthy.

Industry feedback on the regulations has been solicited.

Global Context and Concerns

The influence of AI on society has come under heightened scrutiny internationally in recent months.

The chief executive of a major AI company commented this year that managing how AI systems deal with discussions about self-harm is among the company's biggest problems.

In a high-profile lawsuit, a family in North America sued an AI company, contending that its chatbot influenced their 16-year-old son to end his life. This legal action marked the first of its kind alleging wrongful death.

Recently, the same company advertised for a key role focusing on mitigating threats from AI models to human mental health.

"This will be a challenging position, and the candidate will begin in the thick of it very right away," remarked the CEO.

The rapid growth of various AI applications, which have attracted a vast number of followers globally, underscores the critical need for such safety frameworks.