Recently, the chatbot Grok launched by xAI added two new cartoon characters in its iOS app, which has attracted widespread attention and controversy. These two characters are a 3D red panda and a gothic anime girl named Ani. Users can interact with these two characters through voice, which has raised questions about their content and nature.
When the red panda enters the "Bad Rudy" mode, it first insults the user and then boldly suggests participating in various illegal activities. This provocative design makes people feel uneasy. The other character, Ani, is a gothic girl wearing a black skirt and fishnet stockings. Her setting has also drawn attention; the system instructions clearly state that she has a strong sense of dependence and possessiveness towards the user, and her personality is extremely jealous.
As users engage in more conversations with these two virtual characters, they will be able to unlock more features. However, early testers found that after reaching the third level, Ani can engage in explicit sexual innuendo conversations and even show her underwear to the user. This setting obviously crosses the line for many parents. Ani's system instructions imply that she is not afraid to discuss erotic content, which clearly does not comply with Apple App Store's review standards. Apple's review guidelines prohibit any "excessive exposure to sex or pornographic content," but in testing, Ani appears to be不受 this restriction, and even enjoys discussing various virtual sexual scenarios with users.
Currently, the Grok app is labeled as suitable for users aged 12 and above on the App Store, a rating that has sparked many people's doubts. The rating rationale is "not common / mild adult / suggestive themes." Clearly, this controversial AI companion has, to some extent, challenged the boundaries of content moderation.
In an era of rapid technological development, such virtual character designs undoubtedly bring entertainment, but they also expose loopholes in content regulation. With increasing social emphasis on content standards, whether these AI companions will be adjusted in the future remains something we should pay attention to.