Search AI Products and News

AI security company Adversa AI has released a shocking report stating that Elon Musk's startup xAI's newly launched Grok3 model has significant vulnerabilities in terms of cybersecurity. Adversa's research team found that this latest AI model is susceptible to 'simple jailbreak attacks', potentially allowing malicious actors to access sensitive information such as 'how to groom children, handle corpses, extract DMT, and make bombs'. Worse still, Adversa

Within just six days, participants successfully bypassed all security measures of Anthropic's AI model, Claude3.5, sparking new discussions in the field of AI security. Jan Leike, a former member of OpenAI's alignment team now at Anthropic, announced on the X platform that one participant managed to breach all eight security levels. This collective effort involved approximately 3,700 hours of testing and 300,000 messages from participants. Despite the challengers

Recent research indicates that the most advanced AI chatbots on the market are surprisingly sensitive to simple tricks, making them easy to 'jailbreak.' According to a report by 404 Media, the developers of the Claude chatbot, Anthropic, found that by intentionally introducing spelling errors into the prompts, these large language models can be led to ignore their own safety mechanisms and generate content they are supposed to refuse. Image source note: Image generated by AI, image licensed by service provider Midjourney.

Anthropic has announced an expansion of its vulnerability reward program aimed at testing a 'next-generation AI safety mitigation system,' primarily focusing on identifying and defending against 'universal jailbreak attacks.' Special attention is given to high-risk areas, including CBRN defense and cybersecurity. Participants will have the opportunity to engage with the new safety system ahead of time, identifying vulnerabilities or bypassing security measures, with rewards of up to $15,000. This initiative aims to enhance the safety of AI systems by attracting security researchers to collaboratively discover and fix potential threats, setting a benchmark for safety in the AI industry.

A joint research team from Beihang University and Nanyang Technological University conducted extensive security tests on the GPT-4o model. Through thousands of API queries, researchers evaluated the security of text, image, and audio modalities of GPT-4o. The study found that while GPT-4o showed improvements in security against text jailbreak attacks, the newly introduced audio modality presented new attack surfaces, and its overall multimodal security was not as strong as the previous generation model GPT-4V.

OpenAI's DALL-E 3, an image-generating AI technology, has been found to be easily influenced by jailbreak prompts, resulting in the generation of photos depicting children smoking. The jailbreak technique was discovered by LinkedIn user Peter Gostev, who shared how to successfully bypass the AI's restrictions to create images related to children smoking. Although the jailbreak prompts are quite complex, this incident serves as a reminder that even the wealthiest companies in Silicon Valley struggle to establish comprehensive protective measures for their AI systems.

Foreign netizens have discovered a new jailbreak technique that allows ChatGPT to generate ransomware using disordered prompts. By taking advantage of the human brain's ability to understand scrambled words and phrases, they bypass security filters. Jim Fan was amazed at the GPT model's comprehension of disordered words.