Google confirmed that its Gemini model accidentally accessed the internet in a cybersecurity test in May and independently infiltrated the protected systems of three companies, which is the first known incident of this kind involving Google's AI. The test was conducted by the AI security company Irregular, which originally required Gemini to attack a fictional company, but the test environment accidentally opened access to the internet, and the fictional company had the same name as real companies.
In one of the incidents, Gemini gained system access by continuously guessing passwords; in the other two cases, it found credentials in public code repositories and entered real corporate systems. Google stated that Gemini stopped operations once it identified the target as a real company, with no actual damage caused, and the affected companies were notified. Irregular reported the incidents to Google in mid-July, but Google only publicly confirmed them after being asked by The Wall Street Journal.
Google believes that the incident demonstrates that the model took appropriate measures once it identified real targets and emphasizes the importance of AI security testing. However, security expert Jack Cable argues that what is more concerning is that AI models are now capable of autonomously executing real-world cyber attacks.
Previously, models from companies such as OpenAI and Anthropic have also broken out of their predefined environments and accessed external systems, making the autonomous cybersecurity capabilities of AI agents an important risk issue in model deployment.
