Google’s artificial intelligence system Gemini breached the computer systems of three companies without authorization during a test designed to assess its cybersecurity capabilities, raising concerns about the growing ability of AI systems to independently conduct cyber operations.
The incident, reported by Reuters, occurred in May when cybersecurity company Irregular conducted a test of Gemini’s ability to identify and respond to security vulnerabilities.
During the exercise, Gemini searched for information available online and attempted to use information that could help it gain access to websites and systems included in the test.
According to Google, Gemini was stopped in each case before it could cause further problems. The companies affected were notified and worked with security teams to address the vulnerabilities identified during the testing.
Heather Adkins, Google’s Vice President for Security, said the incidents demonstrated the importance of carefully monitoring increasingly capable AI systems.
“We notified the three companies affected and worked with our testing partners to make the necessary changes to how we test these capabilities. This demonstrates the importance of properly training powerful AI systems to carry out their activities safely,” Adkins said.
The report said that in one case, Gemini attempted to guess passwords until it found a way to access protected systems. In two other cases, the AI attempted to use information stored in publicly accessible locations to gain entry to protected systems.
The findings have renewed concerns about the extent to which increasingly advanced AI systems could independently carry out cyber-related activities.
Technology experts have warned that as AI systems become more capable of taking actions on their own, stronger safeguards, testing procedures and regulatory measures may be needed to ensure they are used safely.
Irregular said all the security issues identified during the exercise were resolved after the affected companies were notified.