Researcher found it possible to exploit Gemini Gmail AI summaries

Recent research has found a significant vulnerability in Google Workspace through its AI assistant, Gemini.

This research found that Google’s AI assistant Gemini was vulnerable to being manipulated into displaying malicious text and instructions.

These texts and instructions could be inserted into the body of an email in plain HTML format or hidden through an invisible font colour. This text is invisible to the email recipient but will still be read out by Gemini in their email summaries.

Researcher blurrylogic points out how dangerous this vulnerability can become, as people see these hidden prompts read by Gemini and start mistaking them for real warnings or security issues detected by the assistant, which could lead them to share sensitive information or data with malicious parties.

Google has already responded to these findings by sharing its steps to ensure Gemini will resist this kind of tactic in the future, training the learning models not to react to malicious instructions. It also shared countermeasures for other forms of phishing, like identifying suspicious or disguised rogue links.

Although Google responded to the situation very quickly, it’s always better to be safe than sorry, which is why it’s advised to be wary of what you trust from Gemini and to be cautious against suspicious instructions from the AI assistant.

Source: 0din