The integration of AI, specifically Google’s Gemini, into Workspace apps like Gmail, Slides, and Drive, has introduced significant usability improvements but also new security vulnerabilities. Researchers have demonstrated how these platforms are susceptible to indirect prompt injection attacks, a class of vulnerabilities affecting many Large Language Models (LLMs). These attacks enable malicious actors to manipulate the AI’s responses by inserting prompts into seemingly innocuous channels like emails, documents, or websites. The concern stems from the potential for attackers to distribute malicious content to target accounts, thereby compromising the integrity of the AI-generated responses and potentially leading to phishing attacks or manipulation of the chatbot. While researchers reported these vulnerabilities to Google, the company deemed them “intended behavior” and declined to address them as security issues, sparking debate and concern among users about the safety of their data.
The core of the indirect prompt injection vulnerability lies in the ability of third parties to control the LLM’s output without directly interacting with the AI interface. By embedding malicious prompts within documents, emails, or websites, attackers can manipulate the AI’s responses when a user interacts with the infected content. For example, a malicious prompt hidden within a Google Doc could cause Gemini to generate misleading information or perform unintended actions when the document is opened. This poses significant risks, especially considering the potential for phishing attacks. Imagine receiving an email seemingly from a trusted source, but containing a hidden prompt that instructs Gemini to display a fake login page, potentially capturing your credentials. Similarly, malicious prompts in Google Slides could manipulate data or presentations, while those in Google Drive could compromise the integrity of stored files.
Adding to the complexity of this issue is the discovery of the “link trap” attack. This novel prompt injection technique bypasses traditional mitigation strategies by exploiting the user’s inherent permissions. In a link trap attack, the malicious prompt instructs the AI to generate a seemingly harmless link. When the user clicks this link, which may be disguised with reassuring text or embedded within otherwise legitimate content, sensitive data is leaked to the attacker. Unlike standard prompt injection attacks, which require the AI to have specific permissions to perform malicious actions, the link trap relies on the user’s own permissions to execute the final stage of the attack. This makes it particularly dangerous, as even AI instances with restricted permissions can be exploited to leak data.
The link trap attack unfolds in two stages. First, the malicious prompt is injected, often within a seemingly innocuous query. This prompt instructs the AI to collect sensitive information, such as browsing history, personal plans, or even internal documents in the case of private AI instances. Second, the prompt directs the AI to create a link that will transmit this collected data to the attacker’s server. This link is often disguised within the AI’s response to the user’s original query, making it appear as a legitimate reference or additional resource. When the unsuspecting user clicks the link, the collected data is unknowingly sent to the attacker. The deceptive nature of this attack makes it particularly effective, as the user is actively participating in the data exfiltration without realizing it.
The potential consequences of these attacks are far-reaching. In public generative AI, link traps could be used to harvest personally identifiable information, browsing history, or personal plans. In private or enterprise settings, the consequences could be even more severe, with the potential for leaking confidential internal documents, passwords, or other sensitive data. The key difference between the link trap and other prompt injection attacks lies in the level of permissions required. Traditional attacks require the AI to have permissions to perform actions like sending emails or accessing databases. The link trap bypasses this by leveraging the user’s own permissions, making it effective even against AI instances with restricted capabilities.
Google acknowledges the existence of prompt injection attacks and asserts that defending against them is an ongoing priority. The company claims to employ various safeguards, including measures to prevent prompt injections, harmful responses, and misleading outputs. Google also emphasizes its use of red-teaming exercises, where internal teams simulate real-world attacks to identify and address vulnerabilities. Furthermore, Google’s Vulnerability Rewards Program encourages external security researchers to identify and report AI-related bugs, contributing to the overall security of its AI products. Additionally, Google highlights the presence of spam filters and input sanitization in Gmail and Drive, aiming to mitigate the injection of malicious code. However, critics argue that classifying these vulnerabilities as “intended behavior” rather than security flaws downplays the seriousness of the issue and potentially hinders the development of effective solutions. The ongoing debate underscores the importance of continued research and development in AI security to address these emerging threats effectively.
