Google Docs' new AI writing features have a gaping security hole that could lead to new kinds of phishing attacks or information poisoning. Available in public beta, the "Refine the selected text," feature allows the user to have an AI bot rewrite large swaths of copy or an entire document to "formalize," "shorten," "elaborate" or "rephrase" it.
Unfortunately, the bot is vulnerable to prompt injection, meaning that a stray line of malicious text in the input can cause it to modify the output in ways that could fool the user or spread dangerous misinformation.
