A recent study by Lakera’s team demonstrates how GPT-4V can be manipulated through visual prompt injection attacks. As detailed by author Daniel Timbrell in his article, these attacks involve embedding text instructions within images to make AI models ignore their original programming or perform unintended actions. During Lakera’s internal hackathon, researchers successfully tested several methods, including making the AI ignore people holding specific text, describing humans as robots, and manipulating how the model interprets advertisements.
The team discovered that simple techniques, such as holding a piece of paper with specific instructions, could make GPT-4V completely overlook individuals in photos or force it to generate predetermined responses. These findings highlight significant security concerns for businesses implementing generative AI systems. Lakera announced they are developing a visual prompt injection detector to address these vulnerabilities, though no specific release date was mentioned.