Context overload attack risk for AI
Last updated: May 27, 2025
Description
Overloading the prompt with excessive tokens, for instance with many-shot examples, can predispose models to a vulnerable state.
Why is context overload attack a concern for foundation models?
Context overload attacks can be used to alter model behavior and benefit the attacker. The content it generates may cause harms for the user or others.
Parent topic: AI risk atlas
We provide examples covered by the press to help explain many of the foundation models' risks. Many of these events covered by the press are either still evolving or have been resolved, and referencing them can help the reader understand the potential risks and work toward mitigations. Highlighting these examples are for illustrative purposes only.
Was the topic helpful?
0/1000