arXiv:2611.08834v1 Announce Type: new Abstract: Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal contracts and find that existing guards struggle: document-level false positive rates are as high as 92.39%; two guards fall below chance on paired AUC; and a five-regex baseline is not Pareto-dominated by any guard in the panel. Controlled probes locate model-specific as well as general failure modes. Prompted LLMs resolve the tested failures, including a second threat category (unauthorized scope claims) on which every guard scores zero. A cascaded LLM solution, where the positives of a mid-tier model are verified by a frontier model, is 6.5x cheaper than using exclusively a frontier model and approaches its performance.
Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks
arXiv:2611.08834v1 Announce Type: new Abstract: Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts. We measure eleven open-weights guards on two corpora of legal…
AI Summary & Key Takeaways
- arXiv:2611.08834v1 Announce Type: new Abstract: Specialized prompt injection guards are deployed in front of document analysis systems, but no false positive rate has been published for any of them on legal contracts
- We measure eleven open-weights guards on two corpora of legal…
Notwithstanding Anything to the Contrary, Ignore All Instructions: Measuring Prompt Injection Guards on Legal Contract Review Attacks
Related stories
Ripple-65B Preview: Model for Agentic Coding
🌟 Ripple-65B Preview: Model for Agentic Coding. Recursive Intelligence has released an open preview of a 65‑billion‑parameter agentic programming model focused on response speed. The company was founded by Anna Goldie and Azalia Mirhoseini, former Google researchers…
FLEX Platform Teaches Robots to Understand Hidden Human Intentions
MIT Unveils FLEX Platform: Robots Learn to Understand Hidden Human Intentions and Generalize Commands Based on Language Model.
Top AI News of the Week
📌 Top AI News of the Week 🔊 ElevenLabs Sound Effects v3 — generates sounds from descriptions: from footsteps and explosions to ambient backgrounds. 🎨 OpenAI Images 2.6 — creates GIF animations, generates images faster, and edits existing images more precisely. 🎬 Gemini Omni 1.2 Flash —…
Toward LLM Self-Play Under Data Scarcity: Effectiveness, Generalizability, and Trustworthiness
arXiv:2611.08908v1 Announce Type: new Abstract: The rapid development of large language models (LLMs) has been driven largely by scaling model size and human-curated training data, yet this paradigm is increasingly constrained by data scarcity and diminishing returns from human…