THRAX: A Security Evaluation Platform for Document Based Prompt Injection in Large Language Models

202111037 Rami Alatrash 

202110027 Basel R M Tarifi

202110524 Yahye Abdihakin

202210663 Moatassim Arbaou

 

Large Language Models (LLMs) are increasingly used in document-processing systems and to aid the decision-making process for users. Documents can, however, have hidden malicious content that changes the way the model acts and causes it to give unsafe or misleading answers. Document-based prompt injection is one of the most serious security threats faced by contemporary applications of LLM, and it is known as such [1].

This project's primary goal was to develop and create a Web-based platform to simulate document-based prompt injection attacks and assess defense mechanisms against them. To this end, a platform called Thrax was created. The system supports uploading of documents, the ability to interact with a specific LLM in controlled environments, running various PIA scenarios, and examining the effectiveness of various defense configurations.

The completed platform was able to successfully deploy six document-based attack categories, as well as a seven-mechanism defense pipeline on both the input and the output. Furthermore, the system featured a Threat Intel Center that allowed for the simulation of both clean and poisoned documents alongside, an InfoBank fixture library, an automated stress testing framework, analytics and trace reporting capabilities, and role-based access control for administrators and general users.

An automated stress testing framework was implemented to test Attack Success Rate (ASR) and Defense Effectiveness, on various attack scenarios and multiple iterations. The evaluation results demonstrated that the layered defense pipeline more successfully curbed successful attack behaviour across various categories, while LLMs without such protection could be manipulated with malicious document content.

The final platform offers a hands-on experience for exploring prompt engineering attacks on document-oriented LLM systems and showcases the effectiveness of multi-layered security for LLM-powered document applications.