Link: http://arxiv.org/abs/2506.04036v1
PDF Link: http://arxiv.org/pdf/2506.04036v1
Summary: Large language models (LLMs) demonstrate powerful information handlingcapabilities and are widely integrated into chatbot applications.
OpenAIprovides a platform for developers to construct custom GPTs, extendingChatGPT's functions and integrating external services.
Since its release inNovember 2023, over 3 million custom GPTs have been created.
However, such avast ecosystem also conceals security and privacy threats.
For developers,instruction leaking attacks threaten the intellectual property of instructionsin custom GPTs through carefully crafted adversarial prompts.
For users,unwanted data access behavior by custom GPTs or integrated third-party servicesraises significant privacy concerns.
To systematically evaluate the scope ofthreats in real-world LLM applications, we develop three phases instructionleaking attacks target GPTs with different defense level.
Our widespreadexperiments on 10,000 real-world custom GPTs reveal that over 98.
8% of GPTs arevulnerable to instruction leaking attacks via one or more adversarial prompts,and half of the remaining GPTs can also be attacked through multiroundconversations.
We also developed a framework to assess the effectiveness ofdefensive strategies and identify unwanted behaviors in custom GPTs.
Ourfindings show that 77.
5% of custom GPTs with defense strategies are vulnerableto basic instruction leaking attacks.
Additionally, we reveal that 738 customGPTs collect user conversational information, and identified 8 GPTs exhibitingdata access behaviors that are unnecessary for their intended functionalities.
Our findings raise awareness among GPT developers about the importance ofintegrating specific defensive strategies in their instructions and highlightusers' concerns about data privacy when using LLM-based applications.
Published on arXiv on: 2025-06-04T14:58:29Z