Link: http://arxiv.org/abs/2502.06390v1
PDF Link: http://arxiv.org/pdf/2502.06390v1
Summary: Vision-Language Models (VLMs) have gained considerable prominence in recentyears due to their remarkable capability to effectively integrate and processboth textual and visual information.
This integration has significantlyenhanced performance across a diverse spectrum of applications, such as sceneperception and robotics.
However, the deployment of VLMs has also given rise tocritical safety and security concerns, necessitating extensive research toassess the potential vulnerabilities these VLM systems may harbor.
In thiswork, we present an in-depth survey of the attack strategies tailored for VLMs.
We categorize these attacks based on their underlying objectives - namelyjailbreak, camouflage, and exploitation - while also detailing the variousmethodologies employed for data manipulation of VLMs.
Meanwhile, we outlinecorresponding defense mechanisms that have been proposed to mitigate thesevulnerabilities.
By discerning key connections and distinctions among thediverse types of attacks, we propose a compelling taxonomy for VLM attacks.
Moreover, we summarize the evaluation metrics that comprehensively describe thecharacteristics and impact of different attacks on VLMs.
Finally, we concludewith a discussion of promising future research directions that could furtherenhance the robustness and safety of VLMs, emphasizing the importance ofongoing exploration in this critical area of study.
To facilitate communityengagement, we maintain an up-to-date project page, accessible at:https://github.
com/AobtDai/VLM_Attack_Paper_List.
Published on arXiv on: 2025-02-10T12:20:08Z