Link: http://arxiv.org/abs/2505.13302v1
PDF Link: http://arxiv.org/pdf/2505.13302v1
Summary: Large language models are increasingly integrated into news recommendationsystems, raising concerns about their role in spreading misinformation.
Inhumans, visual content is known to boost credibility and shareability ofinformation, yet its effect on vision-language models (VLMs) remains unclear.
We present the first study examining how images influence VLMs' propensity toreshare news content, whether this effect varies across model families, and howpersona conditioning and content attributes modulate this behavior.
To supportthis analysis, we introduce two methodological contributions: ajailbreaking-inspired prompting strategy that elicits resharing decisions fromVLMs while simulating users with antisocial traits and political alignments;and a multimodal dataset of fact-checked political news from PolitiFact, pairedwith corresponding images and ground-truth veracity labels.
Experiments acrossmodel families reveal that image presence increases resharing rates by 4.
8% fortrue news and 15.
0% for false news.
Persona conditioning further modulates thiseffect: Dark Triad traits amplify resharing of false news, whereasRepublican-aligned profiles exhibit reduced veracity sensitivity.
Of all thetested models, only Claude-3-Haiku demonstrates robustness to visualmisinformation.
These findings highlight emerging risks in multimodal modelbehavior and motivate the development of tailored evaluation frameworks andmitigation strategies for personalized AI systems.
Code and dataset areavailable at: https://github.
com/3lis/misinfo_vlm
Published on arXiv on: 2025-05-19T16:20:54Z