Link: http://arxiv.org/abs/2503.21464v1
PDF Link: http://arxiv.org/pdf/2503.21464v1
Summary: In this work, we propose a metric called Number of Thoughts (NofT) todetermine the difficulty of tasks pre-prompting and support Large LanguageModels (LLMs) in production contexts.
By setting thresholds based on the numberof thoughts, this metric can discern the difficulty of prompts and support moreeffective prompt routing.
A 2% decrease in latency is achieved when routingprompts from the MathInstruct dataset through quantized, distilled versions ofDeepseek with 1.
7 billion, 7 billion, and 14 billion parameters.
Moreover, thismetric can be used to detect adversarial prompts used in prompt injectionattacks with high efficacy.
The Number of Thoughts can inform a classifier thatachieves 95% accuracy in adversarial prompt detection.
Our experiments addatasets used are available on our GitHub page:https://github.
com/rymarinelli/Number_Of_Thoughts/tree/main.
Published on arXiv on: 2025-03-27T12:54:00Z