Geometric Configurations: How Perturbed Jailbreaks Look to LLMs

String-level perturbations—the tiny tweaks that turn a blocked prompt into a successful jailbreak—aren't just random noise; they actually shift how a model perceives the input in its internal embedding space. A recent deep dive into Qwen-2.5 and Llama-3.2 models (ranging from 1B to 8B parameters) reveals exactly where these "successful" perturbed prompts land compared to those that trigger a refusal.

The research looked at two specific areas: the last-layer-last-token embedding space and the top-50 next-token probability space. The results are pretty telling for anyone interested in LLM security and prompt engineering.

Internal Representation Analysis

• Embedding Space: The last-layer embeddings tend to group prompts based on their surface-level characteristics—things like spelling errors, weird formatting, or specific character perturbations. It's more about the "look" of the prompt than the intent.
• Probability Space: The top-50 next-token space is functionally one-dimensional but way messier to cluster. It tracks what the model is likely to say next.

The most interesting finding is the lack of a "behavioral hyperplane." In simpler terms, there isn't a clean, mathematical line in these spaces that separates "refusal" from "compliance."

However, a few specific tokens act as massive tells for a successful bypass. In the Qwen-2.5-1.5B model, the token "Sure" is heavily associated with a compliant response. For Llama models, the comma (",") and the double newline ("ĊĊ") show a similar correlation.

This suggests that while the model might be "tricked" by the perturbation, the transition from refusal to compliance often hinges on a few critical pivot tokens that signal the model has entered a "helpful" state rather than a "safe" state. For those building a real-world AI workflow or researching LLM agents, this highlights just how fragile the boundary between a guarded response and a jailbroken one actually is.