Language gaps are the biggest loophole in current LLM safety

A recent analysis of Qwen2.5-7B reveals a striking disparity: the model refuses harmful prompts in Turkish roughly three times less often than it does in English. This proves that while the ability to refuse is baked into the model's weights (language-agnostic), the trigger for that refusal is heavily dependent on the language used during the alignment phase (language-specific).

The technical nuance here is that "refusal directions" exist in the model's latent space regardless of the input language. If you extract the refusal vector from a Turkish prompt, it actually transfers across to other languages. This means the model "knows" how to refuse, but the safety guardrails aren't consistently activated when the input isn't in a high-resource language like English.

For anyone doing a deep dive into LLM agent security or red-teaming, this highlights a massive vulnerability. Most safety tuning happens on English datasets, creating a "safety shadow" for other languages. You can essentially bypass complex alignment by simply shifting the linguistic context, even if the underlying logic of the request remains harmful.

This makes the case for more diverse, multilingual alignment rather than just translating English safety prompts. If the refusal mechanism is universal but the activation is fragmented, the system is only as secure as its weakest language pair.