LLMs Fail to Identify Subtle Social Prejudice Within Complex Group Conversations

Safety benchmarks for LLMs usually rely on static prompts like "Is this sentence offensive?" to determine if a model is safe. These tests ignore how audience presence and social pressure change communication in fluid human interactions. A model that passes a basic toxicity test may still fail when five simulated users argue in a group chat, making its actual safety questionable.

Social interaction context is largely missing from current evaluation methods. A ten-message thread can turn a neutral-looking sentence into a reinforced stereotype or a pointed microaggression. Monitoring group chats requires social intelligence, which is a far more difficult task than the simple classification of an isolated string of text.

SDARE-Bench addresses this gap by focusing on conversational stigma detection and response in group and dyadic dialogues. The benchmark uses a dataset of nearly 1,400 group dialogue scenarios and over 1,100 dyadic queries to see if models grasp social prejudice nuances. LLMs must track conversational flow to find marginalized individuals or groups through subtle cues instead of overt slurs.

Testing eight different LLMs revealed a consistent failure pattern. Models struggle significantly more with group dialogue than dyadic scenarios, showing that accuracy plummets as the number of participants and social complexity increase.

Missing stigma in group settings leads to responses that either validate biased perspectives or are too generic to help. Models often ignore stigma to maintain conversational fluidity when multiple simulated users lean into a stereotype, mirroring a "herd mentality" found in the data.

This blind spot exists because training for "helpfulness" and "harmlessness" has happened in a vacuum. A model remains a liability for deployment if it cannot tell a stigmatizing comment from a friendly joke when the context is hidden in a group chat.

Knowledge memorization in LLMs also relates to the frequency of information in pre-training datasets scraped from the web. While some data is ubiquitous, other pieces appear rarely. By entity linking pre-training datasets and counting documents containing the same entities as question-answer pairs, a strong causal and correlational relationship was found between accuracy and relevant document count. This applies to pre-training corpora with 176B parameters and datasets like TriviaQA. Using retrieval-augmentation can capture long-tail information and reduce the need for relevant pre-training data, as shown in the ICML 2023 Camera Ready Version.