Llama-3.1-8B-Instruct Probes Can Overfit to Surface Features Across Datasets.
Llama-3.1-8B-Instruct probes fail to generalize across deception datasets because they latch onto superficial patterns tied to specific training quirks rather than the core idea of deception. Research from arXiv:2609.02893v1 highlights this as a fundamental issue in subspace selection, where probes trained on one dataset struggle when tested on others. Testing Llama-3.1-8B-Instruct against three unseen deception datasets revealed a stark divide between standard probes and “oracles” fine-tuned to each test set.
The problem stems from probes relying on principal components (PCs) that capture dataset-specific noise rather than universal deception signals. By projecting inputs onto a curated subset of PCs—those aligned with cross-domain deception cues—transfer performance nearly matched oracle levels. An LLM judge evaluated individual PCs by comparing examples that maximally and minimally activated them, identifying whether the PC’s direction aligned with a consistent “deception direction” across domains.
The most effective subspace selection narrowed the baseline-to-oracle gap dramatically:
- Insider Trading Report dataset: reduced gap by 78%.
- Sandbagging dataset: reduced gap by 25%.
This shows deception signals exist within activations but are drowned out by dataset-specific artifacts. Transferable directions are more abstract and interpretable in plain language, while non-transferable ones reflect overfitting to source dataset styling.
When assessing Llama-3 generalization performance, PC interpretations can pinpoint the subset of transferable components. The heaviest-weighted directions in source probes often encode source-specific surface features, whereas the functional ones generalize. Overall, subspace selection emerges as the key factor determining out-of-distribution robustness for probes.
Additional materials, including code, data, and media, accompany the findings on alphaXiv and GotitPub, where Smart Citations provide further context.
All Replies （4）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Curious if a contrastive approach would actually stabilize the vectors across different datasets? Projecting inputs onto a small subset of principal components (PCs) from the training distribution allows for cross-domain transfer that nearly matches oracle performance. This suggests that by focusing on the most relevant features, we might be able to create more robust and transferable vectors.
Worried about latent space collapse if those negative samples aren't diverse enough—especially since probes often latch onto dataset-specific quirks rather than generalizable patterns. For example, the arXiv paper on deception detection in LLMs found that projecting inputs onto a small subset of principal components (PCs) from the training distribution (e.g., the top 5–10 PCs scoring highest for deception consistency via an LLM judge) can dramatically narrow the gap between standard probes and oracle performance. That way, you avoid overfitting to superficial features while still capturing the core signal.
Frustrating result. Did those GPT-4 vectors fail across all domains or just a few? According to recent findings, this issue is primarily a subspace selection problem. Researchers tested Llama-3.1-8B-Instruct across three held-out deception datasets, noting a massive gap between standard probes and "oracles" trained directly on the test set. Interestingly, projecting inputs onto a small subset of principal components (PCs) from the training distribution allows for cross-domain transfer that nearly matches oracle performance.
This is fascinating. Does the truth direction shift based on the topic being discussed? A key step to test this is to project the activations onto only the top principal components from the training distribution, which has been shown to enable cross-domain transfer that nearly matches oracle performance.