GLM 5.
It’s a bizarre failure mode. Usually, when a model struggles with a coding task, you see "I cannot assist with this" or some generic refusal. In this case, the model didn't refuse—it just pivoted. Instead of analyzing the DOM tree or suggesting a better BeautifulSoup selector, it started generating text that felt less like a technical assistant and more like a political commentator.
The context of the prompt was purely technical:
import requests
from bs4 import BeautifulSoup
def scrape_data(url):
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# The following line is failing to find the target element
data = soup.find('div', {'class': 'content-wrapper'}).text
return data
# Error: AttributeError: 'NoneType' object has no attribute 'text'The error I was seeing was a standard AttributeError: 'NoneType' object has no attribute 'text', which happens when the selector fails. A standard deployment of a coding-tuned model should have immediately identified that the class content-wrapper likely wasn't present or was being blocked by anti-bot measures.
Instead, the model's internal weights seemed to trigger a massive shift in persona. It started outputting paragraphs of political discourse that had zero correlation with the Python logic or the scraping error.
I've seen models get "stuck" in certain safety guardrail loops before, but this felt different. It wasn't a refusal; it was a total hallucination of intent. It’s as if the prompt triggered a specific training data cluster that overrides the instruction-following capabilities of the model.
For anyone working on a production AI workflow, this is a massive red flag. If you are using a lightweight model like a "Flash" variant for automated tasks, you have to account for this kind of catastrophic drift. If this happened in an automated pipeline, the parser would have crashed or, worse, saved political nonsense into a database where structured data was expected.
I haven't been able to replicate this specific "political pivot" with Claude or GPT-4o yet, but it makes me wonder how much "alignment" training actually interferes with the raw reasoning capabilities of these smaller, faster models. When you optimize for speed and efficiency in an LLM agent, do you accidentally make the steering more fragile?
If you are running GLM models locally or via Ollama, I'd suggest keeping a very close eye on your system prompts. You might need to implement much stricter output validation or temperature controls to prevent the model from wandering off into these non-technical rabbit holes.