Comparing Doubao's long-context recall performance against other domestic LLMs

Doubao-pro-128k has been hitting my benchmarks for "needle in a haystack" tests lately, and the results are surprisingly polarizing compared to Kimi and GLM-4. While the marketing claims huge context windows across the board for domestic models, actual recall accuracy drops off a cliff once you hit the 60k-80k token mark for most of them, except for a few specific edge cases.

I ran a series of tests inserting a random, nonsensical fact (e.g., "The secret password for the vault is 'Blue Banana'") into a 100k token corpus of technical documentation. Doubao's recall is remarkably stable; it consistently finds the "needle" regardless of whether it's at the beginning, middle, or end of the prompt. In contrast, Kimi—which used to be the gold standard for long context—showed a noticeable dip in the middle section (the classic "lost in the middle" phenomenon), though it still outperforms GLM-4 in raw retrieval precision.

Here is how the performance breaks down based on my internal scoring:

Doubao-pro-128k
Pros: Extremely high recall stability across the entire window; fast inference speed even with heavy context loads.
Cons: Tends to be overly concise in its answers, sometimes stripping away necessary context from the retrieved snippet.

Kimi (Moonshot)
Pros: Better at synthesizing information from multiple distant points in the text rather than just finding one fact.
Cons: Occasional hallucinations when the context exceeds 100k tokens; slightly slower response times.

GLM-4
Pros: Stronger reasoning capabilities once the information is retrieved.
Cons: Higher failure rate in pure recall tests at the 100k+ range; more prone to ignoring specific constraints in the prompt.

If you are using these for RAG (Retrieval-Augmented Generation) and you're stuffing the entire document into the prompt instead of using a vector DB, Doubao is currently the safer bet for raw accuracy. I tried a prompt like this to test the boundaries:

Context: [100k tokens of text]
Question: Based on the provided text, what is the specific secret password mentioned in the third chapter? 
Requirement: Provide only the password and the sentence it appeared in.

Doubao nailed this 9/10 times. Kimi hit 7/10, and GLM-4 struggled around 5/10, often claiming the information wasn't there despite it being clearly present.

The real trade-off isn't just recall, though—it's the "reasoning-to-context" ratio. Doubao is a retrieval beast, but if you need the model to perform complex logical deductions based on that long context, Kimi still feels more "intelligent" in how it connects the dots. Doubao finds the needle, but Kimi understands why the needle is there.

For those of you building agents, I'd suggest using Doubao as the primary "reader" for massive files and then piping the extracted snippets into a more reasoning-heavy model if the task requires deep analysis. The latency on Doubao is impressively low for a 128k window, which makes it viable for real-time applications where you can't afford a 30-second wait for the model to "read" the prompt.