Anam cara-4: Real-time emotional avatars

Instead of just generating speech, their LLM interleaves emotional cues like [laughter] or [warm] directly into the text stream. This feeds into a diffusion transformer that translates audio and text into specific motion embeddings—handling head pose, gaze, and lip shapes—before a rendering model applies it to a reference image. This two-stage setup is clever because it avoids the need to fine-tune a new model every time you want to change the character's face.

I looked at their latency benchmarks, and while the total end-to-end delay (user speech to first video frame) sits at a ~1.2s median, the actual model inference is only ~100ms. The rest of that lag is the usual suspects: STT, LLM, and TTS overhead. If you're building real-time agents, that's the bottleneck you actually need to optimize.

They actually ran a blind study with 200 participants via Mabyduck to see if this actually felt "human." According to their data, cara-4 beat out competitors in lip-sync accuracy and overall "naturalness."

Is the emotional signaling via text cues enough to make an LLM agent actually feel empathetic, or is it just a gimmick to mask the latency? If you want to test the responsiveness yourself, you can try it at:

https://anam.ai