Fix: Ideogram 4.

RileyCoder Novice 1h ago 429 views 12 likes 2 min read

I've been chasing this for three days. Running Ideogram 4.0 locally through ComfyUI, every multi-character prompt either renders garbled text or drops characters entirely. The official examples work, mine don't — turns out it's a quantization mismatch in the custom node wrapper.

Fix: Ideogram 4.

The Actual Error

When invoking the Ideogram node with spatial prompts like "A sign reading 'OPEN' hanging above a door", the server log spat this out:

[ERROR] Text encoding failed: expected hidden_states dim [1, 256, 1024], got [1, 203, 1024]
[WARN]  Bounding box projection skipped — token alignment broken

The node loads the model fine, but the text encoder's token padding length doesn't match what the unet expects. This kills spatial bounding boxes and any prompt with more than ~4 words.

Diagnosis

I compared the custom node's model_config.json against Ideogram's reference implementation. Two issues:

1. The node forces max_position_embeddings: 256 while the local checkpoint was trained with 203
2. The tokenizer pads to the config value instead of detecting the actual sequence length

Swapping to dynamic padding based on the real token count fixed the dimension mismatch. I also had to patch the bounding box sampler to re-align tokens after truncation.

Steps That Worked

1. Clone the node into ComfyUI/custom_nodes/
2. Edit nodes/ideogram_node.py — replace the static padding with:

# Before (broken)
encoded = tokenizer(prompt, padding="max_length", max_length=256, truncation=True)

# After (fixed)
encoded = tokenizer(prompt, padding="longest", truncation=True, return_tensors="pt")
max_len = encoded.input_ids.shape[1]
# Resize position embeddings if needed
model.resize_position_embeddings(max_len)

3. In model_config.json, set "max_position_embeddings": 203 to match the checkpoint
4. Restart ComfyUI — the node now preserves character counts and bounding box alignment

What Still Isn't Perfect

The local run is noticeably slower than the cloud API (batch size 1 on a 4090 takes ~18s vs ~6s). Also, I haven't cracked exact spatial prompt syntax yet — "text above image" sometimes renders below. If anyone has the correct token-bounding format for local Ideogram 4.0, I'm all ears.

For now, this patch delivers clean text rendering and functional bounding boxes on consumer hardware. I'll push the fork once I clean up the sampler fix.

Help Wanted
Hands-on notes on AI tools and LLMs are collected in a library of Claude prompt techniques, with plenty of directly applicable cases.

All Replies (3)

T
Taylor27 Intermediate 1h ago
Might be worth checking your prompt length—Ideogram 4 silently truncates anything over 150 tokens.
0 Reply
C
CameronCat Intermediate 1h ago
I hit the same wall—turns out batching too many characters at once makes the model lose track. Spacing them out across separate generations usually fixes it.
0 Reply
R
RayTinkerer Novice 1h ago
Are you passing the prompt through the text encode node before Ideogram? Sometimes the tokenizer splits multi-character names.
0 Reply

Write a Reply

Markdown supported