Optimizing ComfyUI Workflow for Consistent Character LoRA Blending in SDXL

DeepSeek-V3 has become my go-to for writing the JSON logic and custom node expressions needed to stabilize SDXL LoRA blending in ComfyUI, mainly because Claude 3.5 Sonnet tends to be too conservative with weight suggestions. If you're struggling with "character bleed" or losing the likeness of a specific subject when mixing multiple LoRAs, the issue is almost always the interaction between the CLIP text encoder and the UNet weights.

Most people just slap two LoRAs in a chain and guess the weights, but that’s a recipe for fried pixels. I’ve been benchmarking a "Weighted Blend" approach against a "Sequential Application" approach. In my tests, applying LoRAs sequentially (LoRA A -> LoRA B -> KSampler) creates a cumulative effect where the second LoRA often overrides the first, especially if the first one is a strong character trigger.

To get actual consistency, I've shifted to using a "LoRA Stack" logic where I normalize the total weight to 1.0. For example, if I have a character LoRA and a style LoRA, I run them at 0.6 and 0.4 respectively. Anything over a combined 1.2 usually starts introducing that metallic sheen or contrast clipping common in SDXL.

The real breakthrough for me was using a specific prompt weighting syntax to anchor the character's identity while the LoRA handles the visual features. Instead of just listing the trigger word, I use a strength modifier. Here is the prompt structure that actually holds up across different samplers:

(masterpiece:1.2), (photorealistic:1.1), <lora:Char_Identity:0.7> [character_trigger_word:1.3], <lora:Style_Blend:0.3> cinematic lighting, highly detailed skin pores

When comparing the models for generating these complex prompt strings, Gemini 1.5 Pro is surprisingly good at understanding the spatial relationships for ComfyUI's regional prompting, but for raw weight optimization, DeepSeek-V3 gives me much more precise numeric suggestions for the LoRA strength based on the training epoch data.

The Performance Trade-offs:

Sequential LoRA Loading
Pros: Faster to set up, works for simple style overlays.
Cons: Heavy "over-baking" effect; the last LoRA in the chain dominates the image.

Normalized Weight Stacking
Pros: Much higher character fidelity; prevents the image from collapsing into a generic "AI look."
Cons: Requires manual tuning for every new LoRA combination; slower iteration.

If you're seeing the character's face morph when you add a clothing or background LoRA, try moving the character LoRA after the style LoRA in the node chain. It sounds counterintuitive, but in SDXL, the last applied weight has a disproportionate impact on the final latent space.

I also recommend swapping your sampler to dpmpp_2m_sde with karras for these blends. I've benchmarked this against euler_a, and the SDE variant handles the conflicting gradients of two different LoRAs much more gracefully, resulting in fewer artifacts around the edges of the character.