Ace Step1.5 XL DF11 ComfyUI

Providermingyi456
Categorytext-to-audio
Licensemit
Downloads9.7K
Stars0

Overview

Ace Step1.5 XL DF11 is a specialized text-to-audio model optimized for the ComfyUI ecosystem. Unlike general-purpose audio generators, this model focuses on high-fidelity synthesis, allowing developers to integrate complex audio generation into node-based visual workflows. It is particularly useful for game developers and multimedia engineers who need to automate sound effect production or voice synthesis within a modular pipeline. By leveraging the ComfyUI framework, it offers better transparency into the generation process and easier iteration compared to closed-API solutions. The MIT license ensures flexibility for both commercial and open-source integration, making it a lightweight yet powerful addition to any generative AI stack.

Highlights

  • Native ComfyUI integration for modular audio workflows
  • High-fidelity text-to-audio synthesis for diverse use cases
  • Permissive MIT license for commercial application
  • Streamlined pipeline for rapid sound effect iteration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("mingyi456/Ace-Step1.5-XL-DF11-ComfyUI")
tokenizer = AutoTokenizer.from_pretrained("mingyi456/Ace-Step1.5-XL-DF11-ComfyUI")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download mingyi456/Ace-Step1.5-XL-DF11-ComfyUI config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')
tokenizer = AutoTokenizer.from_pretrained('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')

Full Documentation

来源: HuggingFace

---
license: mit
language:

  • en

  • zh

pipeline_tag: text-to-audio
tags:
  • comfyui

  • diffusion-single-file

base_model:
  • ACE-Step/acestep-v15-xl-sft

  • ACE-Step/acestep-v15-xl-base

  • ACE-Step/acestep-v15-xl-turbo

base_model_relation: quantized
---
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11

Feel free to request for other models for compression as well, although models whose architecture I am unfamiliar with might be slightly tricky for me.

How to Use

#### ComfyUI
Install the ComfyUI DFloat11 Extended node via the ComfyUI manager. After installing, simply replace the "Load Diffusion Model" node of an existing workflow with the "Load Diffusion Model" node. If you run into any issues, feel free to leave a comment.

#### Official implementation
This is coming soon, but I suspect that these existing compressed weights might be compatible out-of-the-box with the official implementation.

Compression Details

This is the pattern_dict for compression:

python
pattern_dict_comfyui = {
    r"decoder\.time_embed": (
        "linear_1",
        "linear_2",
        "time_proj",
    ),
    r"decoder\.time_embed_r": (
        "linear_1",
        "linear_2",
        "time_proj",
    ),
    
    r"decoder\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "cross_attn.q_proj",
        "cross_attn.k_proj",
        "cross_attn.v_proj",
        "cross_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    
    r"encoder\.lyric_encoder\.layers\.\d++": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"encoder\.timbre_encoder\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"tokenizer\.attention_pooler\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"detokenizer\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
}
View official source
Join our Telegram