Ace Step1.5 XL DF11 ComfyUI

提供商mingyi456
分类text-to-audio
许可证mit
下载量9.7K
星标0

简介

Ace Step1.5 XL DF11 是一款专为 ComfyUI 生态设计的文本转音频（Text-to-Audio）模型。它将高质量的音频生成能力集成到了节点式工作流中，让开发者能够像控制图像生成一样，通过精准的参数调节来掌控声音的质感与节奏。相比于传统的独立音频工具，它更强调与 AI 创意管线的深度融合，适合需要批量生成音效、为 AI 视频配音或进行音频实验的创作者。上手难度较低，只要熟悉 ComfyUI 的基本逻辑，即可快速搭建一套从文本到音频的自动化生产链路。

核心亮点

  • 深度集成 ComfyUI，支持节点式音频工作流
  • 高效文本转音频，精准还原场景音效
  • MIT 协议开源，方便开发者二次开发与部署
  • 极低上手门槛，适配 AI 视频创作配音场景

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("mingyi456/Ace-Step1.5-XL-DF11-ComfyUI")
tokenizer = AutoTokenizer.from_pretrained("mingyi456/Ace-Step1.5-XL-DF11-ComfyUI")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download mingyi456/Ace-Step1.5-XL-DF11-ComfyUI config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/mingyi456/Ace-Step1.5-XL-DF11-ComfyUI

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')
tokenizer = AutoTokenizer.from_pretrained('mingyi456/Ace-Step1.5-XL-DF11-ComfyUI')

完整文档

来源: HuggingFace

---
license: mit
language:

  • en

  • zh

pipeline_tag: text-to-audio
tags:
  • comfyui

  • diffusion-single-file

base_model:
  • ACE-Step/acestep-v15-xl-sft

  • ACE-Step/acestep-v15-xl-base

  • ACE-Step/acestep-v15-xl-turbo

base_model_relation: quantized
---
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11

Feel free to request for other models for compression as well, although models whose architecture I am unfamiliar with might be slightly tricky for me.

How to Use

#### ComfyUI
Install the ComfyUI DFloat11 Extended node via the ComfyUI manager. After installing, simply replace the "Load Diffusion Model" node of an existing workflow with the "Load Diffusion Model" node. If you run into any issues, feel free to leave a comment.

#### Official implementation
This is coming soon, but I suspect that these existing compressed weights might be compatible out-of-the-box with the official implementation.

Compression Details

This is the pattern_dict for compression:

python
pattern_dict_comfyui = {
    r"decoder\.time_embed": (
        "linear_1",
        "linear_2",
        "time_proj",
    ),
    r"decoder\.time_embed_r": (
        "linear_1",
        "linear_2",
        "time_proj",
    ),
    
    r"decoder\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "cross_attn.q_proj",
        "cross_attn.k_proj",
        "cross_attn.v_proj",
        "cross_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    
    r"encoder\.lyric_encoder\.layers\.\d++": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"encoder\.timbre_encoder\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"tokenizer\.attention_pooler\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
    r"detokenizer\.layers\.\d+": (
        "self_attn.q_proj",
        "self_attn.k_proj",
        "self_attn.v_proj",
        "self_attn.o_proj",
        "mlp.gate_proj",
        "mlp.up_proj",
        "mlp.down_proj",
    ),
}
查看官方来源