Medical X ray image generation stable diffusion

提供商Osama03
分类image-generation
许可证openrail
下载量19
星标0

简介

这是一个基于 Stable Diffusion 微调的医学 X 光图像生成模型。它将通用图像生成的强大能力迁移到了医疗影像领域，能够根据文本描述合成具有医学特征的 X 光片。对于医疗 AI 开发者而言，它解决了医学数据脱敏难、标注成本高的问题，可用于快速构建数据集、训练医学图像识别模型或在教学演示中生成模拟病灶样本。上手门槛较低，只要熟悉 SD 生态的提示词工程即可快速出图，是弥补医学影像数据短缺的实用工具。

核心亮点

  • 将 Stable Diffusion 能力应用于医学 X 光合成
  • 有效解决医疗影像数据脱敏与获取成本问题
  • 适用于医学 AI 模型训练的增强数据集构建
  • 支持通过文本描述精准控制影像特征

使用方法

安装依赖
# 安装 Hugging Face transformers
pip install transformers torch
SDK 使用
# 使用 transformers 加载模型
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Osama03/Medical-X-ray-image-generation-stable-diffusion")
tokenizer = AutoTokenizer.from_pretrained("Osama03/Medical-X-ray-image-generation-stable-diffusion")

Hugging Face 下载

我们推荐使用命令行或者 Hugging Face Hub SDK 来进行模型的下载。

操作指引：在下载前，请先通过如下命令安装 huggingface_hub：

操作指引 
pip install -U huggingface_hub

命令行下载

下载完整模型库

下载完整模型库 
huggingface-cli download Osama03/Medical-X-ray-image-generation-stable-diffusion

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例）

下载单个文件到指定本地文件夹（以下载 config.json 到当前路径下 ./dir 目录为例） 
huggingface-cli download Osama03/Medical-X-ray-image-generation-stable-diffusion config.json --local-dir ./dir

更多命令行下载选项，可参见官方文档

SDK 下载

SDK 下载 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Osama03/Medical-X-ray-image-generation-stable-diffusion')

Git 下载

请确保 lfs 已经被正确安装

Git 下载 
git lfs install
git clone https://huggingface.co/Osama03/Medical-X-ray-image-generation-stable-diffusion

如果您希望跳过 lfs 大文件下载，可以使用如下命令

跳过 LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Osama03/Medical-X-ray-image-generation-stable-diffusion

模型文件托管在 Hugging Face Hub，使用 HF CLI / SDK / Git 直接下载，不经过本站。

PyTorch / Transformers 使用

安装 Transformers

安装 Transformers 
pip install -U transformers torch

模型加载和推理

模型加载和推理 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Osama03/Medical-X-ray-image-generation-stable-diffusion')
tokenizer = AutoTokenizer.from_pretrained('Osama03/Medical-X-ray-image-generation-stable-diffusion')

完整文档

来源: HuggingFace

---
license: openrail
language:

  • en

base_model:
  • CompVis/stable-diffusion-v1-4

pipeline_tag: text-to-image
library_name: diffusers
tags:
  • medical

  • X-ray

  • Diffusion

  • Generation

  • Text-to-image

  • stable-diffusion

  • lora

  • fine_tune

widget:
- text: >-
Hey doc, I've been feeling really out of breath lately,
especially when I'm walking up a flight of stairs or doing some light exercise.
It's like my chest gets tight and I can't catch my breath.
I've also been coughing up some stuff that's not quite right, it's been a few weeks now.
And I've noticed a bit of weight loss, I'm not sure if that's related but it's been on my mind.
I've been to a few doctors already, but they haven't been able to figure out what's going on.
I'm hoping you can help.
output:
url: example.png

---

Symptom-to-Medical-Image Generator

This project introduces a text-to-image diffusion model fine-tuned using LoRA (Low-Rank Adaptation) on top of CompVis/stable-diffusion-v1-4 for the task of medical image generation. The model generates X-ray, CT, or MRI scans based on natural language descriptions of patient symptoms, offering a novel way to visualize potential diagnostic outcomes.

---

What Is This Model?

This is a domain-adapted diffusion model tailored to generate realistic medical scans conditioned on symptom prompts. The model was fine-tuned using LoRA, which allowed for:

  • Efficient training without modifying the original model weights.
  • Adaptation to a smaller, highly-specialized medical dataset.
  • Retention of high-quality generative capabilities from the base model.

Key Features

  • Symptom-to-scan generation: Input symptoms in plain English and receive a plausible X-ray, CT, or MRI image.
  • Multi-modality support: Generate different types of scans (e.g., chest X-rays, brain MRIs) depending on the prompt context.
  • High realism: Outputs are visually realistic and follow anatomical structure, trained using real medical datasets.

---

When Can You Use This Model?

Use Cases

| Application Area | Description |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Medical Research | Generate datasets for hypothesis testing or model training without using real patient data. |
| Education & Training | Teach students about correlations between symptoms and imaging in an interactive way. |
| AI-Aided Prototyping | Test downstream diagnostic pipelines on synthetic but realistic image data. |
| Data Augmentation | Enrich datasets for training classification/segmentation models. |
| Prompt-Based Exploration | Investigate how changes in symptoms affect image generation (e.g., how “fever + cough” differs from “chest pain + shortness of breath”). |

Not for Use In:

  • Real-world clinical diagnosis or decision-making
  • Generating scans for real patients or influencing treatment
  • Bypassing ethical or regulatory controls in medical AI

---

Example Usage

Input Prompt:

> "I've been feeling really out of breath lately, especially when I'm walking up a flight of stairs or doing some light exercise. It's like my chest gets tight and I can't catch my breath. "

Output:

<img src="example.png" alt="Generated Chest X-ray" width="512"/>

> The model generates a chest X-ray image that corresponds to symptoms of a potential pulmonary issue.

---

Under the Hood

  • Base Model: CompVis/stable-diffusion-v1-4
  • Fine-tuning Method: LoRA (efficient, parameter-light adaptation)
  • Dataset: Custom dataset of symptom-to-image pairs, curated for medical imaging consistency
  • Framework: PyTorch + 🤗 Diffusers + Hugging Face Spaces

---

Ethical & Legal Disclaimer

This model is strictly intended for research and educational use. It is not a substitute for professional medical judgment. Use of synthetic medical images should follow all local regulatory and ethical guidelines.

---

查看官方来源