Medical X ray image generation stable diffusion

ProviderOsama03
Categoryimage-generation
Licenseopenrail
Downloads19
Stars0

Overview

This Stable Diffusion fine-tune is engineered specifically for synthesizing medical X-ray imagery, bridging the gap between general latent diffusion and clinical visual requirements. For developers building diagnostic assistants or medical training software, this model provides a controlled way to generate synthetic datasets, reducing reliance on sensitive patient data while maintaining anatomical plausibility. It integrates seamlessly into existing Diffusers pipelines, allowing for precise prompt-based control over pathology simulation and anatomical positioning. Compared to base SD models, it minimizes 'hallucinated' textures and focuses on the grayscale contrast and structural fidelity essential for radiological visualization.

Highlights

  • Generates synthetically plausible medical X-ray imagery
  • Reduces dependency on sensitive private patient datasets
  • Full compatibility with standard Diffusers library pipelines
  • Optimized for radiological contrast and anatomical structure
  • OpenRail license allows for flexible commercial integration

Usage

Install
# Install Hugging Face transformers
pip install transformers torch
SDK Usage
# Load model with transformers
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("Osama03/Medical-X-ray-image-generation-stable-diffusion")
tokenizer = AutoTokenizer.from_pretrained("Osama03/Medical-X-ray-image-generation-stable-diffusion")

Hugging Face Download

We recommend downloading the model via the Hugging Face CLI or Hub SDK.

Guidance：Before downloading, install huggingface_hub with:

Guidance 
pip install -U huggingface_hub

CLI Download

Download the full repository

Download the full repository 
huggingface-cli download Osama03/Medical-X-ray-image-generation-stable-diffusion

Download a single file to a local folder (e.g. config.json into ./dir)

Download a single file to a local folder (e.g. config.json into ./dir) 
huggingface-cli download Osama03/Medical-X-ray-image-generation-stable-diffusion config.json --local-dir ./dir

See the official docs for more CLI options

SDK Download

SDK Download 
# 模型下载
from huggingface_hub import snapshot_download
model_dir = snapshot_download('Osama03/Medical-X-ray-image-generation-stable-diffusion')

Git Download

Make sure git-lfs is installed first

Git Download 
git lfs install
git clone https://huggingface.co/Osama03/Medical-X-ray-image-generation-stable-diffusion

To skip LFS large-file downloads, use:

Skip LFS 
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Osama03/Medical-X-ray-image-generation-stable-diffusion

Model files are hosted on the Hugging Face Hub — download directly via HF CLI / SDK / Git, not through this site.

PyTorch / Transformers Usage

Install Transformers

Install Transformers 
pip install -U transformers torch

Load the model and run inference

Load the model and run inference 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained('Osama03/Medical-X-ray-image-generation-stable-diffusion')
tokenizer = AutoTokenizer.from_pretrained('Osama03/Medical-X-ray-image-generation-stable-diffusion')

Full Documentation

来源: HuggingFace

---
license: openrail
language:

  • en

base_model:
  • CompVis/stable-diffusion-v1-4

pipeline_tag: text-to-image
library_name: diffusers
tags:
  • medical

  • X-ray

  • Diffusion

  • Generation

  • Text-to-image

  • stable-diffusion

  • lora

  • fine_tune

widget:
- text: >-
Hey doc, I've been feeling really out of breath lately,
especially when I'm walking up a flight of stairs or doing some light exercise.
It's like my chest gets tight and I can't catch my breath.
I've also been coughing up some stuff that's not quite right, it's been a few weeks now.
And I've noticed a bit of weight loss, I'm not sure if that's related but it's been on my mind.
I've been to a few doctors already, but they haven't been able to figure out what's going on.
I'm hoping you can help.
output:
url: example.png

---

Symptom-to-Medical-Image Generator

This project introduces a text-to-image diffusion model fine-tuned using LoRA (Low-Rank Adaptation) on top of CompVis/stable-diffusion-v1-4 for the task of medical image generation. The model generates X-ray, CT, or MRI scans based on natural language descriptions of patient symptoms, offering a novel way to visualize potential diagnostic outcomes.

---

What Is This Model?

This is a domain-adapted diffusion model tailored to generate realistic medical scans conditioned on symptom prompts. The model was fine-tuned using LoRA, which allowed for:

  • Efficient training without modifying the original model weights.
  • Adaptation to a smaller, highly-specialized medical dataset.
  • Retention of high-quality generative capabilities from the base model.

Key Features

  • Symptom-to-scan generation: Input symptoms in plain English and receive a plausible X-ray, CT, or MRI image.
  • Multi-modality support: Generate different types of scans (e.g., chest X-rays, brain MRIs) depending on the prompt context.
  • High realism: Outputs are visually realistic and follow anatomical structure, trained using real medical datasets.

---

When Can You Use This Model?

Use Cases

| Application Area | Description |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Medical Research | Generate datasets for hypothesis testing or model training without using real patient data. |
| Education & Training | Teach students about correlations between symptoms and imaging in an interactive way. |
| AI-Aided Prototyping | Test downstream diagnostic pipelines on synthetic but realistic image data. |
| Data Augmentation | Enrich datasets for training classification/segmentation models. |
| Prompt-Based Exploration | Investigate how changes in symptoms affect image generation (e.g., how “fever + cough” differs from “chest pain + shortness of breath”). |

Not for Use In:

  • Real-world clinical diagnosis or decision-making
  • Generating scans for real patients or influencing treatment
  • Bypassing ethical or regulatory controls in medical AI

---

Example Usage

Input Prompt:

> "I've been feeling really out of breath lately, especially when I'm walking up a flight of stairs or doing some light exercise. It's like my chest gets tight and I can't catch my breath. "

Output:

<img src="example.png" alt="Generated Chest X-ray" width="512"/>

> The model generates a chest X-ray image that corresponds to symptoms of a potential pulmonary issue.

---

Under the Hood

  • Base Model: CompVis/stable-diffusion-v1-4
  • Fine-tuning Method: LoRA (efficient, parameter-light adaptation)
  • Dataset: Custom dataset of symptom-to-image pairs, curated for medical imaging consistency
  • Framework: PyTorch + 🤗 Diffusers + Hugging Face Spaces

---

Ethical & Legal Disclaimer

This model is strictly intended for research and educational use. It is not a substitute for professional medical judgment. Use of synthetic medical images should follow all local regulatory and ethical guidelines.

---

View official source
Join our Telegram