“ Stability AI 应该叫 Open AI
Repository: https://github.com/Stability-AI/generative-models
Paper: https://stability.ai/research/adversarial-diffusion-distillation
Demo: http://clipdrop.co/stable-diffusion-turbo
特点
- SDXL Turbo使用了新的对抗扩散蒸馏(ADD)技术，使模型可以在单步中合成图像输出。
- 与其他蒸馏方法相比，ADD可以避免图像失真和模糊。
- 与多步骤模型相比，SDXL Turbo使用极少的步骤就可以达到状态最先进的性能。
- SDXL Turbo可以以207ms的速度生成512x512图像，大大提高了推理速度。
- SDXL Turbo目前以非商业研究许可发布，用户可以在Clipdrop平台上试用。
- 具体蒸馏方法见上论文地址，今天有个新的想法忙着在实验，我没具体看论文，不写解读了。
评估
用法不变
text_to_image
from diffusers import AutoPipelineForText2Image
import torch
pipe = AutoPipelineForText2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16")
pipe.to("cuda")
prompt = "A cinematic shot of a baby racoon wearing an intricate italian priest robe."
image = pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0.0).images[0]
image_to_image
from diffusers import AutoPipelineForImage2Image
from diffusers.utils import load_image
pipe = AutoPipelineForImage2Image.from_pretrained("stabilityai/sdxl-turbo", torch_dtype=torch.float16, variant="fp16")
init_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png").resize((512, 512))
prompt = "cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"
image = pipe(prompt, image=init_image, num_inference_steps=2, strength=0.5, guidance_scale=0.0).images[0]