4、stable diffusion

github

安装anaconda环境

conda env create -f environment.yaml
conda activate ldm

安装依赖

conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
pip install transformers==4.19.2 diffusers invisible-watermark
pip install -e .

安装xformers加速
xformers安装包
找到符合自己python，pytorch和cuda版本的xformers安装包

wget https://anaconda.org/xformers/xformers/0.0.16/download/linux-64/xformers-0.0.16-py39_cu11.3_pyt1.12.1.tar.bz2
conda install xformers-0.0.16-py310_cu11.3_pyt1.12.1.tar.bz2
python -m xformers.info

安装成功为下述图片

下载预训练模型权重
文生图
SD2.1-v
Image Inpainting with Stable Diffusion
512-inpainting-ema.ckpt

从Hugging Face下载encoding
当我们运行scripts/txt2img.py时，会发现还需要从hugging face下载encoding，但是国内无法下载，这时候我们会科学上网，同时安装下述依赖，这样就可以下载了

os.environ["http_proxy"] = "http://ip:7890"
os.environ["https_proxy"] = "https://ip:7890"pip install urllib3==1.25.11
pip install requests==2.27.1

文生图

python scripts/txt2img.py 
--prompt
"a professional photograph of an astronaut riding a horse"
--ckpt
/devdata/chengan/stablediffusion/v2-1_768-ema-pruned.ckpt
--config
configs/stable-diffusion/v2-inference-v.yaml
--H
768
--W
768
--device 
cuda

注意： --device 默认为cpu，要换成 cuda，否则报错

RuntimeError: expected scalar type BFloat16 but found Float

图生图

python scripts/gradio/inpainting.py configs/stable-diffusion/v2-inpainting-inference.yaml /devdata/chengan/stablediffusion/512-inpainting-ema.ckpt