4 points
As far as I know, Stable Diffusion is a far smaller model than Llama. The fact that a model as large as LLaMa can even run on consumer hardware is a big achievement.
2 points
*
2 points
*
Both SD 1.5 and SDXL run on 4g cards, you really want fp16 though.
In principle it should be possible to get decentish performance out of e.g. an RX480 by using the (forced) 32-bit precision to do bigger winograd convolutions (severely reducing the number of fmas needed) but don’t expect AMD to write kernels for that, ROCm is barely working on mid range cards in the first place.
Meanwhile, I actually ended up doubling my swap because 16G RAM are kinda borderline to merge SDXL models. OOM might kick in, it might not, and in any case your system is going to lock without earlyoom.
