Angel sucking at life, via stable diffusion via amd raedon vii
its tricky to set up an AMD gpu for use with pytorch
I remember a few years ago my co-collaborator on a poker ai project told me DO NOT buy AMD gpus because the tooling was "bleeding edge". Me being the contrarian I am, thought "Ah, but I'll be better for figuring out how to get it to work!". Nope. It turns out my Raedon VII would just be used for the 12 inch monitor that I use for bios on my Ubuntu Server box.
Fast forward to the last half of 2022. Now with an Nvidia GPU, I thought one weekend, "why don't I brush off that old Raedon VII and see if I can get it to work with pytorch"? And after nearly bricking my computer, the answer was still, Nope.
Here we are in October of 2023. I've been watching the growing tinygrad project since 2020. (Ironically, I had a small contribution in 2020 due to some missing wheel for pyopencl).
Tinygrad is dope
I love the idea behind Tinygrad. From their github:
This may not be the best deep learning framework, but it is a deep learning framework.
Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. If XLA is CISC, tinygrad is RISC:ALEReset
And here's a great video from the Lex and George Hotz interview (highly recommend the interview)
So, the main idea for Tinygrad is simplicity, and I love simplicity. Also, I can get behind the idea of making it easier for non-Nvidia gpus to be used for deeplearning.
I decided to give AMD another go but using Tinygrad as the NN framework.
Setting up the AMD drivers with OpenCL
- here are the drivers for the Raedon VII for Ubuntu 22.04.3
- here are the drivers for other AMD gpus for various OS/distributions.
- here is the set up for the amd gpu drivers. I would highly recommend reading that for all the nuances relating to having had installed a amd drivers before, and other edge cases.
In my case for ubuntu, you get a .deb file. You can get this installed a couple of ways, but I chose:
sudo dpkg -i amdgpu-install_5.7.50700-1_all.deb
Now, you have the amdgpu-install cli.
amdgpu-install -h
Now, tinygrad has an OpenCL accelerator and amdgpu-install has an OpenCL component.
Since all we require is opencl, we can just run
amdgpu-install --usecase=opencl --opencl=rocr
Now you will have to see if your AMD gpu is before or after vega 10. Since the Raedon VII is vega 20, I use rocm, but before vega 10, you use --opencl=legacy. See here for more details
Now you should be able to run
clinfo
Now, you may see that the number of devices is 0. This is likely because you need to change some permissions for OpenCL. That should sort you! You may have to reboot if you get stuck.
Running tinygrad with the GPU (OpenCL) accelerator
Cool, so now just install.
And voila!
GPU=1 python3 examples/stable_diffusion.py --prompt "angel sucking at life"
The GPU env var, when set to one, will use any OpenCl device available.