If you need this frequently, I really suggest you look into GPU forwarding. I have a Windows VM setup with a second card and it works perfectly, I use it for games and CAD all the time. Figure out your iommu groups, pop a second card in your computer (and optionally a second nvme drive if you want max performance), and use virt-manager and the arch wiki to set it up.

For accessing the machine you can use a second monitor input, or you can get a window to the machine with looking glass or moonlight. I use moonlight as it lets me play games from my laptop on the couch, and looking glass was causing windows to crash sometimes.

It’s a bit of work to set it all up but when you’re done it should just be one XML file and maybe one modprobe.d config file.

I think I’ve been using this for over a year now and the single pain point I encountered in all that time was maybe that usb input hotplug isn’t supported, though there’s ways to fix that, but I haven’t bothered.