I would avoid it, if you care at all about availability and downtime. The result will probably not be great, you need to ensure the server side gets enough resources under load, and setting it up may require constant restarts if things aren’t immediately working as expected.
Nonetheless, here is a link where someone did essentially exactly that on NixOS: https://astrid.tech/2022/09/22/0/nixos-gpu-vfio/
It’s remarkable that you took the time to write this essay just to reply to that post. Thank you for the effort and your insights, it was very interesting to read and I’m glad I stumbled across it!