Just started using the Inter Display fonts and IBM Plex Mono fonts for my GNOME desktop.
Both are packaged in Debian.
There is even a discussion about making Inter the default font for GNOME: https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/52
Not sure whether it can fix the font problem, but in general Flatseal allows you to customise permissions for installed flatpaks.
https://flathub.org/apps/com.github.tchx84.Flatseal