OpenSSH contains a GTK-3 askpass program in the contrib directory which can easily be built using "make gnome-ssh-askpass3", see https://github.com/openssh/openssh-portable/blob/c9792783a98881eb7ed295680013ca97a958f8ac/contrib/Makefile#L16. I'm not sure if it still makes sense to add the option to build the GTK2 version as well. This should obviously live behind a USE flag, I guess that should be "gtk" not "gnome" since there are no dependencies on GNOME libraries? P.S. I've seen https://bugs.gentoo.org/5584 but that was closed a very long time ago, didn't think it made sense to comment there. Also that was about GTK-2, this is about the GTK-3 version for which there are no alternatives.
Note that this is now in GURU as net-misc/gnome-ssh-askpass (https://github.com/gentoo/guru/commit/f1c6442b9f3939e01510693fd5c1afad8eff6279).
@Sam James: Thanks for the heads up, using that now and it's working as expected. Thanks!