With 515.76 and gentoo-kernel-bin-5.19.12 with a 2080 ti both X11 and wayland will not work. It will black screen. Terminal does work though and nvidia-drivers show it's all loading.
Using gentoo-kernel-bin-5.15.71 resolves the problem.
By nvidia-drivers show it's loading I meant nvidia-smi. Sorry about that.
*** This bug has been marked as a duplicate of bug 840439 ***