In the course to reducing a box to MATE + LightDM + systemd with no trace of gdm, I used the following commands : emerge --deselect gdm emerge --depclean -a and I had the surprise to have all the graphics stack packages proposed for removal. After some thinking, I have done : emerge --noreplace xorg-server emerge --depclean -a and this time the list of package proposed was OK for me. Digging a bit in the xdm and lightdm ebuilds, I found no trace of dependency to xorg-server. It seemed to me that the gdm et kdm depend effectively upon xorg-server. I suggest to add at least to the xdm and lightdm ebuilds a dependency to xorg-server. I might be wrong.
The thing is, neither display manager actually needs a locally installed display server. Whether to have the display server on the same system is a decision for the system administrator to make.
This system administration decision could be made dynamically, as an exemple through a /etc.conf.d/ parameter, or statically through a package USE flag. The last way seems easy, just add a new USE flag "notlocal" and a line in the ebuild RDEPEND like : !notlocal? ( x11-base/xorg-server ) I might be wrong (TM).