daimonin-client depends on libsdl. If you emerge libsdl without X support, you can't use diamonin-client with X no matter what you enable X use flag or not. Please remove virtual/x11 because it's no use and causes xorg to downgrade to 6.9 for modular X users.
You can also use built_with_use to abort if libsdl was not built with X support.
There doesn't seem to be much reason to depend on X there since the use flag isn't used explicitely anywhere else (unless it's in some eclass). Unless some sort of configure option is implemented, it may as well be removed.
There is more to consider here. Refer to bug 126351. Those wishing to build a headless system with no X11 at all should be able to build libsdl without X support. Disabling the X USE means that we can't get kernel sources without loading X! The alternative would be to make links load, by default, with -sdl if -X is present. This would actually make more sense to the average headless server builder, because generally links is used to browse system documentation text in those cases.
X use flag removed, thanks for report