i use ffmpeg version 3.2.2 on my ~amd64-system and when trying to play a file ffplay fails with: "Could not initialize SDL - No available video device (Did you set the DISPLAY variable?)" i tested this with mkv, avi, ogv, mp4 and mp3 files. ffmpeg was compiled with USE=sdl" and libsdl-1.2.15-r9 (and ffmpeg) was recompiled recently. (please see my forum-post: https://forums.gentoo.org/viewtopic-p-8022458.html )
for another pkg (media-video/mpv-0.23.0-r1) i had to recompile libsdl2 with USE="X" and by accident i noticed that ffplay runs again. after a quick look it seems to me that there is quite a mixup of libsdl and libsdl2-useflags since USE="sdl" for mpv seems to pull in media-libs/libsdl2 with USE="X" but for ffmpeg media-libs/libsdl. recompiling ffmpeg did not solve that dependency before.
I'm not sure if USE=X is mandatory for having display support in libsdl2; it can probably run on top of the framebuffer or some embedded device display driver. CCing libsdl2 maintainers for input.
dawgg, mpv's dependency is libsdl2[X?], which means it'll only require X when mpv itself is set to use X. I don't believe ffmpeg requires an X flag as playing videos is not its primary purpose and even then, we don't want fbdev,wayland,X flags on every package using SDL. mpv is a special case as it has its own X-specific functionality. If you are running an X-based desktop system then you should be using a desktop Portage profile, which will enable the X flag by default. I suggest this be closed as WONTFIX. As for libsdl2's output choices, it can do X or Wayland. Framebuffer support was dropped but there is a fork that is trying to get it restored officially.
mpv is set to use X; i use a desktop-profile (forgot to update its symlink on this new install; THX for reminding me) but activate X only on pkgs that i want to run under X. i use ffplay under X with one specific filter which needs (works best with?; i haven't tested anything else) X: -vf "cropdetect=A:B:C" this issue seemed courious to me. i keep ffmpeg [X sdl ...] because other pkgs depend on it; to me the issue seems fixed.