From using upstream mesa through package.provided I can confirm that I've used gst-plugins-vaapi with mesa w/o X11 libraries with no apparent issues. The mesa[X] USE dependency might be changed to || ( mesa[X| mesa[wayland] ) ?
See also: bug 560096
This is still unchanged with 1.16.2. The X USE flag on mesa is unnecessary. I have a local overlay where the X USE flag is removed and everything works fine, as far as I can see.