Totem doesn't require these package to be compiled with Xorg support when building or running in Wayland environment: >=media-libs/gst-plugins-base-1.6.0:1.0[X] nor >=x11-libs/gtk+-3.19.4:3[X]
It's not optional. Look for gdk-x11 in meson.build.
Well, technically gst-plugins-base[X] might not be needed, but given that totem still requires gtk+[X] and libX11, it's best to ensure xvimagesink is present too.