Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 380907 - x11-drivers/xf86-input-vmmouse-12.7.0: /usr/bin/vmmouse_detect segfaults during boot
Summary: x11-drivers/xf86-input-vmmouse-12.7.0: /usr/bin/vmmouse_detect segfaults duri...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Tiziano Müller (RETIRED)
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2011-08-28 06:00 UTC by Faustus
Modified: 2017-01-26 05:56 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch accepted in Debian and Redhat (enable-detect-in-kvm.diff,772 bytes, patch)
2012-07-11 11:59 UTC, Faustus
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Faustus 2011-08-28 06:00:00 UTC
vmmouse_detect causes a segmentation fault during boot (in udev) in non-VMWare guest, and also fails in KVM. The solution reached in other distros is to return the iopl(3) call that was present in previous versions.

See the following discussions:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=525039
https://bugzilla.novell.com/show_bug.cgi?id=604966
https://bugzilla.redhat.com/show_bug.cgi?id=604660
Comment 1 Faustus 2012-07-11 11:56:38 UTC
Hi, any feedback? It's a simple patch that should also make xf86-input-vmmouse work in kvm (an intended functionality of the package):


https://bugzilla.novell.com/show_bug.cgi?id=604966#c15

> Anyway, the issue here was slightly convoluted. The standard detection
> mechanism we use has always been to do this port-poke, and if it's
> not a VM, you get a segfault - and you do need iopl() set to allow that
> to work. When I published the vmmouse source, I had the iopl() call in
> there and that was an issue for non-Linux operating systems, but I also
> observed that it wasn't really necessary because the X server did iopl()
> itself - not really a surprise. Later on, I added vmmouse_detect to allow
> the HAL/udev based device detection to work, and those are standalone and
> so what the X server does is irrelevant.
> 
> Now, why isn't this an issue for anyone else? It does the detection
> correctly on Ubuntu and other distros (and no one cares about the segfault
> in the failure case). I guess they have funny core handling going on?
> 
> Anyway, it seems the right fix is to add the iopl() call in, perhaps only
> in vmmouse_detect as it's still irrelevant in the X server. It also doesn't
> matter in the X server now as the driver isn't loaded unless the device is
> detected.
> 
> It also needs to be properly guarded for LINUX. Who knows what's up on BSD
> or similar.


https://bugzilla.novell.com/show_bug.cgi?id=604966#c28

I did some tests here and I'd like to share the results:

physical# strace -e trace=iopl,exit_group vmmouse_detect.orig
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
exit_group(1)

physical# strace -e trace=iopl,exit_group vmmouse_detect.iopl
iopl(0x3)                               = 0
exit_group(1)                           = ?

kvm-guest# strace -e trace=iopl,exit_group vmmouse_detect.orig
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
exit_group(1)                           = ?

kvm-guest# strace -e trace=iopl,exit_group vmmouse_detect.iopl
iopl(0x3)                               = 0
exit_group(0)                           = ?

vmware-guest# strace -e trace=iopl,exit_group vmmouse_detect.iopl
iopl(0x3)                               = 0
exit_group(0)                           = ?

As you can see, vmmouse_detect without the iopl() call also doesn't work on KVM
guests. This problem was fixed on Debian with the same patch:


Patch: http://bugzillafiles.novell.org/attachment.cgi?id=366092
Comment 2 Faustus 2012-07-11 11:59:50 UTC
Created attachment 317908 [details, diff]
Patch accepted in Debian and Redhat
Comment 3 Chí-Thanh Christopher Nguyễn gentoo-dev 2012-07-11 14:29:16 UTC
What is the upstream status of this patch?
Comment 4 Faustus 2012-07-11 14:38:11 UTC
(In reply to comment #3)
> What is the upstream status of this patch?

I understand that it will not be integrated upstream, because they don't want to put in Linux-specific code. But upstream is ok with distributions putting the patch in, as the referenced threads show.

http://cgit.freedesktop.org/xorg/driver/xf86-input-vmmouse/tree/tools/vmmouse_detect.c
Comment 5 Chí-Thanh Christopher Nguyễn gentoo-dev 2012-07-11 14:42:08 UTC
The reason I am asking about upstream is because we are not keen on maintaining such a patch indefinitely.

> it will not be integrated upstream, because they don't want to put in Linux-specific code.

So what? they could just #ifdef LINUX or similar.
Comment 6 Faustus 2012-07-11 14:52:20 UTC
(In reply to comment #5)
> So what? they could just #ifdef LINUX or similar.

Well, the (weird) impression I got from the threads is that no one wanted to bother the upstream with a request to put back code that was just removed, in an #ifdef or otherwise:
http://cgit.freedesktop.org/xorg/driver/xf86-input-vmmouse/commit/tools/vmmouse_detect.c?id=bcdec3d0cd4434770cd841c33c030e0d7203881f

Maybe the reason was that it became clear that the patch is actually necessary (not just advantageous) on Linux only after the developer was contacted about it. Perhaps you want to contact him again, as a Gentoo developer:

Philip Langdale <philipl@fido2.homeip.net>
Comment 7 Faustus 2012-07-11 14:54:13 UTC
It's a very small file to be patched, too -- unlikely to need any maintenance. Maybe that contributed to fast patch acceptance in Debian and Redhat.
Comment 8 Matt Turner gentoo-dev 2017-01-26 05:56:08 UTC
Looks like this was fixed in upstream commit

commit d5c1493b5fe0871f4f2823125f0d49f8e1edcda7
Author: Thomas Hellstrom <thellstrom@vmware.com>
Date:   Thu Jan 15 10:14:30 2015 +0100

    vmmouse: Run vmmouse_detect as an io privileged process

which is included in version 13.1.0.