Bug 537612 – app-emulation/spice with app-emulation/qemu-2.1.2 - when trying to debug any program with Ida inside a Windows 7 VM, segmentation fault in spice_char_device_write_to_device (dev=dev@entry=0xfd83fd843d0) at char_device.c:443

• Status: RESOLVED TEST-REQUEST
• Product: Gentoo Linux

[Description Dejan Lukan 2015-01-25 00:04:11 UTC]

Step to reproduce:

1) USE=spice emerge -1 =app-emulation/qemu-2.1.2-r2
2) Start Windows 7 VM by using the following command:
# /usr/bin/qemu-system-x86_64 \
  -machine accel=kvm \
  -cpu core2duo         \
  -name Windows7 \
  -drive file=windows7.qcow2,if=ide,cache=writeback \
  -m 1G \
  -enable-kvm \
  -monitor stdio \
  -smp 1,sockets=1,cores=1,threads=1 \
  -nodefaults \
  -usbdevice tablet \
  -rtc base=localtime \
  -vga qxl \
  -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on \
  -device virtio-serial-pci \
  -device virtserialport,chardev=spicechannel0,name=com.redhat.spice.0 \
  -chardev spicevmc,id=spicechannel0,name=vdagent \
  -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
3. Once the VM boots, login to Windows, start putty.exe and Ida debugger. When attaching to putty.exe process by using the local windows debugger, the VM will crash.


The segmentation fault occurs in char_device.c at line 443 as presented below.

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000694bc83687dc in spice_char_device_write_to_device (dev=dev@entry=0xfd83fd843d0) at char_device.c:443
443         sif = SPICE_CONTAINEROF(dev->sin->base.sif, SpiceCharDeviceInterface, base)

The instructions at the 443 line are the following:

(gdb) list 443
438         }
439
440         spice_char_device_state_ref(dev);
441         core->timer_cancel(dev->write_to_dev_timer);
442
443         sif = SPICE_CONTAINEROF(dev->sin->base.sif, SpiceCharDeviceInterface, base);
444         while (dev->running) {
445             uint32_t write_len;
446
447             if (!dev->cur_write_buf) {

The dev->sin is NULL, which cannot be dereferenced in "dev->sin->base.sif" above, which is why a SEGFAULT occurs.

(gdb) print dev
$6 = (SpiceCharDeviceState *) 0xfd83fd843d0
(gdb) print *dev
$7 = {running = 1, active = 0, wait_for_migrate_data = 0, refs = 3, write_queue = {prev = 0xfd83fd843e0, next = 0xfd83fd843e0}, write_bufs_pool = {prev = 0xfd83fd842e0, 
    next = 0xfd83fd842e0}, cur_write_buf = 0x0, cur_write_buf_pos = 0xfd83fd844a0 "\002", write_to_dev_timer = 0xfd83fd88220, num_self_tokens = 1, clients = {
    prev = 0xfd83fd84360, next = 0xfd83fd84360}, num_clients = 1, client_tokens_interval = 5, sin = 0x0, during_read_from_device = 0, cbs = {
    read_one_msg_from_device = 0x694bc83aa673 <vdi_port_read_one_msg_from_device>, ref_msg_to_client = 0x694bc83a8780 <vdi_port_ref_msg_to_client>, 
    unref_msg_to_client = 0x694bc83aa069 <vdi_port_unref_msg_to_client>, send_msg_to_client = 0x694bc83abddd <vdi_port_send_msg_to_client>, 
    send_tokens_to_client = 0x694bc83a946d <vdi_port_send_tokens_to_client>, on_free_self_token = 0x694bc83ad403 <vdi_port_on_free_self_token>, 
    remove_client = 0x694bc83a8ccf <vdi_port_remove_client>}, opaque = 0x0}

The backtrace of every thread:


(gdb) thread apply all bt full

Thread 4 (Thread 0x694bbb17f700 (LWP 24088)):
#0  0x0000694bcaaa9331 in sem_timedwait () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00000fd83cc53315 in qemu_sem_timedwait (sem=sem@entry=0xfd83fa05798, ms=ms@entry=10000) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/util/qemu-thread-posix.c:257
        rc = <optimized out>
        ts = {tv_sec = 1422143062, tv_nsec = 478178000}
        __func__ = "qemu_sem_timedwait"
#2  0x00000fd83cbf0dcb in worker_thread (opaque=0xfd83fa05700) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/thread-pool.c:96
        req = <optimized out>
        ret = <optimized out>
        pool = 0xfd83fa05700
#3  0x0000694bcaaa3013 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4  0x0000694bc71ff6ad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 3 (Thread 0x694bb97ff700 (LWP 24090)):
#0  0x0000694bc71f602d in poll () from /lib64/libc.so.6
No symbol table info available.
#1  0x0000694bc83a8110 in poll (__timeout=<optimized out>, __nfds=20, __fds=0x694bb00008f8) at /usr/include/bits/poll2.h:46
No locals.
#2  red_worker_main (arg=<optimized out>) at red_worker.c:11994
        i = <optimized out>
        num_events = <optimized out>
        timers_queue_timeout = 7626
        worker = 0x694bb00008c0
        __FUNCTION__ = "red_worker_main"
#3  0x0000694bcaaa3013 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4  0x0000694bc71ff6ad in clone () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---
No symbol table info available.

Thread 2 (Thread 0x694bba57c700 (LWP 24089)):
#0  0x0000694bcaaa70ac in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x00000fd83cc530f8 in qemu_cond_wait (cond=<optimized out>, mutex=mutex@entry=0xfd83d106ee0 <qemu_global_mutex>)
    at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/util/qemu-thread-posix.c:135
        err = <optimized out>
        __func__ = "qemu_cond_wait"
#2  0x00000fd83c955d04 in qemu_kvm_wait_io_event (cpu=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/cpus.c:843
No locals.
#3  qemu_kvm_cpu_thread_fn (arg=0xfd83fcd2450) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/cpus.c:879
        cpu = 0xfd83fcd2450
        r = <optimized out>
#4  0x0000694bcaaa3013 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#5  0x0000694bc71ff6ad in clone () from /lib64/libc.so.6
No symbol table info available.

Thread 1 (Thread 0x694bcdf5a900 (LWP 24087)):
#0  0x0000694bc83687dc in spice_char_device_write_to_device (dev=dev@entry=0xfd83fd843d0) at char_device.c:443
        sif = <optimized out>
        total = 0
        n = <optimized out>
        __FUNCTION__ = "spice_char_device_write_to_device"
#1  0x0000694bc83695b8 in spice_char_device_start (dev=0xfd83fd843d0) at char_device.c:798
        __FUNCTION__ = "spice_char_device_start"
#2  0x0000694bc83b3b31 in spice_server_vm_start (s=<optimized out>) at reds.c:3795
        st_item = 0xfd83fd908e0
        item = 0xfd83fd908e0
        __FUNCTION__ = "spice_server_vm_start"
---Type <return> to continue, or q <return> to quit---
#3  0x00000fd83cbce2e6 in qemu_spice_display_start () at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/ui/spice-core.c:925
No locals.
#4  0x00000fd83cb1df5d in qxl_hard_reset (d=0xfd83fd4c1b0, loadvm=loadvm@entry=0) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/display/qxl.c:1176
        startstop = true
#5  0x00000fd83cb1e40b in qxl_reset_handler (dev=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/display/qxl.c:1184
        d = <optimized out>
#6  0x00000fd83caf1a31 in device_reset (dev=dev@entry=0xfd83fd4c1b0) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:1049
        klass = <optimized out>
        __func__ = "device_reset"
#7  0x00000fd83caf1a79 in qdev_reset_one (dev=dev@entry=0xfd83fd4c1b0, opaque=opaque@entry=0x0) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:241
No locals.
#8  0x00000fd83caf1100 in qdev_walk_children (dev=0xfd83fd4c1b0, pre_devfn=pre_devfn@entry=0x0, pre_busfn=pre_busfn@entry=0x0,
    post_devfn=post_devfn@entry=0xfd83caf1a4c <qdev_reset_one>, post_busfn=post_busfn@entry=0xfd83caef370 <qbus_reset_one>, opaque=opaque@entry=0x0)
    at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:464
        bus = 0x0
        err = <optimized out>
#9  0x00000fd83caf11fe in qbus_walk_children (bus=bus@entry=0xfd83f9e2020, pre_devfn=pre_devfn@entry=0x0, pre_busfn=pre_busfn@entry=0x0,
    post_devfn=post_devfn@entry=0xfd83caf1a4c <qdev_reset_one>, post_busfn=post_busfn@entry=0xfd83caef370 <qbus_reset_one>, opaque=opaque@entry=0x0)
    at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:422
        kid = 0xfd83fd3fd40
        err = <optimized out>
#10 0x00000fd83caf10df in qdev_walk_children (dev=0xfd83f9df840, pre_devfn=pre_devfn@entry=0x0, pre_busfn=pre_busfn@entry=0x0,
    post_devfn=post_devfn@entry=0xfd83caf1a4c <qdev_reset_one>, post_busfn=post_busfn@entry=0xfd83caef370 <qbus_reset_one>, opaque=opaque@entry=0x0)
    at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:456
        bus = 0xfd83f9e2020
        err = <optimized out>
#11 0x00000fd83caf11fe in qbus_walk_children (bus=0xfd83fb6a920, pre_devfn=pre_devfn@entry=0x0, pre_busfn=pre_busfn@entry=0x0,
    post_devfn=post_devfn@entry=0xfd83caf1a4c <qdev_reset_one>, post_busfn=post_busfn@entry=0xfd83caef370 <qbus_reset_one>, opaque=opaque@entry=0x0)
    at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:422
        kid = 0xfd83fce4950
        err = <optimized out>
---Type <return> to continue, or q <return> to quit---
#12 0x00000fd83caf128d in qbus_reset_all (bus=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:262
No locals.
#13 0x00000fd83caf12d0 in qbus_reset_all_fn (opaque=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/hw/core/qdev.c:268
        bus = <optimized out>
#14 0x00000fd83ca5d670 in qemu_devices_reset () at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/vl.c:1830
        re = <optimized out>
        nre = 0xfd83fd7d8c0
#15 0x00000fd83ca5d715 in qemu_system_reset (report=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/vl.c:1843
        mc = <optimized out>
        __func__ = "qemu_system_reset"
#16 0x00000fd83ca62f41 in main_loop_should_exit () at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/vl.c:1974
        r = <optimized out>
#17 main_loop () at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/vl.c:2014
        nonblocking = <optimized out>
        last_io = 1
#18 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at /usr/src/debug/app-emulation/qemu-2.1.2-r2/qemu-2.1.2/vl.c:4541
        i = <optimized out>
        snapshot = 0
        linux_boot = 0
        icount_option = 0x0
        initrd_filename = 0x0
        kernel_filename = 0x0
        kernel_cmdline = 0xfd83ccdb7ac ""
        boot_order = <optimized out>
        ds = <optimized out>
        cyls = 0
        heads = 0
        secs = 0
        translation = <optimized out>
        hda_opts = <optimized out>
        opts = <optimized out>
        machine_opts = <optimized out>
        olist = <optimized out>
        optind = 40
        optarg = 0x71876b5b4636
        loadvm = 0x0
        machine_class = 0xfd83f9cdca0
        cpu_model = 0x71876b5b43ec "core2duo"
        vga_model = <optimized out>
        qtest_chrdev = 0x0
        qtest_log = 0x0
        pid_file = 0x0
        incoming = 0x0
        defconfig = <optimized out>
        userconfig = <optimized out>
        log_mask = <optimized out>
        log_file = 0x0
        mem_trace = {malloc = 0xfd83ca5b811 <malloc_and_trace>, realloc = 0xfd83ca5b7c9 <realloc_and_trace>, free = 0xfd83ca5b786 <free_and_trace>, calloc = 0x0,
          try_malloc = 0x0, try_realloc = 0x0}
        trace_events = 0x0
        trace_file = 0x0
        maxram_size = 1073741824
        ram_slots = 0
        vmstate_dump_file = 0x0
        __func__ = "main"

[Comment 1 Matthias Maier 2015-01-25 10:08:39 UTC]

Which version of app-emulation/spice do you use?

There is an upstream bug report and patch [1] for spice 0.12.4.

Can you test the proposed patch please? (If this patch is not already in the latest version, i.e. 0.12.5-r1).

[1] https://freedesktop.org/patch/24594/

[Comment 2 Dejan Lukan 2015-01-25 14:03:11 UTC]

(In reply to Matthias Maier from comment #1)
> Which version of app-emulation/spice do you use?
> 
> There is an upstream bug report and patch [1] for spice 0.12.4.
> 
> Can you test the proposed patch please? (If this patch is not already in the
> latest version, i.e. 0.12.5-r1).
> 
> [1] https://freedesktop.org/patch/24594/

Hi,

I'm using the spice-0.12.5-r1 version where the patch wasn't yet applied. I've applied it manually, but the problem is NOT completely resolved now. The VM still crashes, but immediately reboots without crashing the whole qemu process.

The details are presented below.

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000694bc83687dc in spice_char_device_write_to_device (dev=0xfd83fd843d0) at char_device.c:443
443         if (dev->sin == NULL || dev->sin->base.sif == NULL) {

The contents of the dev variable are shown below, where the sin is set to 0x0:

(gdb) print *dev
$3 = {running = 1, active = 0, wait_for_migrate_data = 0, refs = 3, write_queue = {prev = 0xfd83fd843e0, next = 0xfd83fd843e0}, write_bufs_pool = {prev = 0xfd83fd842e0, 
    next = 0xfd83fd842e0}, cur_write_buf = 0x0, cur_write_buf_pos = 0xfd83fd844a0 "\002", write_to_dev_timer = 0xfd83fd88220, num_self_tokens = 1, clients = {
    prev = 0xfd83fd84360, next = 0xfd83fd84360}, num_clients = 1, client_tokens_interval = 5, sin = 0x0, during_read_from_device = 0, cbs = {
    read_one_msg_from_device = 0x694bc83aa673 <vdi_port_read_buf_get+467>, ref_msg_to_client = 0x694bc83a8780 <red_worker_main+4733>, 
    unref_msg_to_client = 0x694bc83aa069 <vdi_port_read_buf_process+174>, send_msg_to_client = 0x694bc83abddd <reds_handle_auth_mechanism+347>, 
    send_tokens_to_client = 0x694bc83a946d <reds_handle_ssl_accept+175>, on_free_self_token = 0x694bc83ad403 <reds_handle_agent_mouse_event+230>, 
    remove_client = 0x694bc83a8ccf <reds_channel_do_link+350>}, opaque = 0x0}

[Comment 3 Pacho Ramos 2017-04-21 17:16:11 UTC]

can you try with 0.13.3?