Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 929313 - app-emulation/virtualbox: needs upgrade to Python 3.12 (PythonCompatUpdate)
Summary: app-emulation/virtualbox: needs upgrade to Python 3.12 (PythonCompatUpdate)
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: Normal normal
Assignee: Viorel Munteanu
URL: https://www.virtualbox.org/ticket/22017
Whiteboard:
Keywords: UPSTREAM
Depends on:
Blocks: 921826 929314
  Show dependency tree
 
Reported: 2024-04-13 08:29 UTC by Michał Górny
Modified: 2024-10-31 01:28 UTC (History)
30 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patch duplicates the changes made to 7.0.18-r2 (python.patch,2.84 KB, patch)
2024-05-29 20:39 UTC, Jeff Wood
Details | Diff
Patch that duplicates Viorel Munteanu's work, but for 6.1.50-r2 (python.patch,2.99 KB, patch)
2024-05-29 21:02 UTC, Jeff Wood
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-04-13 08:29:03 UTC
This package does not support Python 3.12.  Please test it with Python 3.12 as a matter of urgency as the default will change in early June, as per message on gentoo-dev: https://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg99104.html.  Please check the information provided in the tracker.  Ask in #gentoo-python on IRC if need any help.
Comment 1 Larry the Git Cow gentoo-dev 2024-05-25 09:53:11 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4d068d135757ec370cd2fd8c8c707891cc135241

commit 4d068d135757ec370cd2fd8c8c707891cc135241
Author:     Viorel Munteanu <ceamac@gentoo.org>
AuthorDate: 2024-05-25 07:10:47 +0000
Commit:     Viorel Munteanu <ceamac@gentoo.org>
CommitDate: 2024-05-25 09:52:42 +0000

    app-emulation/virtualbox: make python dependency optional
    
    Do not depend on python if built with USE=-python.
    This should make the upgrade less annoying when python 3.12 becomes the
    default.
    
    Bug: https://bugs.gentoo.org/929313
    Signed-off-by: Viorel Munteanu <ceamac@gentoo.org>

 .../virtualbox/virtualbox-7.0.18-r1.ebuild         | 740 +++++++++++++++++++++
 1 file changed, 740 insertions(+)
Comment 2 Jeff Wood 2024-05-29 19:13:19 UTC
Can you do this with 6.1.50-r2? I tried, but I know zilch about how portage actually works internally. I *think* I copied all your changes to make python go away with the -python flag, but when I added: 

        else
                cat >> AutoConfig.kmk <<-EOF || die
                        VBOX_WITH_PYTHON:=
                EOF
        fi

down around line 376 in the 6.1.50-r2 ebuild, it fails.

Sorry if this is formatted incorrectly.

Ultimately, I kludged around by just changing the 11 to 12 in the "PYTHON_COMPAT=( python3_{10..12} )" line, I know it says it'll silently fail, but I'm not even sure where/how Python gets used, I'm not doing any scripting with VBox.
Comment 3 Jeff Wood 2024-05-29 20:39:37 UTC
Created attachment 894627 [details, diff]
patch duplicates the changes made to 7.0.18-r2
Comment 4 Jeff Wood 2024-05-29 21:02:31 UTC
Created attachment 894628 [details, diff]
Patch that duplicates Viorel Munteanu's work, but for 6.1.50-r2

Messed up first time, this one has all of the changes to make the package compile with Python 3.12. Works locally.
Comment 5 Viorel Munteanu gentoo-dev 2024-05-30 05:06:14 UTC
The 6.1 branch is no longer supported upstream.  I'll keep 6.1.50 in the tree  as long as it is practical, but it will go away eventually (sooner or later it will no longer compile on newer kernels, or someone will find security issues), so it's better to try to migrate to the 7.0 branch.

This being said, I'll try to find some time in the weekend to decouple it from python.
Comment 6 Jeff Wood 2024-05-30 09:37:01 UTC
Thank You. 7 has a weird bug, on multiple monitors, in full screen, the drop-down bar will be in a different screen unless I flip from full screen to windowed and back. It's why I'm clinging to 6.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-06-03 03:44:57 UTC
ceamac, could you stable a version w/ the Python detangling done?
Comment 8 Pacho Ramos gentoo-dev 2024-06-20 09:22:31 UTC
In arch they are applying this python 3.12 patch:
https://gitlab.archlinux.org/archlinux/packaging/packages/virtualbox/-/blob/main/020-python-3-12.patch?ref_type=heads
Comment 9 Boris 2024-10-09 19:45:37 UTC
Version 7.1.2 has already been released. Is there still no support for python 3.12?
Comment 10 Viorel Munteanu gentoo-dev 2024-10-10 04:32:09 UTC
Last I checked it still crashed (in 7.1.0), and they did not reply to my ticket.

I'll test it again in the weekend.
Comment 11 Viorel Munteanu gentoo-dev 2024-10-13 13:27:12 UTC
It still crashes.

I did one more test: I copied VBoxPython3.so from the package for Fedora 40, which claims to support python 3.12, and it also crashes in the same spot.  If I run the same binary with python 3.11, it works.

I don't have access to a Fedora 40 machine to check if it works there.

In conclusion, if you do use the Python bindings, you must stay at python 3.11.  If you do not use them, simply build virtualbox with USE=-python.
Comment 12 Boris 2024-10-15 17:10:15 UTC
I tried to build it myself with python 3.12 and it worked for me.

Here is my configuration:
$ sudo emerge --info
Portage 3.0.66.1 (python 3.12.7-final-0, default/linux/amd64/23.0/no-multilib/systemd, gcc-14, glibc-2.40-r5, 6.11.3-gentoo x86_64)
=================================================================
System uname: Linux-6.11.3-gentoo-x86_64-13th_Gen_Intel-R-_Core-TM-_i9-13900F-with-glibc2.40
KiB Mem:    65389272 total,  40344168 free
KiB Swap:   16777212 total,  16728828 free
Head commit of repository gentoo: 144f96b834ea9b6360814604e0473ed0ae4e5723

sh bash 5.2_p37
ld GNU ld (Gentoo 2.43 p2) 2.43.1
app-misc/pax-utils:        1.3.8::gentoo
app-shells/bash:           5.2_p37::gentoo
dev-build/autoconf:        2.13-r8::gentoo, 2.72-r1::gentoo
dev-build/automake:        1.17-r1::gentoo
dev-build/cmake:           3.30.5::gentoo
dev-build/libtool:         2.5.3::gentoo
dev-build/make:            4.4.1-r100::gentoo
dev-build/meson:           1.5.2::gentoo
dev-java/java-config:      2.3.4::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           3.12.7_p1::gentoo
dev-lang/rust:             1.81.0::gentoo
sys-apps/baselayout:       2.15::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-apps/systemd:          256.7::gentoo
sys-devel/binutils:        2.43-r1::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/clang:           18.1.8::gentoo, 19.1.1::gentoo
sys-devel/gcc:             14.2.1_p20240921::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             18.1.8::gentoo
sys-devel/llvm:            18.1.8-r4::gentoo, 19.1.1::gentoo
sys-kernel/linux-headers:  6.11::gentoo (virtual/os-headers)
sys-libs/glibc:            2.40-r5::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: git
    sync-uri: https://github.com/gentoo/gentoo.git
    priority: -1000
    volatile: False

locale
    location: /usr/local/portage
    masters: gentoo
    priority: 0
    volatile: True

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE linux-fw-redistributable NPSL vim.org intel-ucode NVIDIA-r2 PUEL-12 googleearth NVIDIA-CUDA RAR unRAR all-rights-reserved bh-luxi MPEG-4 YDSLA myspell-ru_RU-AlexanderLebedev Yandex-EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=raptorlake -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/alsa/alsa.conf.d /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.9/conf"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-march=raptorlake -O2 -pipe"
DISTDIR="/usr/src/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n --quiet-build=y"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait metadata-transfer multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="ru_RU.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
LINGUAS="en en_US"
MAKEOPTS="-j32"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/mnt/tmp"
SHELL="/bin/bash"
USE="7zip \ a52 aac aacs acl acpi activities alsa amd64 amr ao apng appstream assistant audiofile autotools avi bash-completion battery berkdb bl bluray boost branding brotli btrfs bzip2 cairo caps cdr cet chm chromium cjk clang cli cmake codec2 color-console compositor context corefonts crypt cryptsetup css ctype cups curl cxx cyrillic dbus debugger dec265 declarative designer detex device-mapper dga dictionary distinct-l divx divx4linux djvu dmx dnscrypt dnstap dri dts dvd dvdr dvdread ecs editorconfig egl elf emoji emovix enc265 enca encode epub eselect evdev exif expat extra extras f2fs faac faad fakevim farstream fat fbcon fdk ffmpeg filter flac fontconfig fortran fpx freetype frei0r ftp gbm gcj gdbm geoip geoip2 ggi gif git gles2 glut gmp gmplayer gnutls gost gpg gpgme gphoto2 gpm graphics graphviz gstreamer harfbuzz hash hddtemp hdri highlight hostonly htmlthumbs http http2 humanities icons iconv icu id3tag imagemagick imap imlib initramfs installkernel introspection ipv6 jadetex jbig jit jpeg jpeg2k json kdesu kf6compat kipi kpathsea lame lcms libass libcaca libdrm libkms libproxy libssh libssh2 libtirpc live llvm lm-sensors lm_sensors lto lz4 lzma lzo mad markdown math matroska md5sum mediawiki mesa midi mikmod minizip mjpeg mng mod mozdom mp3 mp4 mpeg mpi mpi-threads mplayer mtp multimedia multitarget mumps musepack musicbrainz natspec ncurses networkmanager nfs nls nptl nss ntfs ntp nvenc nvidia ogg oggvorbis openal opencl openexr opengl openh264 openmp openrc openssl opus osmesa otf ozone pam pci pcre pcre32 pdf php php5 pic pipewire pipewire-alsa pkcs11 png pnm policykit posix postproc ppds printsupport profiler pstricks publishers pulseaudio python qdoc qemu qml qt6 quicktime radio rar raw readline real realmedia renderdoc rfc3779 rfc4175 romio rpc rss rtc rtmp rustfmt sasl sbc science screencast script scripttools sctp sdl seccomp semantic-desktop sensord simplexml skype slang slp smp smtp sndfile snmp sockets sound speex spell spoof-source sql sqlite ssh ssl svg symlink system-llvm systemd taglib test-rust theora thounderbird threads tiff timidity toolbar tools truetype ttf tty-helpers udev udev-acl udisks uki unicode unrar upnp usb usbredir user-session utf8 utils utp uvm v4l vaapi vdpau vim vim-pager vim-syntax vim-with-x virtualbox vorbis vpx vulkan wav wavpack wayland webchannel webkit webp widgets wma wmf x264 x265 xa xattr xcb xml xmlpatterns xmp xmss xpm xvfb xvid xwayland yt-dlp yuv4mpeg zeroconf zimg zip zlib zstd" ABI_X86="64" ADA_TARGET="gcc_12" ALSA_CARDS="hda-intel usb-audio mpu401" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3 vpclmulqdq" CURL_SSL="openssl" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64 pc" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="evdev" KERNEL="linux" L10N="ru" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_SLOT="18" LLVM_TARGETS="AArch64 NVPTX" LUA_SINGLE_TARGET="lua5-4 luajit" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" QEMU_SOFTMMU_TARGETS="arm x86_64" QEMU_USER_TARGETS="arm armeb x86_64" RUBY_TARGETS="ruby32" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 13 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-15 19:48:39 UTC
I don't think that means too much unless you share the ebuild change you made and test the Python bindings. Did you?
Comment 14 Boris 2024-10-15 22:59:15 UTC
(In reply to Sam James from comment #13)
> I don't think that means too much unless you share the ebuild change you
> made and test the Python bindings. Did you?

I am ready to share any information.

In virtualbox-7.1.2.ebuild, I fixed only this:
```
PYTHON_COMPAT=( python3_{10..12} )
```                           ^

VirtualBox was built with the following flags:
```
[ebuild   R    ] app-emulation/virtualbox-7.1.2:0/7.1::gentoo  USE="alsa dbus gui nls opengl pam pulseaudio python sdk udev vmmraw -debug -doc -dtrace -java -lvm (-pch) -sdl -vboxwebsrv -vde -vnc" PYTHON_SINGLE_TARGET="python3_12 -python3_10 -python3_11"
```
Comment 15 Viorel Munteanu gentoo-dev 2024-10-16 06:55:51 UTC
Please run `vboxshell.py`

Or try directly in python 3.12:
$ python3.12
Python 3.12.6 (main, Oct  9 2024, 13:03:44) [GCC 13.3.1 20240614] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from vboxapi import VirtualBoxManager
>>> manager = VirtualBoxManager()
Segmentation fault

Both work in Python 3.11 for me, but crash in Python 3.12+.
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-16 07:06:06 UTC
(In reply to Viorel Munteanu from comment #15)
> Please run `vboxshell.py`
> 
> Or try directly in python 3.12:
> $ python3.12
> Python 3.12.6 (main, Oct  9 2024, 13:03:44) [GCC 13.3.1 20240614] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from vboxapi import VirtualBoxManager
> >>> manager = VirtualBoxManager()
> Segmentation fault
> 
> Both work in Python 3.11 for me, but crash in Python 3.12+.

We should consider adding a trivial src_test (extension) with USE=python containing this.
Comment 17 Viorel Munteanu gentoo-dev 2024-10-16 07:08:17 UTC
Upstream just released 7.0.22 and 7.1.4, I'll add it to that release.
Comment 18 Larry the Git Cow gentoo-dev 2024-10-18 17:16:50 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=754efcbdcd799f8a8f71805b543c638e2759ad6b

commit 754efcbdcd799f8a8f71805b543c638e2759ad6b
Author:     Viorel Munteanu <ceamac@gentoo.org>
AuthorDate: 2024-10-18 15:59:31 +0000
Commit:     Viorel Munteanu <ceamac@gentoo.org>
CommitDate: 2024-10-18 17:13:10 +0000

    app-emulation/virtualbox: add 7.0.22
    
    Bug: https://bugs.gentoo.org/929313
    Signed-off-by: Viorel Munteanu <ceamac@gentoo.org>

 app-emulation/virtualbox/Manifest                 |   2 +
 app-emulation/virtualbox/files/test_python.py     |  16 +
 app-emulation/virtualbox/virtualbox-7.0.22.ebuild | 767 ++++++++++++++++++++++
 3 files changed, 785 insertions(+)
Comment 19 Viorel Munteanu gentoo-dev 2024-10-18 17:24:41 UTC
I added the test and the results are:
- python 3.11 works
- python 3.12 builds but crashes
- python 3.13 does not build.

If I patch the Makefile to add python 3.13 I get compile errors, some functions have been removed (PyEval_CallObject, PyObject_AsReadBuffer, maybe others).