qemu-guest-agent is missing the arm64 keyword. I have tested it on an arm64/aarch64 qemu soft emulation install of Gentoo, and it works as it should. # uname -a Linux gentoo-aarch64 6.1.57-gentoo-dist #1 SMP PREEMPT_DYNAMIC Wed Oct 11 01:44:07 -00 2023 aarch64 GNU/Linux # qemu-ga --version QEMU Guest Agent 8.0.3 # emerge --info Portage 3.0.51 (python 3.11.6-final-0, default/linux/arm64/17.0, gcc-13, glibc-2.37-r7, 6.1.57-gentoo-dist aarch64) ================================================================= System uname: Linux-6.1.57-gentoo-dist-aarch64-with-glibc2.37 KiB Mem: 4009992 total, 3422468 free KiB Swap: 4194300 total, 4194300 free Timestamp of repository gentoo: Sat, 04 Nov 2023 21:15:01 +0000 Head commit of repository gentoo: 2ed1b38745b4e09895853a78d51ec022c34769dc sh bash 5.1_p16-r6 ld GNU ld (Gentoo 2.40 p5) 2.40.0 app-misc/pax-utils: 1.3.7::gentoo app-shells/bash: 5.1_p16-r6::gentoo dev-lang/perl: 5.38.0-r1::gentoo dev-lang/python: 3.11.6::gentoo, 3.12.0::gentoo dev-util/meson: 1.2.1-r1::gentoo sys-apps/baselayout: 2.14::gentoo sys-apps/openrc: 0.48::gentoo sys-apps/sandbox: 2.38::gentoo sys-devel/autoconf: 2.71-r6::gentoo sys-devel/automake: 1.16.5-r1::gentoo sys-devel/binutils: 2.40-r5::gentoo sys-devel/binutils-config: 5.5::gentoo sys-devel/gcc: 13.2.1_p20230826::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/libtool: 2.4.7-r1::gentoo sys-devel/make: 4.4.1-r1::gentoo sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.37-r7::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: rsync sync-uri: rsync://mirrors.tnonline.net/gentoo-portage priority: -1000 volatile: False sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 sync-rsync-verify-metamanifest: yes sync-rsync-extra-opts: Binary Repositories: gentoobinhost priority: 1 sync-uri: https://mirrors.tnonline.net/gentoo/gentoo-distfiles/releases/arm64/binpackages/17.0/arm64 ACCEPT_KEYWORDS="arm64" ACCEPT_LICENSE="* @FREE @BINARY-REDISTRIBUTABLE" CBUILD="aarch64-unknown-linux-gnu" CFLAGS="-O2 -pipe" CHOST="aarch64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS=" --getbinpkg" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="rsync://mirrors.tnonline.net/gentoo-distfiles/" LANG="en_GB.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LEX="flex" MAKEOPTS="-j4 -l6" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="acl arm64 bzip2 cli crypt dri fortran gdbm iconv ipv6 libtirpc ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl test-rust unicode xattr zlib" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp v8 vfp vfp-d32 vfpv3 vfpv4" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Created attachment 876022 [details] build.log Seems to need some sort of qemu executable available?
Matoro, if you read your build log carefully, you'll see the guest agent compiled successfully, but failed during the unit test phase. It makes sense that would fail w/o the emulator available. The unit tests can't/won't run without the QEMU system emulator aka (qemu-system-aarch64/qemu-system-arm) being available. The guest agent itself appears to build/install just fine without the emulator using the command: env ACCEPT_KEYWORDS="*" emerge --ask=n --autounmask-write=y --autounmask-continue=y app-emulation/qemu-guest-agent Of course that command doesn't run the unit tests. Please add me to the list of people who would like to see this package enabled/available by default on ARM32/ARM64. I currently maintain Gentoo Vagrant base box images for X86/AMD64, and recently acquired an ARM build server. Which means I'm also going to start building ARM64 images (at least for the QEMU/libvirt providers), and would like to pre-install the guest agent in those box images.
(In reply to Ladar Levison from comment #2) > Matoro, if you read your build log carefully, you'll see the guest agent > compiled successfully, but failed during the unit test phase. It makes sense > that would fail w/o the emulator available. The unit tests can't/won't run > without the QEMU system emulator aka (qemu-system-aarch64/qemu-system-arm) > being available. The guest agent itself appears to build/install just fine > without the emulator using the command: > > env ACCEPT_KEYWORDS="*" emerge --ask=n --autounmask-write=y > --autounmask-continue=y app-emulation/qemu-guest-agent > > Of course that command doesn't run the unit tests. > > Please add me to the list of people who would like to see this package > enabled/available by default on ARM32/ARM64. > > I currently maintain Gentoo Vagrant base box images for X86/AMD64, and > recently acquired an ARM build server. Which means I'm also going to start > building ARM64 images (at least for the QEMU/libvirt providers), and would > like to pre-install the guest agent in those box images. Hi Ladar, yes thank you for your work on robox, I've even contributed to the Gentoo support there. The issue is that we require a passing test suite in order to mark a package as working, as it's what allows us to identify regressions and really say that a package is "working" rather than just "builds". I've been thinking about how to go about fixing this. We're kind of stuck since the test suite really expects the corresponding qemu binaries to be present in the build tree, and it needs to be the relevant ones for the current arch. It's possible the correct solution is to merge into a USE-flag of app-emulation/qemu. Let me raise this as a discussion to the maintainers.
Hi Matoro, I don't know the Gentoo packaging system well enough, so I didn't mention this above, but is possible to stipulate additional dependencies that are only required for testing? What makes things a little unusual in this instance is the guest agent source tarball already contains the code for the emulator/system binaries. They just aren't being compiled. Is it possible, alternatively, to adjust the configure options to build the emulator when your also running the tests? Or re-run configure and enable the emulator when starting the test step? One possible issue, which I don't know off-hand, is if it's possible to build the emulator just for testing, but not install it. I know it's possible to build the bundled firmware/ROMs for test purposes, but never looked for something like that related to the emulator binaries. How does the X86/AND64 version of the guest agent package handle this issue? I imagine it would run into the same problem. P.S. In the process of releasing the first A64 Gentoo box. Feel free to check it out and provide any relevant feedback.
(In reply to Ladar Levison from comment #4) > Hi Matoro, I don't know the Gentoo packaging system well enough, so I didn't > mention this above, but is possible to stipulate additional dependencies > that are only required for testing? Yes of course, but as you mention below it specifically needs emulator binaries in the source tree, not just on the system. > What makes things a little unusual in this instance is the guest agent > source tarball already contains the code for the emulator/system binaries. > They just aren't being compiled. > > Is it possible, alternatively, to adjust the configure options to build the > emulator when your also running the tests? Or re-run configure and enable > the emulator when starting the test step? > > One possible issue, which I don't know off-hand, is if it's possible to > build the emulator just for testing, but not install it. I know it's > possible to build the bundled firmware/ROMs for test purposes, but never > looked for something like that related to the emulator binaries. The former approach is one possible option, but I do not know off the top of my head how to have it build those emulators and then not install them (short of deleting the binaries post-test, but there might be additional misc files that it builds when those options are flipped on) > How does the X86/AND64 version of the guest agent package handle this issue? > I imagine it would run into the same problem. It doesn't even though it's supposed to. The fact that the maintainers did not check it is a mistake, but it needs to be corrected for all arches before we can mark it as keyworded. > P.S. In the process of releasing the first A64 Gentoo box. Feel free to > check it out and provide any relevant feedback.
(In reply to matoro from comment #5) > (In reply to Ladar Levison from comment #4) > > Hi Matoro, I don't know the Gentoo packaging system well enough, so I didn't > > mention this above, but is possible to stipulate additional dependencies > > that are only required for testing? > > Yes of course, but as you mention below it specifically needs emulator > binaries in the source tree, not just on the system. > > > What makes things a little unusual in this instance is the guest agent > > source tarball already contains the code for the emulator/system binaries. > > They just aren't being compiled. > > > > Is it possible, alternatively, to adjust the configure options to build the > > emulator when your also running the tests? Or re-run configure and enable > > the emulator when starting the test step? > > > > One possible issue, which I don't know off-hand, is if it's possible to > > build the emulator just for testing, but not install it. I know it's > > possible to build the bundled firmware/ROMs for test purposes, but never > > looked for something like that related to the emulator binaries. > > The former approach is one possible option, but I do not know off the top of > my head how to have it build those emulators and then not install them > (short of deleting the binaries post-test, but there might be additional > misc files that it builds when those options are flipped on) Couldva solution be to add a "test" USE-flag. When enabled, it builds full qemu sources with --enable-guest-agent, runs its tests, and if ok, can continue with rebuilding without the tests in a second run? > > > How does the X86/AND64 version of the guest agent package handle this issue? > > I imagine it would run into the same problem. > > It doesn't even though it's supposed to. The fact that the maintainers did > not check it is a mistake, but it needs to be corrected for all arches > before we can mark it as keyworded. > > > P.S. In the process of releasing the first A64 Gentoo box. Feel free to > > check it out and provide any relevant feedback. Another option, can we rely on app-emulation/qemu tests on the arch, and assume that the guest-agent will be ok? We could include something like ?test (--enable-guest-agent).
Created attachment 882690 [details] ebuild qemu-8.2.0.ebuild test log I enabled "--enable-guest-agent" in qemu-8.2.0.ebuild and ran "ebuild qemu-8.2.0.ebuild test" on my x86 box (the aarch64 vm is very slow so it'll take a while before it finishes). # Start of qga tests ok 1 /qga/sync-delimited ok 2 /qga/sync ok 3 /qga/ping ok 4 /qga/info ok 5 /qga/network-get-interfaces ok 6 /qga/get-vcpus ok 7 /qga/get-fsinfo ok 8 /qga/get-memory-block-info ok 9 /qga/get-memory-blocks ok 10 /qga/file-ops ok 11 /qga/file-write-read ok 12 /qga/get-time ok 13 /qga/id ok 14 /qga/invalid-oob ok 15 /qga/invalid-cmd ok 16 /qga/invalid-args ok 17 /qga/fsfreeze-status ok 18 /qga/blockedrpcs # slow test /qga/blockedrpcs executed in 1,08 secs ok 19 /qga/allowedrpcs # slow test /qga/allowedrpcs executed in 1,00 secs ok 20 /qga/config ok 21 /qga/guest-exec ok 22 /qga/guest-exec-separated ok 23 /qga/guest-exec-merged ok 24 /qga/guest-exec-invalid ok 25 /qga/guest-get-osinfo # slow test /qga/guest-get-osinfo executed in 1,00 secs ok 26 /qga/guest-get-host-name ok 27 /qga/guest-get-timezone ok 28 /qga/guest-get-users # End of qga tests
Created attachment 882695 [details] aarch64 testlog of qemu-8.2.0.ebuild Finished the ebuild test on the aarch64 # Start of qga tests ok 1 /qga/sync-delimited ok 2 /qga/sync ok 3 /qga/ping ok 4 /qga/info ok 5 /qga/network-get-interfaces ok 6 /qga/get-vcpus ok 7 /qga/get-fsinfo ok 8 /qga/get-memory-block-info ok 9 /qga/get-memory-blocks # slow test /qga/get-memory-blocks executed in 0.67 secs ok 10 /qga/file-ops ok 11 /qga/file-write-read ok 12 /qga/get-time ok 13 /qga/id ok 14 /qga/invalid-oob ok 15 /qga/invalid-cmd ok 16 /qga/invalid-args ok 17 /qga/fsfreeze-status ok 18 /qga/blockedrpcs # slow test /qga/blockedrpcs executed in 1.22 secs ok 19 /qga/allowedrpcs # slow test /qga/allowedrpcs executed in 1.18 secs ok 20 /qga/config ok 21 /qga/guest-exec ok 22 /qga/guest-exec-separated ok 23 /qga/guest-exec-merged ok 24 /qga/guest-exec-invalid ok 25 /qga/guest-get-osinfo # slow test /qga/guest-get-osinfo executed in 1.20 secs ok 26 /qga/guest-get-host-name ok 27 /qga/guest-get-timezone ok 28 /qga/guest-get-users # End of qga tests gentoo-aarch64 # qemu-ga --version QEMU Guest Agent 8.2.0
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=99decf1e1533778a58f3c9bc7898c58526e7b780 commit 99decf1e1533778a58f3c9bc7898c58526e7b780 Author: Matoro Mahri <matoro_gentoo@matoro.tk> AuthorDate: 2024-01-18 06:39:01 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-02-02 03:54:45 +0000 app-emulation/qemu-guest-agent: add 8.2.0, limit test suite As noted, the only relevant test for the guest agent is test-qga, which is part of the qobject test suite, so limit to just this test suite. Bug: https://bugs.gentoo.org/917080 Signed-off-by: Matoro Mahri <matoro_gentoo@matoro.tk> Closes: https://github.com/gentoo/gentoo/pull/34876 Signed-off-by: Sam James <sam@gentoo.org> app-emulation/qemu-guest-agent/Manifest | 1 + .../qemu-guest-agent/qemu-guest-agent-8.2.0.ebuild | 88 ++++++++++++++++++++++ 2 files changed, 89 insertions(+)
arm64 done all arches done
