Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 778356

Summary: sys-apps/portage-3.0.17-r2 fails tests: SimpleEmergeTestCase: Invalid or incomplete multibyte or wide character
Product: Gentoo Linux Reporter: Agostino Sarubbo <ago>
Component: Current packagesAssignee: Portage team <dev-portage>
Status: CONFIRMED ---    
Severity: normal CC: sam
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: build.log
portage-3.0.17-r2 successful unit tests on amd64
build.log (3.0.17-r2 on arm failing)
build.log (ppc, 3.0.49-r2)

Description Agostino Sarubbo gentoo-dev 2021-03-26 10:23:23 UTC
https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/

Issue: sys-apps/portage-3.0.17-r2 fails tests.
Discovered on: ppc (internal ref: ppc_at)
Comment 1 Agostino Sarubbo gentoo-dev 2021-03-26 10:23:42 UTC
Created attachment 694083 [details]
build.log

build log and emerge --info
Comment 2 Agostino Sarubbo gentoo-dev 2021-03-26 10:53:56 UTC
Reproducible also on x86
Comment 3 Zac Medico gentoo-dev 2021-03-26 19:55:14 UTC
Created attachment 694530 [details]
portage-3.0.17-r2 successful unit tests on amd64

(In reply to Agostino Sarubbo from comment #1)
> Created attachment 694083 [details]
> build.log
> 
> build log and emerge --info

Is this result reproducible? I have executed the tests successfully on amd64, as show in the attached log.
Comment 4 Agostino Sarubbo gentoo-dev 2021-03-26 20:04:25 UTC
(In reply to Zac Medico from comment #3)
> Created attachment 694530 [details]
> portage-3.0.17-r2 successful unit tests on amd64
> 
> (In reply to Agostino Sarubbo from comment #1)
> > Created attachment 694083 [details]
> > build.log
> > 
> > build log and emerge --info
> 
> Is this result reproducible? I have executed the tests successfully on
> amd64, as show in the attached log.

I can reproduce on x86 and ppc atm.
Comment 5 Zac Medico gentoo-dev 2021-03-26 20:41:31 UTC
(In reply to Agostino Sarubbo from comment #4)
> (In reply to Zac Medico from comment #3)
> > Created attachment 694530 [details]
> > portage-3.0.17-r2 successful unit tests on amd64
> > 
> > (In reply to Agostino Sarubbo from comment #1)
> > > Created attachment 694083 [details]
> > > build.log
> > > 
> > > build log and emerge --info
> > 
> > Is this result reproducible? I have executed the tests successfully on
> > amd64, as show in the attached log.
> 
> I can reproduce on x86 and ppc atm.

Could you reproduce both test failures? These were these two independent failures:

portage.tests.emerge.test_simple.SimpleEmergeTestCase:

> testSimple (portage.tests.emerge.test_simple.SimpleEmergeTestCase) ...  * This action can remove important packages! In order to be safer, use
>  * `emerge -pv --depclean <atom>` to check for reverse dependencies before
>  * removing packages.
> >>> Creating Manifest for /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/repositories/test_repo/dev-libs/A
>  * checking ebuild checksums ;-) ...                                     [ ok ]
>  * checking miscfile checksums ;-) ...                                   [ ok ]
>  * called pkg_pretend for dev-libs/A-1
> >>> Unpacking source...
> >>> Source unpacked in /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work
> >>> Preparing source in /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work ...
> >>> Source prepared.
> >>> Configuring source in /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work ...
> >>> Source configured.
> >>> Compiling source in /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work ...
> >>> Source compiled.
> >>> Test phase [not enabled]: dev-libs/A-1
> 
> >>> Install dev-libs/A-1 into /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/image/
>  * installing something...
> /var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/usr/bin/install: cannot change permissions of ‘/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/image/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/usr/lib/A-1/latin-1-\251-directory’: Invalid or incomplete multibyte or wide character
>  * ERROR: dev-libs/A-1::test_repo failed (install phase):
>  *   insinto failed
>  * 
>  * Call stack:
>  *               ebuild.sh, line 125:  Called src_install
>  *             environment, line 148:  Called insinto '/usr/lib/A-1/latin-1-�-directory'
>  *        phase-helpers.sh, line  62:  Called __helpers_die 'insinto failed'
>  *   isolated-functions.sh, line 112:  Called die
>  * The specific snippet of code:
>  * 
>  * If you need support, post the output of `emerge --info '=dev-libs/A-1::test_repo'`,
>  * the complete build log and the output of `emerge -pqv '=dev-libs/A-1::test_repo'`.
>  * The complete build log is located at '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/dev-libs:A-1:20210326-092450.log'.
>  * For convenience, a symlink to the build log is located at '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/temp/build.log'.
>  * The ebuild environment file is located at '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/temp/environment'.
>  * Working directory: '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work'
>  * S: '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/tmp/portage/portage/dev-libs/A-1/work'
> FAIL
> 
> ======================================================================
> FAIL: testSimple (portage.tests.emerge.test_simple.SimpleEmergeTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/tests/__init__.py", line 209, in run
>     testMethod()
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/tests/emerge/test_simple.py", line 224, in testSimple
>     loop.run_until_complete(asyncio.ensure_future(
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/util/_eventloop/asyncio_event_loop.py", line 121, in _run_until_complete
>     return self._loop.run_until_complete(future)
>   File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
>     return future.result()
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/tests/emerge/test_simple.py", line 560, in _async_test_simple
>     self.assertEqual(os.EX_OK, proc.returncode,
> AssertionError: 0 != 1 : emerge failed with args ('/usr/bin/python3.8', '-b', '-Wd', '/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/bin/ebuild', '/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmphs1qr4g6/var/repositories/test_repo/dev-libs/A/A-1.ebuild', 'manifest', 'clean', 'package', 'merge')

portage.tests.util.futures.test_retry.RetryForkExecutorTestCase>

> testOverallTimeoutWithException (portage.tests.util.futures.test_retry.RetryForkExecutorTestCase) ... FAIL
> 
> ======================================================================
> FAIL: testOverallTimeoutWithException (portage.tests.util.futures.test_retry.RetryForkExecutorTestCase)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/tests/__init__.py", line 209, in run
>     testMethod()
>   File "/var/tmp/portage/sys-apps/portage-3.0.17-r2/work/portage-3.0.17-python3_8/lib/portage/tests/util/futures/test_retry.py", line 142, in testOverallTimeoutWithException
>     self.assertTrue(isinstance(done.pop().exception().__cause__, SucceedNeverException))
> AssertionError: False is not true
Comment 6 Zac Medico gentoo-dev 2021-03-26 20:42:22 UTC
I've unblocked bug 778185, since these test failures do not appear to be regressions.
Comment 7 Zac Medico gentoo-dev 2021-03-26 22:50:40 UTC
It's interesting that same tests pass consistently for all supported versions of python in github actions:

https://github.com/gentoo/portage/actions
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-03-28 14:25:29 UTC
Created attachment 695526 [details]
build.log (3.0.17-r2 on arm failing)

I occasionally hit testSimple (portage.tests.emerge.test_simple.SimpleEmergeTestCase) but not been able to pin down what it is. It's not arch specific or anything for me.

----
Portage 3.0.13 (python 3.8.8-final-0, default/linux/arm/17.0/armv7a, gcc-10.2.0, glibc-2.33, 5.4.105-gentoo-dist armv8l)
=================================================================
System uname: Linux-5.4.105-gentoo-dist-armv8l-ARMv8_Processor_rev_1_-v8l-with-glibc2.4
KiB Mem:   131636512 total,  91468072 free
KiB Swap:   16777212 total,  16777212 free
Timestamp of repository gentoo: Sun, 28 Mar 2021 13:50:41 +0000
sh bash 5.0_p18
ld GNU ld (Gentoo 2.35.1 p2) 2.35.1
ccache version 4.2 [disabled]
app-shells/bash:          5.0_p18::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          3.8.8::gentoo, 3.9.2::gentoo
dev-util/ccache:          4.2::gentoo
dev-util/cmake:           3.18.5::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1-r1::gentoo
sys-apps/sandbox:         2.20::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.16.2-r1::gentoo
sys-devel/binutils:       2.35.1-r1::gentoo
sys-devel/gcc:            10.2.0-r5::gentoo
sys-devel/gcc-config:     2.3.2-r1::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.33::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts:

ACCEPT_KEYWORDS="arm"
ACCEPT_LICENSE="*"
CBUILD="armv7a-unknown-linux-gnueabihf"
CFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard -frecord-gcc-switches"
CHOST="armv7a-unknown-linux-gnueabihf"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard -frecord-gcc-switches"
DISTDIR="/var/cache/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --complete-graph --usepkg=y --with-bdeps=y --deep"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard -frecord-gcc-switches"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard -frecord-gcc-switches"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0"
MAKEOPTS="-j6"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl arm berkdb bzip2 cli crypt dbus dri elogind fortran gdbm iconv ipv6 libglvnd ncurses nls nptl openmp pam pcre readline seccomp split-usr ssl tcpd unicode xattr zlib" ADA_TARGET="gnat_2018" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp thumb thumb2 v4 v5 v6 v7 vfp" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python3_8 python3_9" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="exynos fbdev omap dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 9 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-03-28 14:35:12 UTC
Actually, I see that "1/image/var/tmp/portage/sys-apps/portage-3.0.17-r2/temp/tmpv18htmc6/usr/lib/A-1/latin-1-\251-directory’: Invalid or incomplete multibyte or wide character" junk quite a lot with Python packages.

I'm not sure what the issue is though:

# eselect locale list
Available targets for the LANG variable:
  [1]   C
  [2]   C.utf8
  [3]   en_GB.utf8
  [4]   en_US.utf8 *
  [5]   POSIX
  [ ]   (free form)

# locale-gen
 * Generating 3 locales (this might take a while) with 8 jobs
 *  (3/3) Generating C.UTF-8 ...                                                                                                                                 [ ok ]
 *  (2/3) Generating en_US.UTF-8 ...                                                                                                                             [ ok ]
 *  (1/3) Generating en_GB.UTF-8 ...                                                                                                                             [ ok ]
 * Generation complete
 * Adding locales to archive ...                                                                                                                                 [ ok ]
Comment 10 Zac Medico gentoo-dev 2021-03-28 14:41:18 UTC
I wonder if you're forcing a particular encoding at the filesystem driver level, and that causes the install command to fail.
Comment 11 ernsteiswuerfel archtester 2023-09-15 22:02:49 UTC
Created attachment 870661 [details]
build.log (ppc, 3.0.49-r2)

Still seems a problem. Got that on ppc.