Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 879891

Summary: sys-apps/diffutils-3.9 fails 'color' test
Product: Gentoo Linux Reporter: Agostino Sarubbo <ago>
Component: Current packagesAssignee: Gentoo's Team for Core System packages <base-system>
Status: RESOLVED FIXED    
Severity: normal CC: tka
Priority: Normal Keywords: PATCH, TESTFAILURE
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://debbugs.gnu.org/59905
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 879081    
Attachments: build.log
1-test-suite.log
non-LTO diffutils-3.9 build.log
non-LTO diffutils-3.9 test-suite.log

Description Agostino Sarubbo gentoo-dev 2022-11-05 16:55:15 UTC
https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/

Issue: sys-apps/diffutils-3.8 fails tests (LTO-SYSTEM).
Discovered on: amd64 (internal ref: lto_tinderbox)

NOTE:
(LTO-SYSTEM) in the summary means that bug was found on a machine that runs lto but this bug MAY or MAY NOT BE related to lto.

This machine uses lto with CFLAGS=-flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing

Here is a bit of explanation:

-Werror=lto-type-mismatch:
User to find possible runtime issues in packages. It likely means the package is unsafe to build & use with LTO.
For projects using the same identifier but with different types across different files, they must be fixed to be consistent across the codebase.

-Werror=odr:
Used to find possible runtime issues in packages. These bugs are a problem anyway but may be even worse when combined with LTO. C++ code must comply with the One Definition Rule (ODR) - see https://en.cppreference.com/w/cpp/language/definition#One_Definition_Rule.

-Werror=strict-aliasing:
Used to find possible runtime issues in packages. These bugs are a problem anyway but may be even worse when combined with LTO.

Workarounds:
- If upstream is friendly and still active, file a bug upstream. For emulators, codecs, games, or multimedia packages, it may be worth just applying a workaround instead, as upstreams sometimes aren't receptive to these bugs (VALID FOR ALL).
- Use the new 'filter-lto' from flag-o-matic.eclass as it's likely to be unsafe with LTO (VALID FOR lto-type-mismatch - odr).
- Fix it yourself if interested, of course (VALID FOR ALL).
- Append-flags -fno-strict-aliasing (VALID FOR strict-aliasing).
- Use memcpy() but a union is sometimes suitable too (VALID FOR strict-aliasing).
- -fstrict-aliasing is implied by -O2, so this must be addressed in some form (VALID FOR strict-aliasing).

See also: https://marc.info/?l=gentoo-dev&m=165639574126280&w=2
Comment 1 Agostino Sarubbo gentoo-dev 2022-11-05 16:55:18 UTC
Created attachment 827849 [details]
build.log

build log and emerge --info
Comment 2 Agostino Sarubbo gentoo-dev 2022-11-05 16:55:19 UTC
Created attachment 827851 [details]
1-test-suite.log

1-test-suite.log
Comment 3 Agostino Sarubbo gentoo-dev 2022-11-05 16:55:21 UTC
Error(s) that match a know pattern:


FAIL colors (exit status: 1)
FAIL: colors
Comment 4 unhappy-ending 2022-11-28 21:18:56 UTC
Happens on my LLVM toolchain + Clang LTO system as well. I did notice that compile phase fails when passing the flag -Werror=incompatible-library-redeclaration, not sure if that has anything to do with this.
Comment 5 unhappy-ending 2022-11-28 21:20:55 UTC
FWIW, I also built diffutils without -flto and it still fails the test. There is only 4 dependencies so it would be pretty easy to build those without -flto and try to re-run the tests. If it fails, I'm guessing it's not LTO related.
Comment 6 tka 2023-01-18 21:00:49 UTC
Created attachment 848797 [details]
non-LTO diffutils-3.9 build.log

The colors test just failed on my non-LTO system for diffutils-3.9. I tried to rebuild 3.8, but that now also fails. Thus, the error is not related to LTO and also affects 3.9.
Comment 7 tka 2023-01-18 21:03:25 UTC
Created attachment 848799 [details]
non-LTO diffutils-3.9 test-suite.log

# emerge -pqv '=sys-apps/diffutils-3.9::gentoo'
[ebuild     U ] sys-apps/diffutils-3.9 [3.8] USE="verify-sig -nls -static" 


# emerge --info '=sys-apps/diffutils-3.9::gentoo'
Portage 3.0.44 (python 3.10.9-final-0, default/linux/amd64/17.1/no-multilib/hardened, gcc-12, glibc-2.36-r6, 6.1.7 x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-6.1.7-x86_64-Intel-R-_Core-TM-_i7-4810MQ_CPU_@_2.80GHz-with-glibc2.36
KiB Mem:    16284928 total,   9937088 free
KiB Swap:   20971516 total,  20971516 free
Timestamp of repository gentoo: Wed, 18 Jan 2023 16:30:01 +0000
Head commit of repository gentoo: 132f0105f90e0efcb482bb87e34748191ffadd95
sh bash 5.2_p15-r1
ld GNU ld (Gentoo 2.40 p1) 2.40
app-misc/pax-utils:        1.3.6-r1::gentoo
app-shells/bash:           5.2_p15-r1::gentoo
dev-java/java-config:      2.3.1::gentoo
dev-lang/perl:             5.36.0-r1::gentoo
dev-lang/python:           3.10.9::gentoo, 3.11.1::gentoo
dev-lang/rust:             1.66.1::gentoo
dev-util/cmake:            3.25.1::gentoo
dev-util/meson:            1.0.0::gentoo
sys-apps/baselayout:       2.9::gentoo
sys-apps/openrc:           0.46::gentoo
sys-apps/sandbox:          2.30-r1::gentoo
sys-devel/autoconf:        2.13-r7::gentoo, 2.71-r5::gentoo
sys-devel/automake:        1.16.5::gentoo
sys-devel/binutils:        2.40::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang:           15.0.7-r1::gentoo
sys-devel/gcc:             12.2.1_p20221231::gentoo
sys-devel/gcc-config:      2.10::gentoo
sys-devel/libtool:         2.4.7-r1::gentoo
sys-devel/lld:             15.0.7::gentoo
sys-devel/llvm:            15.0.7::gentoo
sys-devel/make:            4.4::gentoo
sys-kernel/linux-headers:  6.1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.36-r6::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    volatile: True
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-verify-max-age: 24
    sync-rsync-extra-opts: --new-compress

local
    location: /var/lib/portage-local/local
    masters: gentoo
    priority: 0
    volatile: True

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -mtune=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -mtune=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live cgroup collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu"
LEX="flex"
LINGUAS=""
MAKEOPTS="-j8"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--new-compress"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 acl acpi alsa amd64 avx bzip2 cairo caps cdda cddb cdparanoia cli crypt cups dbus dri dts dvd elogind fam ffmpeg fftw flac fontconfig gdbm gif gimp gmp gnutls gtk hardened iconv icu idn ipv6 jpeg lame lcms libglvnd libnotify libtirpc lzma mad matroska mmap mmx mmxext mp3 mpeg ncurses networkmanager nptl ogg opengl openmp pam pcre pie png policykit ppds readline sasl sdl seccomp split-usr sse sse2 sse3 sse4 sse4_1 sse4_2 ssl ssp ssse3 startup-notification svg test test-rust theora threads tiff truetype udev udisks unicode upower vaapi verify-sig vorbis wayland x264 xattr xcb xinerama xml xtpax xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_10" RUBY_TARGETS="ruby27 ruby30" USERLAND="GNU" VIDEO_CARDS="crocus intel nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LANG, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-19 23:06:06 UTC
tka, could you report this upstream to bug-diffutils and link it here?
Comment 9 tka 2023-01-28 14:15:47 UTC
The issue was already reported [1] and fixed upstream. According to the comments, it is caused by a bug in make 4.4. Nevertheless, a fix for the test was committed to the diffutils repository [2].

I tested the fix by adding it to /etc/portage/patches and the test suite passed.

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59905
[2] https://git.savannah.gnu.org/cgit/diffutils.git/commit/?id=89f3daeaaf021a77177daec8666bcd80ec476833
Comment 10 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-29 07:29:40 UTC
(In reply to tka from comment #9)
> The issue was already reported [1] and fixed upstream. According to the
> comments, it is caused by a bug in make 4.4. Nevertheless, a fix for the
> test was committed to the diffutils repository [2].
> 
> I tested the fix by adding it to /etc/portage/patches and the test suite
> passed.
> 
> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59905
> [2]
> https://git.savannah.gnu.org/cgit/diffutils.git/commit/
> ?id=89f3daeaaf021a77177daec8666bcd80ec476833

Thanks, sorry for missing that.
Comment 11 Larry the Git Cow gentoo-dev 2023-01-29 07:33:25 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cc023c47f0a30174d43fb940109fce5781edda95

commit cc023c47f0a30174d43fb940109fce5781edda95
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-01-29 07:32:40 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-01-29 07:33:13 +0000

    sys-apps/diffutils: fix tests w/ make 4.4
    
    Closes: https://bugs.gentoo.org/879891
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/diffutils/diffutils-3.9.ebuild            | 10 ++++++--
 .../files/diffutils-3.9-make-4.4-test-color.patch  | 30 ++++++++++++++++++++++
 2 files changed, 38 insertions(+), 2 deletions(-)