| Summary: | sys-apps/portage - libncurses.so.5 was not preserved | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | stanton_arch |
| Component: | Unclassified | Assignee: | Portage team <dev-portage> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | axiator, base-system, esigra, floppym, gletonai, graaff, hossie, jer, nbowler, pacho, teika |
| Priority: | Normal | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 240323, 611328 | ||
| Attachments: | ncurses install log with postinst failure (compressed) | ||
|
Description
stanton_arch
2016-10-25 15:36:44 UTC
The portage feature "preserve-libs" should have prevented libncurses.so.5 from being removed before all reverse dependencies were rebuilt. I am therefore calling this a portage bug until proven otherwise. ;) Somebody should unpack an hppa stage3, chroot in there, and see if the ncurses libraries are properly preserved when ncurses is unmerged:
emerge --unmerge ncurses
There is nothing HPPA specific about that. I've seen it on other architectures as well. Created attachment 456112 [details] ncurses install log with postinst failure (compressed) Just hit this on x32. The ncurses upgrade explodes in postinst because bash is hosed at this point (libs were not preserved). Then all subsequent package installations fail immediately. (In reply to Zac Medico from comment #2) > Somebody should unpack an hppa stage3, chroot in there, and see if the > ncurses libraries are properly preserved when ncurses is unmerged: > > emerge --unmerge ncurses With current stage3-x32-20161208.tar.bz2 the libs are preserved with this command. However, they do not seem to be properly preserved if you attempt to downgrade to ncurses-5 (unfortunately, stages with ncurses-5 seem to be no longer available to test the upgrade path): % emerge --unmerge sys-libs/ncurses (libs are preserved, cool!) % emerge -1 '=sys-libs/ncurses-5.9-r5' (for some reason, most of the preserved ncurses-6 libs disappear after this, breaking many packages, but bash and portage are still working...) % emerge -1 sys-libs/readline (attempt to rebuild ncurses reverse dependency. When this package is reinstalled, all remaining preserved libs disappear and the installation fails in postinst. All subsequent emerge attempts fail.) Portage 2.3.0 (python 3.4.3-final-0, default/linux/amd64/13.0/x32, gcc-4.9.3, glibc-2.22-r4, 4.5.4 x86_64) ================================================================= System uname: Linux-4.5.4-x86_64-Intel-R-_Core-TM-_i7-5600U_CPU_@_2.60GHz-with-gentoo-2.2 KiB Mem: 4048828 total, 242884 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Tue, 13 Dec 2016 00:45:01 +0000 sh dash 0.5.8.2 ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1 app-shells/bash: 4.3_p46-r1::gentoo dev-lang/perl: 5.22.2::gentoo dev-lang/python: 2.7.10-r1::gentoo, 3.4.3-r1::gentoo dev-util/cmake: 3.5.2-r1::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.21.7::gentoo sys-apps/sandbox: 2.10-r1::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69::gentoo sys-devel/automake: 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25.1-r1::gentoo sys-devel/gcc: 4.9.3::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.6::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 4.3::gentoo (virtual/os-headers) sys-libs/glibc: 2.22-r4::gentoo Repositories: gentoo location: /srv/repos/gentoo sync-type: webrsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage sync-user: portage priority: -1000 gentoo-draconx location: /srv/repos/gentoo-draconx masters: gentoo ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnux32" CFLAGS="-O2 -march=broadwell -pipe" CHOST="x86_64-pc-linux-gnux32" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=broadwell -pipe" DISTDIR="/var/cache/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --dynamic-deps=n --autounmask-write=n --quiet-build --binpkg-respect-use=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs binpkg-multi-instance config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync webrsync-gpg xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" INSTALL_MASK="/etc/portage/bin/post_sync" LANG="en_CA.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j2" PKGDIR="/var/cache/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X acl amd64 berkdb bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 mmx mmxext modules multilib ncurses nls nptl openmp pam pcre png readline seccomp session sse sse2 ssl svg tcpd truetype unicode xattr xft zlib" ABI_X86="x32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="modesetting" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON *** Bug 602672 has been marked as a duplicate of this bug. *** Confirm on AMD64 I think the problem is not about preserving the libs, but about not emerging a usable ncurses. For me, this happened: (doing some updates) [...] <<< dir /usr/share/doc/kate-16.08.3 * Updating icons cache ... [ ok ] * Updating .desktop files database ... [ ok ] >>> Regenerating /etc/ld.so.cache... >>> Original instance of package unmerged safely. * Updating icons cache ... [ ok ] * Updating .desktop files database ... [ ok ] >>> kde-apps/kate-16.12.0 merged. >>> Auto-cleaning packages... sys-libs/ncurses selected: 6.0-r1 protected: 6.0 omitted: none All selected packages: =sys-libs/ncurses-6.0-r1 Afterwards, the libncurses was preserved, but all files from ncurses were gone: ~ $ htop Error opening terminal: xterm. ~ $ ldd $(which xterm) | grep ncurses libncurses.so.6 => /lib64/libncurses.so.6 (0x00007f6081c30000) ~ $ ls -l /lib/*ncurses* lrwxrwxrwx 1 root root 17 Dec 17 10:50 /lib/libncurses.so.6 -> libncurses.so.6.0 -rwxr-xr-x 1 root root 382680 Dec 17 10:51 /lib/libncurses.so.6.0 lrwxrwxrwx 1 root root 18 Dec 17 10:51 /lib/libncursesw.so.6 -> libncursesw.so.6.0 -rwxr-xr-x 1 root root 469144 Dec 17 10:51 /lib/libncursesw.so.6.0 ~ $ equery f ncurses * Searching for ncurses ... * Contents of sys-libs/ncurses-6.0: also happened yesterday for me: I emerged a package, did not pay attention to what's happening and suddenly all ncurses related files were gone and all newly started terminal programs unusable to due missing terminfo database :-( Immediately rebuilding ncurses failed with "tic missing" something, but then I somehow succeeded and got a working system back Almost the same as Comment #7, after upgrading a package. Backgrounds: On Dec 15, I tried "emerge -pDu world", and portage said ------------------------------------------------------------------------ The following packages are causing rebuilds: (sys-libs/ncurses-6.0-r1:0/6::gentoo, ebuild scheduled for merge) causes rebuilds for: (app-text/aspell-0.60.6.1-r1:0/0::gentoo, ebuild scheduled for merge) .... ------------------------------------------------------------------------ Then I manually did: $ emerge --oneshot app-text/aspell ... sys-libs/ncurses:{0,5} It caused the updating of ncurses 5.9-r5 -> 6.0-r1, and 5.9-r99 -> 6.0 Today, portage removed ncurses-6.0-r1, but 6.0 remained, as reported by Daniel Augstin in Comment #7 BTW: I had to manually scroll my X terminal to find this message: ------------------------------------------------------------------------ >>> Auto-cleaning packages... sys-libs/ncurses selected: 6.0-r1 protected: 6.0 omitted: none All selected packages: =sys-libs/ncurses-6.0-r1 ------------------------------------------------------------------------ Is it possible to save these messages by elog, using make.conf? I've got PORT_LOGDIR=/my/path/to/elog, and per-package logs get saved, but not the above one. Thanks gentoo devs. Happy New Year. (In reply to teika from comment #9) > Today, portage removed ncurses-6.0-r1, but 6.0 remained, as reported by > Daniel Augstin in Comment #7 > > BTW: I had to manually scroll my X terminal to find this message: > ------------------------------------------------------------------------ > >>> Auto-cleaning packages... > > sys-libs/ncurses > selected: 6.0-r1 > protected: 6.0 > omitted: none > > All selected packages: =sys-libs/ncurses-6.0-r1 > ------------------------------------------------------------------------ I've filed bug 603658 for that issue. (In reply to Nick Bowler from comment #4) > Created attachment 456112 [details] > ncurses install log with postinst failure (compressed) Using stage3-x32-20161201, I was able to to partially reproduce the problem shown in the above log. In my case, libncurses.so.5 was successfully preserved, but not libncursesw.so.5 and libpanelw.so.5: >>> needed sym /libx32/libncurses.so.5 >>> needed obj /libx32/libncurses.so.5.9 >>> needed sym /libx32/libncursesw.so.5 >>> needed obj /libx32/libncursesw.so.5.9 >>> needed sym /usr/lib64/libncurses.so.5 >>> needed obj /usr/lib64/libncurses.so.5.9 >>> needed sym /usr/libx32/libpanelw.so.5 >>> needed obj /usr/libx32/libpanelw.so.5.9 >>> Safely unmerging already-installed instance... >>> Regenerating /etc/ld.so.cache... >>> Original instance of package unmerged safely. >>> sys-libs/ncurses-6.0-r1 merged. >>> Regenerating /etc/ld.so.cache... <<< !needed sym /libx32/libncursesw.so.5 <<< !needed obj /libx32/libncursesw.so.5.9 <<< !needed sym /usr/libx32/libpanelw.so.5 <<< !needed obj /usr/libx32/libpanelw.so.5.9 Afterwards less was broken: less: error while loading shared libraries: libncursesw.so.5: cannot open shared object file: No such file or directory The stage3-x32-20161201 appears to have correct NEEDED.ELF.2 data involving libncursesw.so.5: X86_64;/libx32/libncursesw.so.5.9;libncursesw.so.5;;libc.so.6;x86_x32 X86_64;/usr/bin/less;;;libncursesw.so.5,libpcre.so.1,libc.so.6;x86_x32 I'll have to use pdb to see what's going on here. Patch posted for review: https://archives.gentoo.org/gentoo-portage-dev/message/54cc47ef5b19ffe46e18ef3839d3edf6 https://github.com/gentoo/portage/pull/78 (In reply to Morton Pellung from comment #8) > Immediately rebuilding ncurses failed with "tic missing" something, > but then I somehow succeeded and got a working system back "Somehow." Aaargh. :-) In case anyone else needs a workaround: it seems to help to first re-emerge the ncurses-5.9:0 version and then upgrade again to ncurses-6.0:0. This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=cbaee3c3b28f52947c142da8df24d6b0817962f9 (In reply to Zac Medico from comment #11) > (In reply to Nick Bowler from comment #4) > > Created attachment 456112 [details] > > ncurses install log with postinst failure (compressed) > > Using stage3-x32-20161201, I was able to to partially reproduce the problem > shown in the above log. In my case, libncurses.so.5 was successfully > preserved, but not libncursesw.so.5 and libpanelw.so.5: > > >>> needed sym /libx32/libncurses.so.5 > >>> needed obj /libx32/libncurses.so.5.9 > >>> needed sym /libx32/libncursesw.so.5 > >>> needed obj /libx32/libncursesw.so.5.9 > >>> needed sym /usr/lib64/libncurses.so.5 > >>> needed obj /usr/lib64/libncurses.so.5.9 > >>> needed sym /usr/libx32/libpanelw.so.5 > >>> needed obj /usr/libx32/libpanelw.so.5.9 > >>> Safely unmerging already-installed instance... > >>> Regenerating /etc/ld.so.cache... > >>> Original instance of package unmerged safely. > >>> sys-libs/ncurses-6.0-r1 merged. > >>> Regenerating /etc/ld.so.cache... > <<< !needed sym /libx32/libncursesw.so.5 > <<< !needed obj /libx32/libncursesw.so.5.9 > <<< !needed sym /usr/libx32/libpanelw.so.5 > <<< !needed obj /usr/libx32/libpanelw.so.5.9 That looks a bit like this: 1) Forcibly remove some unneeded and masked version of dev-lang/ruby. 2) Then immediately forcibly remove all virtual/perl- packages: >>> 'Protected' and 'omitted' packages will not be removed. >>> Waiting 5 seconds before starting... >>> (Control-C to abort)... >>> Unmerging in: 5 4 3 2 1 >>> Unmerging (1 of 71) virtual/perl-Archive-Tar-2.40.0-r1... <<< !needed sym /usr/lib/libruby20.so.2.0 <<< !needed obj /usr/lib/libruby20.so.2.0.0 >>> Unmerging (2 of 71) virtual/perl-Attribute-Handlers-0.970.0... Wait what? Removing a new-style virtual which installs (or removes) no files triggers the removal of preserved libraries! (In reply to Jeroen Roovers from comment #15) > Wait what? Removing a new-style virtual which installs (or removes) no files > triggers the removal of preserved libraries! Yes, that's consistent with the bug fixed in cbaee3c3b28f52947c142da8df24d6b0817962f9. After some libraries are preserved, the first call to _find_unused_preserved_libs would behave incorrectly because LinkageMapElf used _approx_multilib_categories for preserved libraries, instead of using compute_multilib_category with the ELF header. Fixed in portage-2.3.5. |
