Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 297549

Summary: sys-apps/portage --binpkg-respect-use should be the default
Product: Portage Development Reporter: Petr Polezhaev <NightNord>
Component: Binary packages supportAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal CC: dschridde+gentoobugs
Priority: Normal Keywords: InVCS
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 300071, 381649    

Description Petr Polezhaev 2009-12-19 17:43:58 UTC
Vital for containers: if you want to speedup update of many container, which shares same hardware, you are probably using 'emerge -kb' for merges inside containers.

But, if you have a slight different configuration inside containers, for example no-multilib and multilib profiles and update would be done for no-multilib container at first, all multilib systems would be destroyed in no time (if there whould be glibc update, for example), as portage will merge binaries disregarding of USE-flags, even hard-forced/masked.

IMO, portage should merge with --use-binpkgs only if there is _exact_ match of _all_ parameters.

Reproducible: Always

Steps to Reproduce:
1. emerge -kb glibc on no-multilib
2. emerge -kb glibc on multilib
3. forgot about it and run full system update on multilib.
4. Have fun recovering your 32-bit applications



Expected Results:  
glibc should be rebuild. Actually, it would be preffered to keep packages, so that many different configurations of one package may be existing in same PKGDIR. Maybe use sha1 suffix of parameters to difference them? Packages seems to be unusable without "${PKGDIR}/Packages" file, so it should be a great problem, imo.

====emerge --info of one of destroyed containers====

Portage 2.2_rc60 (hardened/linux/amd64/10.0, gcc-4.4.2, glibc-2.11-r1, 2.6.32-gentoo-hellgate-0.10 x86_64)
=================================================================
System uname: Linux-2.6.32-gentoo-hellgate-0.10-x86_64-Intel-R-_Xeon-R-_CPU_E5504_@_2.00GHz-with-gentoo-2.0.1
Timestamp of tree: Fri, 18 Dec 2009 23:45:02 +0000
app-shells/bash:     4.0_p35
dev-lang/python:     2.6.4, 3.1.1-r1
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.5.3
sys-apps/sandbox:    2.2
sys-devel/autoconf:  2.64
sys-devel/automake:  1.10.3, 1.11.1
sys-devel/binutils:  2.20
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -mtune=native -O2 -pipe -floop-interchange -floop-strip-mine -floop-block"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -mtune=native -O2 -pipe -floop-interchange -floop-strip-mine -floop-block"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going"
FEATURES="assume-digests distlocks fixpackages news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LANG="ru_RU.UTF-8"
LC_ALL=""
LDFLAGS="-Wl,--sort-common -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu"
LINGUAS="ru"
MAKEOPTS="-j16"
PKGDIR="/usr/local/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/hardened-development /usr/local/portage/layman/niifaq /usr/local/portage/layman/sunrise /usr/local/portage/layman/rion /usr/local/portage/layman/ltsp"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="acl acpi afs amd64 berkdb bzip2 cli cracklib crypt cups cxx dri gdbm hardened iconv ipv6 justify kerberos ldap mmx modules mudflap multilib ncurses nls nptl nptlonly openmp pam pcre perl pic pppd python readline reflection session snmp spl sse sse2 sse4 ssl ssse3 sysfs tcpd threads unicode urandom vim-syntax xorg zlib zsh-completion" ELIBC="glibc" INPUT_DEVICES="keyboard" KERNEL="linux" LINGUAS="ru" RUBY_TARGETS="ruby18" USERLAND="GNU" 
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Petr Polezhaev 2009-12-19 17:46:39 UTC
s/--use-binpkg/--usepkg/g
Comment 2 Zac Medico gentoo-dev 2009-12-21 02:27:56 UTC
You can use --binpkg-respect-use for this (and --newuse also does it).
Comment 3 Petr Polezhaev 2010-01-01 20:32:19 UTC
I guess it couldn't be a perfect solution. At least, it should be screamed with big red letters before merging, as, I suppose, not much people reading ChangeLogs or checking man for new features for every new portage =). Or, default should be to _respect_ use flags, as I (personally) don't see much sense in merging software with wrong use-flags - it's just useless and may lead to unpredicted situations (I know about EMERGE_DEFAULT_OPTS, thanks).

Also, this not solving problem with multiply rebuilded packages, if containers/hosts where this packages are build are not sorted manually in optimal order. Maybe it would be really better to some kind separate packages with different use-flags? This will also solve a problem with packages for non-native arches, for example, when building i686 packages on amd64 machine, as arch is also use-flag internally.
Comment 4 Zac Medico gentoo-dev 2010-01-02 21:38:53 UTC
(In reply to comment #3)
> I guess it couldn't be a perfect solution. At least, it should be screamed with
> big red letters before merging, as, I suppose, not much people reading
> ChangeLogs or checking man for new features for every new portage =). Or,
> default should be to _respect_ use flags, as I (personally) don't see much
> sense in merging software with wrong use-flags - it's just useless and may lead
> to unpredicted situations (I know about EMERGE_DEFAULT_OPTS, thanks).

I'd prefer not to change the default behavior since that tends to upset users.

> Also, this not solving problem with multiply rebuilded packages, if
> containers/hosts where this packages are build are not sorted manually in
> optimal order. Maybe it would be really better to some kind separate packages
> with different use-flags? This will also solve a problem with packages for
> non-native arches, for example, when building i686 packages on amd64 machine,
> as arch is also use-flag internally.

Yes, we would like to do that (bug #150031).
Comment 5 Sebastian Luther (few) 2011-09-21 14:46:27 UTC
(In reply to comment #4)
> [...]
> I'd prefer not to change the default behavior since that tends to upset users.

Can we please reconsider this?

The old default might have been a usable solution in the days when no such things as use deps existed. The complexity of today's ebuild dependencies confuses both the dependency resolver and the user often enough. We shouldn't add more confusion by letting it randomly ignore user's setting.
Comment 6 Zac Medico gentoo-dev 2011-09-21 15:28:56 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > [...]
> > I'd prefer not to change the default behavior since that tends to upset users.
> 
> Can we please reconsider this?

I still feel the same way about this. If a user specifies --usepkg, it's confusing if the package is silently rejected for some reason. If we make that the default, then at least we have to make it easily detectable in the user interface, and possibly suggest to use --binpkg-respect-use=n in cases when it would make a difference (similar to how we suggest --autounmask=n).

> The old default might have been a usable solution in the days when no such
> things as use deps existed. The complexity of today's ebuild dependencies
> confuses both the dependency resolver and the user often enough. We shouldn't
> add more confusion by letting it randomly ignore user's setting.

I think it's still manageable. You can easily trigger similar conflicts when installed packages get pulled into the graph, since installed packages are also a form of binary packages.
Comment 7 Sebastian Luther (few) 2011-09-21 16:33:14 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > (In reply to comment #4)
> > > [...]
> > > I'd prefer not to change the default behavior since that tends to upset users.
> > 
> > Can we please reconsider this?
> 
> I still feel the same way about this. If a user specifies --usepkg, it's
> confusing if the package is silently rejected for some reason. If we make that
> the default, then at least we have to make it easily detectable in the user
> interface, and possibly suggest to use --binpkg-respect-use=n in cases when it
> would make a difference (similar to how we suggest --autounmask=n).

Maybe we can print something like the missed update messages in this case.

> 
> > The old default might have been a usable solution in the days when no such
> > things as use deps existed. The complexity of today's ebuild dependencies
> > confuses both the dependency resolver and the user often enough. We shouldn't
> > add more confusion by letting it randomly ignore user's setting.
> 
> I think it's still manageable. You can easily trigger similar conflicts when
> installed packages get pulled into the graph, since installed packages are also
> a form of binary packages.

It's not about conflicts, it's about confusion. Sure there are two different ways to confuse the user here, either by ignoring his binary packages or by ignoring his use settings. The first is already easy to detect as emerge prints them differently in the merge list and counts them in the summary below the merge list. The latter is really silent.

Would you change the default, if we had a message like described above?
Comment 8 Zac Medico gentoo-dev 2011-09-21 16:58:25 UTC
(In reply to comment #7)
> Would you change the default, if we had a message like described above?

Sure, sounds reasonable.
Comment 10 Zac Medico gentoo-dev 2011-09-29 18:46:02 UTC
This is fixed in 2.1.10.20 and 2.2.0_alpha60.