Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 435094 - Optional multilib USE flag could cause problems, devs and users could be unaware (was: Force multilib USE flag only on core packages)
Summary: Optional multilib USE flag could cause problems, devs and users could be unaw...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-09-15 06:04 UTC by Matt Turner
Modified: 2014-03-28 15:52 UTC (History)
14 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Turner gentoo-dev 2012-09-15 06:04:05 UTC
For purposes of supporting 32-bit WINE, I'd like to be able to provide 32-bit Mesa. I'd like to be able to add a multilib USE flag to it and its dependencies, thereby allowing users to compile 32-bit Mesa as it's updated, instead of depending on slowly changing emul- binary packages.

I can imagine scenarios where users would like to have a multilib-enabled amd64 profile (and have 32-bit gcc libraries and 32-bit glibc) but not 32-bit Mesa.

I suggest that instead of use.force'ing multilib for all packages, that we package.use.force multilib on just system-essential packages, such as gcc and glibc. We can add multilib to features/multilib's default use flags so that users don't notice a difference unless they explicitly turn multilib off.

The commit diff is trivial, but I guess we need to discuss and write a news item.
Comment 1 Chí-Thanh Christopher Nguyễn gentoo-dev 2012-09-15 11:23:30 UTC
www-plugins/adobe-flash worked around this issue by using the "32bit" flag instead of "multilib".
Comment 2 SpanKY gentoo-dev 2012-09-16 05:16:57 UTC
i'm tempted to drop the use.force completely.  have multilib profiles just default to USE=multilib.  if someone wants to turn it off, well that's their business and they get the pieces.
Comment 3 Matt Turner gentoo-dev 2012-09-16 05:28:18 UTC
(In reply to comment #2)
> i'm tempted to drop the use.force completely.  have multilib profiles just
> default to USE=multilib.  if someone wants to turn it off, well that's their
> business and they get the pieces.

The issue I see with that is that if you turn off multilib entirely, you can't just reset USE=multilib and rebuild everything. I question what's the value of having multilib/no-multilib profiles at all in that case though.
Comment 4 SpanKY gentoo-dev 2012-09-16 20:25:50 UTC
(In reply to comment #3)

switching on the fly between multilib/no-multilib is discussed in another bug.  in reality, gcc & glibc are the only things you have to worry about.  and the SYMLINK_LIB junk.
Comment 5 Matt Turner gentoo-dev 2012-09-16 22:37:07 UTC
(In reply to comment #4)
> (In reply to comment #3)
> 
> switching on the fly between multilib/no-multilib is discussed in another
> bug.  in reality, gcc & glibc are the only things you have to worry about. 
> and the SYMLINK_LIB junk.

Okay. If you're satisfied with removing the use.force completely, it suits my purposes. Thanks!
Comment 6 Ben Kohler gentoo-dev 2012-09-19 16:01:04 UTC
I don't think it's a good idea to allow users to toggle this themselves on the core packages where it's not reversible.  Some users set USE="-* foo bar baz" in a (maybe misguided) attempted to prune down a system, for example.  I'm sure there are many other ways an unsuspecting user can get in trouble with this.  It's a lot easier to accidentally unset multilib than to accidentally switch to a no-multilib profile.

Is there any good reason for not forcing this on the toolchain pieces?
Comment 7 Matt Turner gentoo-dev 2013-01-06 23:10:15 UTC
vapier: are we ready to do this?
Comment 8 Matt Turner gentoo-dev 2013-01-23 03:41:59 UTC
(In reply to comment #7)
> vapier: are we ready to do this?

Maybe now?
Comment 9 SpanKY gentoo-dev 2014-03-21 18:57:19 UTC
it's been a while since i broke something big, so let's do this already

multilib is now only force on for gcc, glibc, and sandbox:

http://sources.gentoo.org/profiles/features/multilib/package.use.force?rev=1.1
http://sources.gentoo.org/profiles/features/multilib/use.mask?rev=1.2
Comment 10 SpanKY gentoo-dev 2014-03-21 18:57:37 UTC
+release team as FYI in case something goes wrong there
Comment 11 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-03-22 14:05:19 UTC
(In reply to SpanKY from comment #9)
> it's been a while since i broke something big, so let's do this already
> 
> multilib is now only force on for gcc, glibc, and sandbox:
> 
> http://sources.gentoo.org/profiles/features/multilib/package.use.force?rev=1.
> 1
> http://sources.gentoo.org/profiles/features/multilib/use.mask?rev=1.2

Hello vapier

There are some possible problems as the result of this change that were brought up on IRC by developers (jlec, Poly-C, dilfridge):

A) The USE flag multilib changed from enabled to disabled on multilib profiles.

B) Packages that assume the presence of that use.force in the profiles, assume multilib is enabled; however, as -multilib is the default, this could break those packages.

An example in an IRC discussion was mentioned to be app-emulation/virtualbox; the maintainer would have been unaware of this change, as well as could be unsure on how to change this to guarantee what the assumption brought before.

C) We might be wrong on this; but it might be possible that due to moving from forcing all multilib implementations (use.force:multilib) to not forcing them all, that this requires the introduction of a construct like [${PYTHON_USEDEP}] as to be able to pick form for example MULTILIB_ABIS="n32 o32 n64" in mips.

Given these concerns, I suggest the following workflow to improve the situation:

1) Set the USE flag multilib to be enabled by default on multilib profiles; more specifically, by placing it in IUSE in make.defaults on the multilib feature.

2) Bring this up for discussion on the gentoo-dev ML such that people that would be concerned are aware about this and can comment on the situation and/or adapt to it; the result of this discussion, to make sure everyone is aware, could be send to the gentoo-dev-announce ML.

3) A news item could be send to the users, to make aware they have more control;  alternatively, or as well, a news item for the particular packages seeing changes could be brought out too. Like Matt's WINE users being able to use Mesa.

As far as I am aware from multiple communication mediums, there is no perceived breakage this early. If there's breakage somewhere or provable that breakage is definitely possible, I'll go ahead and apply step (1) from above. As I think at least bringing its default back to the enabled state, without reverting to force it; doesn't object the idea that's being brought forward here.

What are your thoughts? Can you do the above or can we? Do you want to revert?

Thank you in advance.
Comment 12 Jorge Manuel B. S. Vicetto (RETIRED) Gentoo Infrastructure gentoo-dev 2014-03-22 15:59:00 UTC
It seems Mike have failed to notice this yesterday on IRC:

20:41 < jmbsvicetto> Following a link given by vapier, it seems like sources.go is "slightly" broken
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/use.mask?revision=1.2 <- can you see rev 1.2 on
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/use.mask?view=log ?
21:30 <@robbat2> jmbsvicetto, no, the varnish cache for that page hasn't expired yet
21:32 < jmbsvicetto> ok
21:33 < jmbsvicetto> robbat2: but if it's hitting the cache, shouldn't it show up an error as there was no rev 1.2 before?
21:33 <@robbat2> 404 get much shorter cache times than 200 ok changed pages
21:33 <@robbat2> nothing would have linked to 1.2 before       
21:38 < jmbsvicetto> I see, thanks
21:39 < jmbsvicetto> http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/use.mask?revision=1.2 is the same as http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/use.mask?revision=1.1, though
21:41 <@robbat2> that's weirder
21:42 < jmbsvicetto> and http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/use.mask?revision=1.3
21:42 <@robbat2> actually, look at the 1.2 page closer for the annotate view
21:42 <@robbat2> it's also the 1.1 rev
21:42 <@robbat2> when did he commit?
21:43 < jmbsvicetto> I've poked vapier as I don't see the commit in #-commits
21:43 < jmbsvicetto> asking if he did actually commit it or not
21:43 <@robbat2> RCS file: /var/cvsroot/gentoo-x86/profiles/features/multilib/use.mask,v
21:43 <@robbat2> Working file: use.mask
21:43 <@robbat2> head: 1.1
21:43 <@robbat2> i don't think he did
21:44 < jmbsvicetto> vapier: so you forgot to commit

Tom,

it seems like Mike only got half of https://bugs.gentoo.org/show_bug.cgi?id=435094#c9 done
Comment 13 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-03-22 16:29:10 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #12)
> it seems like Mike only got half of
> https://bugs.gentoo.org/show_bug.cgi?id=435094#c9 done

Either that or he mentioned use.mask instead of use.force by accident, which I think is more evident given that he didn't mention the removal of use.force; as well as by the revisions of the following URL and the commit that happened.

http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/features/multilib/?hideattic=0
Comment 14 Norman Back 2014-03-23 11:14:01 UTC
(In reply to SpanKY from comment #9)
> it's been a while since i broke something big, so let's do this already
> 
> multilib is now only force on for gcc, glibc, and sandbox:
> 
> http://sources.gentoo.org/profiles/features/multilib/package.use.force?rev=1.
> 1
> http://sources.gentoo.org/profiles/features/multilib/use.mask?rev=1.2

Well, That was a nasty suprise. Removing multilib caught me in the middle trying to resolve an issue between mythtv and nvidia-drivers-334.21 and made it even more confusing. After restoring from backups, eix-sync, masking nvidia-drivers-334.21, adding multilib to USE in make.conf and emerge -DuNavt workd, I am now (after 2 days) running OK.

Please confirm adding USE in make.conf is the right solution to this issue.

emerge --info
Portage 2.2.8-r1 (default/linux/amd64/13.0/desktop, gcc-4.7.3, glibc-2.17, 3.13.6-gentoo-64 x86_64)
=================================================================
System uname: Linux-3.13.6-gentoo-64-x86_64-AMD_Phenom-tm-_II_X4_965_Processor-with-gentoo-2.2
KiB Mem:    16446240 total,   9607900 free
KiB Swap:   34602996 total,  34602996 free
Timestamp of tree: Sun, 23 Mar 2014 02:45:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
distcc 3.1 x86_64-pc-linux-gnu [disabled]
ccache version 3.1.9 [disabled]
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r3, 3.3.3
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.9.6-r3, 1.11.6, 1.12.6, 1.13.4
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.6.3, 4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo local ikelos systemd
Installed sets: @kde-4.11, @kernels
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=amdfam10 -mtune=amdfam10 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/themes/oxygen-gtk/gtk-2.0 /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=amdfam10 -mtune=amdfam10 -O2 -pipe"
DISTDIR="/mnt/portage.autofs/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y -j9 --load-average=8 --quiet-build=y"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirror.qubenet.net/mirror/gentoo/ http://mirrors.linuxant.fr/distfiles.gentoo.org/ ftp://mirror.qubenet.net/mirror/gentoo/ http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo ftp://ftp.free.fr/mirrors/ftp.gentoo.org/ http://mirror.leaseweb.com/gentoo/ http://gentoo.modulix.net/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://mirror.bytemark.co.uk/gentoo/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j9 --load-average=8"
PKGDIR="/mnt/portage.autofs/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /var/lib/layman/ikelos /var/lib/layman/systemd"
SYNC="rsync://datastoreone/gentoo-portage"
USE="3dnow X a52 aac accessibility acl acpi alsa amarok amd64 amr authdaemond bash-completion berkdb bluetooth bluray branding bzip2 cairo caps cdda cdr cli consolekit cracklib crypt cups cxx dbus dri dts dv dvd dvdr dvdread emboss encode exif fam firefox flac fortran gdbm gif gpm gtk iconv ieee1394 ipv6 jpeg kde lcms ldap libnotify lm_sensors lock mad mmx mng modules mp3 mp4 mpeg multilib mysql ncurses network nls nptl ofx ogg opengl openmp oss pam pango pcre pdf png policykit ppds qt3 qt3support qt4 readline samba sasl sdl semantic-desktop session spell sse sse2 ssl startup-notification svg tcpd thunar tiff truetype udev udisks uk_bleb uk_rt unicode upower usb v4l v4l2 vdpau vorbis wxwidgets x264 xcb xine xinerama xml xv xvid xvmc zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias cgi cgid" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="lvm syslog" DVB_CARDS="usb-dib0700" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19 ruby20" SANE_BACKENDS="hp5590 hp net abaton agfafocus apple artec artec_eplus48u as6e avision bh canon canon630u canon_dr canon_pp cardscan coolscan coolscan2 coolscan3 dc210 dc240 dc25 dell1600n_net dmc epjitsu epson epson2 fujitsu genesys gt68xx hp3500 hp3900 hp4200 hp5400 hpljm1005 hpsj5s hs2p ibm kodak kvs1025 kvs20xx leo lexmark ma1509 magicolor matsushita microtek microtek2 mustek mustek_pp mustek_usb nec niash p5 pie pixma plustek plustek_pp pnm qcam ricoh rts8891 s9036 sceptre sharp sm3600 sm3840 snapscan sp15c st400 stv680 tamarack teco1 teco2 teco3 test u12 umax umax1220u umax_pp xerox_mfp" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 15 Nicholas O'Connor 2014-03-24 18:16:38 UTC
(In reply to Norman Back from comment #14)
> Please confirm adding USE in make.conf is the right solution to this issue.

Adding USE="multilib" to make.conf and rebuilding affected packages solved the problem for me (in my case it was libstdc++ and nvidia-drivers to solve Steam not having access to graphics libraries)
Comment 16 Ben Kohler gentoo-dev 2014-03-24 18:34:23 UTC
Is there some reason multilib is not set in make.defaults for multilib profiles, as comment #2 suggested?  Moving from FORCED ON to default-off seems like an odd jump, shouldn't it be default-on still?
Comment 17 SpanKY gentoo-dev 2014-03-25 16:03:57 UTC
(In reply to SpanKY from comment #9)

i didn't modify use.mask, i deleted use.force:
http://sources.gentoo.org/profiles/features/multilib/use.force?view=log

(In reply to Tom Wijsman (TomWij) from comment #11)

there might be a bit of breakage with USE=-multilib, but i doubt it's big, and it's not easy to reverse it.  people can fix it as they notice.

there is no need for USEDEP junk just for USE=multilib.  if a package wants to convert to the multilib framework, they're free to do so, and that's completely orthogonal to USE=multilib handling.

i'm ambivalent to setting USE=multilib in make.defaults.  packages should generally migrate away from it to the multilib eclasses if they want to handle things like dependencies on diff ABIs.  i know some packages in the tree def should not be utilizing USE=multilib (like libaio & apitrace) and have always been kind of broken.
Comment 19 SpanKY gentoo-dev 2014-03-27 01:55:44 UTC
USE=multilib is on by default now

http://sources.gentoo.org/profiles/features/multilib/make.defaults?r1=1.11&r2=1.12
Comment 20 SpanKY gentoo-dev 2014-03-27 02:09:29 UTC
Commit message: Convert from USE=multilib to multilib-minimal eclass
http://sources.gentoo.org/dev-libs/libaio/libaio-0.3.109-r5.ebuild?rev=1.1
Comment 21 Justin Lecher (RETIRED) gentoo-dev 2014-03-27 07:08:51 UTC
(In reply to SpanKY from comment #19)
> USE=multilib is on by default now
> 
> http://sources.gentoo.org/profiles/features/multilib/make.defaults?r1=1.
> 11&r2=1.12

Thanks Mike!