I recently switched my system shell to dash. Some days later, after re-emerging bash, I noticed that the /bin/sh symlink had been removed by portage. <snip> <<< sym /bin/sh --- replaced sym /bin/rbash --- replaced obj /bin/bash --- replaced dir /bin </snip> I'm not sure whether this is a bug or intended behaviour, but in any case I'd like ebuilds to stop messing with /bin/sh and preferably handle that symlink through an eselect module as suggested in bug #214817. Reproducible: Always Steps to Reproduce: 1. cd /bin 2. ln -sf bash sh 3. emerge -1 bash (/bin/sh is recorded in CONTENTS for app-shells/bash) 4. ln -sf dash sh 5. emerge -1 bash (bash ebuild doesn't recreate the symlink, then portage unmerges it) 6. ls -l sh Actual Results: ls: cannot access sh: No such file or directory Expected Results: lrwxrwxrwx 1 root root 4 May 18 22:21 /bin/sh -> dash Portage 2.1.5 (default-linux/x86/2007.0/desktop, gcc-4.2.3, glibc-2.7-r2, 2.6.25-gentoo-r4 i686) ================================================================= System uname: 2.6.25-gentoo-r4 i686 Intel(R) Pentium(R) 4 CPU 3.00GHz Timestamp of tree: Sun, 18 May 2008 10:15:01 +0000 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 1.3.7, 2.1.6 dev-lang/python: 2.5.2-r3 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 2.0.0 sys-apps/openrc: 9999 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.62 sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 2.2.4 virtual/os-headers: 2.6.25-r3 ACCEPT_KEYWORDS="x86 ~x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=prescott -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/splash /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -march=prescott -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://mirror.ing.unibo.it/gentoo/ ftp://ftp.unina.it/pub/linux/distributions/gentoo/ ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo/" LANG="it_IT.UTF-8" LC_ALL="it_IT.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" LINGUAS="it" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_EXTRA_OPTS="--prune-empty-dirs" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/pesa /usr/portage/local/layman/gentopia" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X a52 aac acl acpi adns alsa audiofile avahi bash-completion berkdb bzip2 cairo caps cddb cdinstall cdr cli crypt cups curl dbus dri dts dv dvd dvdr dvdread emacs emboss encode evo exif expat fam ffmpeg fftw flac ftp gd gdbm geoip gif glut gmp gnutls gpm graphviz hal iconv idn ieee1394 imagemagick imlib innodb isdnlog jabber jack javascript jbig jpeg jpeg2k kde kdeenablefinal lcms ldap libsamplerate mad matroska midi mmap mmx mng mozilla mp3 mpeg mplayer msn mudflap mule musepack musicbrainz mysql mysqli ncurses nls nptl nptlonly nsplugin ocaml offensive ogg opengl openmp pam pcre pdf png pppd pulseaudio python qt3 qt3support qt4 quicktime readline reflection ruby samba sasl sdl session slang sndfile snmp socks5 speex spell spl sqlite sqlite3 sse sse2 ssl svg tcpd tetex theora threads tiff truetype unicode vcd vorbis wavpack win32codecs wmf x264 x86 xcb xcomposite xine xml xorg xosd xpm xulrunner xv xvid zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="it" USERLAND="GNU" VIDEO_CARDS="nvidia nv vesa" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
From bash-3.2_p39.ebuild: pkg_preinst() { [...] # If /bin/sh does not exist or is bash, then provide it # Otherwise leave it alone if [[ ! -e ${ROOT}/bin/sh ]] ; then ln -s bash "${ROOT}"/bin/sh elif [[ -L ${ROOT}/bin/sh ]] ; then case $(readlink "${ROOT}"/bin/sh) in bash|/bin/bash) cp -pPR "${ROOT}"/bin/sh "${D}"/bin/ ;; esac fi }
If there's a file collision, the packages need to block each other. You can set FEATURES="collision-protect" in make.conf if you want to try to protect yourself from bugs like this.
(In reply to comment #2) > If there's a file collision, the packages need to block each other. You can set > FEATURES="collision-protect" in make.conf if you want to try to protect > yourself from bugs like this. > A file collision? Between which packages? I always emerged only bash...
I has assumed that dash owned the /bin/sh symlink, but I guess not. Nevermind.
latest bash should make sure /bin/sh always exists ... portage will probably cull it for you during the pkg_* stage, but the ebuild will make sure to recreate it http://sources.gentoo.org/app-shells/bash/bash-3.2_p39.ebuild?r1=1.3&r2=1.4
And the package manager would remove it if that symlink is part of the version you are upgrading from. That shouldn't be done on pkg_postinst. - ferdy
I meant to say '...is part of the CONTENTS of the version...'.
it should not be part of CONTENTS ... that's what caused problems in the first place
(In reply to comment #8) > it should not be part of CONTENTS ... that's what caused problems in the first > place The point is that when upgrading to the new version of bash, /bin/sh is removed, leaving the system broken - sure, it's easy to fix, but users shouldn't have to do that (and it won't be much fun for openrc users if they don't notice before the next boot....)
(In reply to comment #9) > (In reply to comment #8) > > it should not be part of CONTENTS ... that's what caused problems in the first > > place > > The point is that when upgrading to the new version of bash, /bin/sh is > removed, leaving the system broken - sure, it's easy to fix, but users > shouldn't have to do that (and it won't be much fun for openrc users if they > don't notice before the next boot....) Which should be fixed before being RESOLVED FIXED...
Ingmar, take a look at ebuild. As I see it is FIXED in bash-3.2_p39: 31 May 2008; Mike Frysinger <vapier@gentoo.org> bash-3.2_p39.ebuild: Make sure /bin/sh always exists #222721 by Davide Pesavento. pkg_postinst() { # If /bin/sh does not exist, provide it [[ ! -e ${ROOT}/bin/sh ]] && ln -sf bash "${ROOT}"/bin/sh }
It's not fixed. Think about what happens when you upgrade when /bin/sh is owned by an older version.
Yeah, NOT fixed for me too. IMHO the only clean solution to this issue is handling /bin/sh through an eselect module (to be written), as already suggested in bug #214817.
which wouldnt help this issue at all. /bin/sh cannot be part of the CONTENTS.
Well, the current 'fix' doesn't work.
We all agree that /bin/sh must not be part of CONTENTS of any package, don't we? What can be done to remove that symlink from bash's CONTENTS without breaking users' systems? Now, as far as I can see, every user that upgrades bash will be left without /bin/sh, which is very bad. A possible workaround is to recreate the symlink in pkg_postrm(), which is run last during upgrades. Anyway I still have to understand why portage unmerges a file which has been modified since the installation.
(In reply to comment #16) > Anyway I still have to understand why portage unmerges a file which has been > modified since the installation. It is due to the unmerge-orphans feature (see bug #195527). With that feature enabled, files are uninstalled regardless of their mtime (unless they are protected by CONFIG_PROTECT).
I've tested uninstalling bash with CONFIG_PROTECT="/bin/sh" and it correctly leaves the /bin/sh symlink intact while uninstalling the rbash symlink: --- !mtime sym /bin/sh <<< sym /bin/rbash Since /bin/sh is so vital, it might be useful to install a /etc/env.d/00shell file setting CONFIG_PROTECT="/bin/sh".
no one has shown how it is broken for them. it works fine on my systems where /bin/sh is part of CONTENTS and then removed (and portage trims the symlink). so either post real information or leave the bug alone.
(In reply to comment #19) > no one has shown how it is broken for them. it works fine on my systems where > /bin/sh is part of CONTENTS and then removed (and portage trims the symlink). > so either post real information or leave the bug alone. Suppose you have an older bash ebuild installed, that has the symlink in CONTENTS (an actual previous version, not the current version from before the ebuild was changed) and you want to upgrade. The sequence is (skipping irrelevant steps): * install new bash to the filesystem, sans /bin/sh * run pkg_postinst for the new bash + pkg_postinst sees that the symlink already exists on the filesystem, so doesn't touch it * remove the old bash, including /bin/sh since it isn't part of the new version and hasn't been modified or had its mtime updated
Can something be done about this, please?
Soo... investigating this some further it turns out that portage changed its phase ordering on upgrades between latest stable, 2.1.4.4 and 2.1.5.2. So with ~arch portage (which I assume vapier used to test this) pkg_postinst is indeed after the unmerge. Likewise for pkgcore 0.4.7.3. So for those the fix to recreate /bin/sh works. For stable portage and paludis on the other hand, pkg_postinst is before the unmerge (which is consistent with PMS and with long-term portage behaviour). Zac's proposal to CONFIG_PROTECT /bin/sh would have worked, had it been done ages ago. But letting bash install it or export the variable does not affect the unmerge at all. The only sensible solution I can come up with is to rewrite the symlink in pkg_postinst if it exists. This fixes the issue for for portage when FEATURES=unmerge-orphans is disabled as well as for paludis. Therefore I have committed a fix that makes it die in pkg_setup when unmerge-orphans is enabled, portage < 2.1.5 and bash < 3.2_p39. Further the fix rewrites the symlink in pkg_postinst. http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-shells/bash/bash-3.2_p39.ebuild?r1=1.4&r2=1.5 I am marking this bug fixed as this should resolve all issues with /bin/sh which is what this bug is really about. I do, however, think it's wrong to suddenly change phase ordering like this with no notice and no EAPI bump. CC'ing pms-bugs to let them know about it.
(In reply to comment #22) > do, however, think it's wrong to > suddenly change phase ordering like this with no notice and no EAPI bump. > CC'ing pms-bugs to let them know about it. Older portage actually uses a different phase order whether it's a reinstall of the same version or an upgrade. New portage uses the same order in both cases (the same order which was only used for reinstall in older portage).
(In reply to comment #23) > (In reply to comment #22) > > do, however, think it's wrong to > > suddenly change phase ordering like this with no notice and no EAPI bump. > > CC'ing pms-bugs to let them know about it. > > Older portage actually uses a different phase order whether it's a reinstall of > the same version or an upgrade. New portage uses the same order in both cases > (the same order which was only used for reinstall in older portage). Very good, you've shown us that you know what "change" means. Half the point of EAPI¹ is exactly to make changes like this one while hiding the ebuild from package managers that implement the old behaviour. In this case, before zlin fixed it, anyone upgrading to the new version of bash with a PMS-compliant package manager was left with a broken system, because the ebuild assumed that the package manager would implement the new behaviour but had to way to enforce that. The new behaviour /is/ probably a good idea, yes, but it's EAPI 2 material, not something to just do whenever you feel like it. [1] the other half is to make changes that would break existing ebuilds, by allowing the ebuilds to declare whether they want the old or new behaviour
(In reply to comment #24) > Very good, you've shown us that you know what "change" means. Please don't use that tone with me, son. (In reply to comment #24) > assumed that the package manager would implement the new behaviour but had to > way to enforce that. The new behaviour /is/ probably a good idea, yes, but > it's EAPI 2 material, not something to just do whenever you feel like it. I'd say that EAPI 2 should mandate the new behavior. However, I don't think we should mandate the old behavior for < EAPI2, it's beneficial to have the EAPI 0 and 1 ebuilds using the same order as EAPI 2 for consistency's sake. It shouldn't make a difference in the vast majority of cases. We might hit a few snags, but overall it's probably better to have consistent phase order between EAPI 0, 1, and 2.
(In reply to comment #23) > Older portage actually uses a different phase order whether it's a reinstall > of the same version or an upgrade. Which is consistent with long-term portage behaviour and PMS. > New portage uses the same order in both cases (the same order which was only > used for reinstall in older portage). Does this mean that you intentionally changed this without telling anybody about it?
(In reply to comment #26) > > New portage uses the same order in both cases (the same order which was only > > used for reinstall in older portage). > > Does this mean that you intentionally changed this without telling anybody > about it? Well, considering that it already behaved this way for reinstall of the same package, I didn't consider it to be especially noteworthy. In retrospect however, I think it would have been better to do an RFC or at least an announcement of some kind. (In reply to comment #22) > Therefore I have committed a fix that makes it die in pkg_setup when > unmerge-orphans is enabled, portage < 2.1.5 and bash < 3.2_p39. Further the fix > rewrites the symlink in pkg_postinst. > > http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-shells/bash/bash-3.2_p39.ebuild?r1=1.4&r2=1.5 I think a blocker would be a much better solution. If we make bash-3.2_p39 block <portage-2.1.5 then the blocker will behave much like the one from bug 190128. The blocker will be solved automatically for people upgrading from portage-2.1.4.x to portage-2.5.5.x since portage-2.1.4.x has the ability to resolve blockers by adjusting install order so that portage is upgraded to portage-2.1.5.x before bash is upgraded to bash-3.2_p39.
I've replaced the !<sys-apps/portage-2.1.4_rc1 blocker (from bug #190128) with !<sys-apps/portage-2.1.5 to ensure that phase execution order is such that the /bin/sh symlink will be created in the postinst phase when necessary. This eliminates the need for an associated die call in pkg_setup(). http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-shells/bash/bash-3.2_p39.ebuild?r1=1.5&r2=1.6
(In reply to comment #25) > I'd say that EAPI 2 should mandate the new behavior. However, I don't think we > should mandate the old behavior for < EAPI2, it's beneficial to have the EAPI 0 > and 1 ebuilds using the same order as EAPI 2 for consistency's sake. Doing that means people might inadvertently start relying on the new behaviour even in EAPI 0 and 1, as happened with this bug.
Uh, doesn't the Portage change break people who were relying upon using has_version to detect upgrades / downgrades / reinstalls in pkg_*?
Using has_version to detect upgrades / downgrades / reinstalls in pkg_preinst should work fine. They can cache the result in an environment variable if they need it for postinst.
And how does that work for EAPI 0/1 things that don't do that currently?
Much of the breakage is probably going to be in pkg_postinst() has_version conditionals that are meant to be triggered on upgrade or downgrade. The state during execution of the pkg_postinst() phase will be practically identical how it is for a reinstall operation, so it it should behave well in most cases. Hopefully this /bin/sh problem will be the worst of the issues, but only time will tell.
Quoting zmedico in bug 240227 comment 2: > This sort of change requires an EAPI bump since we don't want ebuild authors > relying on new behavior in ebuilds that might be installed by an older version > of portage. Please give one good reason why this logic does not also apply to the phase ordering change (this being completely independent of the issue of breaking existing ebuilds by changing documented, long-established behaviour).
(In reply to comment #34) > Please give one good reason why this logic does not also apply to the phase > ordering change (this being completely independent of the issue of breaking > existing ebuilds by changing documented, long-established behaviour). If if we make the phase order dependent upon EAPI then the logical order of execution is ambiguous in cases when the package being installed and the one it replaces have different EAPIs. I've given some examples in the following email: http://archives.gentoo.org/gentoo-dev/msg_c6c89f64fe5734230b672a45a5f3d352.xml
So do it based upon the EAPI of the replacing package.
(In reply to comment #36) > So do it based upon the EAPI of the replacing package. Perhaps we can, but it seems more desirable for the order to be uniform in all cases. If we make if vary depending on the replacing EAPI then we won't have that kind of uniformity until the older EAPIs have completely disappeared.
As opposed to having it based upon some arbitrary Portage version which you're never sure whether or not you'll have, which is how it is now.
In the vast majority of cases, ebuilds work fine with either order. For example, the ones which can be fixed by moving has_version calls from postinst to preinst will work with either order. For ebuilds that want to require the new order, the ebuild can use a blocker to ensure that it is not installed by an older package manager that uses the older order. There don't appear be very many cases like this. Other than this bug, the only other one that's similar at this time is bug 231039.
How many packages in overlays are there that rely upon behaviour the devmanual tells people to use?
To answer, I'll post some of the content of my email that I referenced earlier: The breakage to ebuilds, if they are broken in any way, seems to be non-critical in the vast majority of cases since the new phase order is identical to the phase order that portage has always used when replacing a given ebuild with another ebuild of the same version. I consider the use of has_version calls in pkg_postinst to detect the previous installed version of a package, as suggested in devmanual, to be an illogical or counter-intuitive approach. The counter-intuitive nature of the approach explains why it was only used in a small fraction of the ebuilds in the tree. Ebuilds that used this approach were easily fixed by moving the has_version calls to pkg_preinst and storing the results in environment variables. When moving has_version calls from pkg_postinst to pkg_preinst for all ebuilds in the tree, I noticed that the calls were typically used to trigger einfo or ewarn messages. So, the majority of such breakage simply resulted in failure to generate einfo or ewarn messages in some upgrade or downgrade scenarious.
(In reply to comment #41) > The breakage to ebuilds, if they are broken in any way, seems to be > non-critical in the vast majority of cases That's an awful lot of maybes. Doing it on an EAPI change involves no maybes. > I consider the use of has_version calls in pkg_postinst to detect > the previous installed version of a package, as suggested in > devmanual, to be an illogical or counter-intuitive approach. You might think so, but the fact that it's been around and recommended for so long suggests that you're in a minority there. > The counter-intuitive nature of the approach explains why it was only > used in a small fraction of the ebuilds in the tree. No, the fact that it's not a particularly requirement action explains why it's not a particularly widely used approach. > When moving has_version calls from pkg_postinst to pkg_preinst for > all ebuilds in the tree, I noticed that the calls were typically > used to trigger einfo or ewarn messages. So, the majority of such > breakage simply resulted in failure to generate einfo or ewarn > messages in some upgrade or downgrade scenarious. Except for the ones that instead break people's systems, yes.
You still haven't convinced me that making the order depend on EAPI is worth the non-uniformity that it introduces. I'd still prefer to use blockers to avoid conflicts when necessary.
Uh. You've already introduced non-uniformity. Doing it by EAPI merely makes it controlled non-uniformity.
Well, each approach has benefits and drawbacks. If we can't agree on which is better then I suppose the council will have to vote on it.
(In reply to comment #35) > If if we make the phase order dependent upon EAPI then the logical order of > execution is ambiguous in cases when the package being installed and the one it > replaces have different EAPIs. I've given some examples in the following email: > > http://archives.gentoo.org/gentoo-dev/msg_c6c89f64fe5734230b672a45a5f3d352.xml > Or we could (gasp shock horror) come up with a simple rule to decide which EAPI wins. (In reply to comment #37) > Perhaps we can, but it seems more desirable for the order to be uniform in all > cases. The whole point of EAPI is non-uniformity - ebuilds are processed according to the rules they were written against, not whatever the latest shiny non-backwards-compatible flavour of the month is. (In reply to comment #37) > Perhaps we can, but it seems more desirable for the order to be uniform in all > If we make if vary depending on the replacing EAPI then we won't have > that kind of uniformity until the older EAPIs have completely disappeared. > Compatibility's a bitch. Get over it. (In reply to comment #39) > In the vast majority of cases, ebuilds work fine with either order. In the vast majority of cases, ebuilds work fine whether or not the default src_compile knows about ECONF_SOURCE. In the vast majority of cases, ebuilds work fine whether or not doman knows about language codes in filenames. But we made both of those depend on EAPI, because "in the vast majority of cases" isn't good enough for an allegedly stable format. Why is the phase ordering change, which surely has a bigger impact, any different? > For example, the ones which can be fixed by moving has_version calls from postinst > to preinst will work with either order. Not "fixed", because they were never broken. They were relying on documented, long-established behaviour. > For ebuilds that want to require the new order, the ebuild can use a blocker to > ensure that it is not installed by an older package manager that uses the older > order. That's a joke, right? Why don't we just give up on EAPI altogether if that's the way we're going to do things? > There don't appear be very many cases like this. Except that people have been supporting the change because they think it'll make their lives easier. They may be right, but only if they have a non-retarded way to guarantee that the package manager will give them the behaviour they want. (In reply to comment #41) > The breakage to ebuilds, if they are broken in any way, seems to be > non-critical in the vast majority of cases since the new phase order > is identical to the phase order that portage has always used when > replacing a given ebuild with another ebuild of the same version. So? That's a different situation. It's like saying that it's OK for gcc to make "class" a reserved word in C, because it's always been a reserved word in C++.
I'm still convinced that having a uniform phase order for all EAPIs is the best choice. It's a complex issue, so I don't expect everyone to agree with me.
*** Bug 249220 has been marked as a duplicate of this bug. ***
FYI, this is still happening. I just updated one of my systems (whose system shell has never been anything other than bash) from bash-3.2_p33 to bash-3.2_p39 and the /bin/sh symlink was removed.
