Summary: | =sys-apps/portage-2.2.18: emerge pre-inst and post-inst step produce numerous "$: bad substitution" lines | ||
---|---|---|---|
Product: | Portage Development | Reporter: | zlg (RETIRED) <zlg> |
Component: | Core - Interface (emerge) | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | aidecoe, Ikonta, M4rkusXXL, mike, mva, newchief, reavertm |
Priority: | Normal | Keywords: | InVCS |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
URL: | http://www.cmake.org/pipermail/cmake/2008-January/019290.html | ||
See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=547520 https://bugs.gentoo.org/show_bug.cgi?id=664100 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 484436 | ||
Attachments: |
emerge --debug output
root's .bashrc make.conf |
Description
zlg (RETIRED)
2015-03-10 09:15:45 UTC
Created attachment 398576 [details] emerge --debug output I've attached `emerge --debug` output for others to inspect. The package is from my personal overlay (a simple Python utility). Package doesn't matter since it happens every time, but its source can be viewed at https://github.com/sporkbox/sporkbox-overlay/app-misc/dupekill/dupekill-9999.ebuild . Created attachment 398578 [details]
root's .bashrc
Attached root's .bashrc
Created attachment 398580 [details]
make.conf
(In reply to Daniel Campbell from comment #1) > Created attachment 398576 [details] > emerge --debug output > > I've attached `emerge --debug` output for others to inspect. The package is > from my personal overlay (a simple Python utility). Package doesn't matter > since it happens every time, but its source can be viewed at > https://github.com/sporkbox/sporkbox-overlay/app-misc/dupekill/dupekill-9999. > ebuild . Sorry, that's https://raw.githubusercontent.com/sporkbox/sporkbox-overlay/master/app-misc/dupekill/dupekill-9999.ebuild I am having the same problems. I just bisected portage and found the bad commit: $ git bisect log git bisect start # bad: [f44548b24df12f324b5560355e4e6947a9be5f3b] setup.py: Set version for new release git bisect bad f44548b24df12f324b5560355e4e6947a9be5f3b # good: [fc0567d794a4bf122baa3884f15c497d9d863734] setup.py: version bump, make sync modules selectable git bisect good fc0567d794a4bf122baa3884f15c497d9d863734 # bad: [c1489985f64443c4fba0b9661eee60f61e470d37] repoman: skip vcs calls for manifest modes (bug 540882) git bisect bad c1489985f64443c4fba0b9661eee60f61e470d37 # bad: [f1c1b8a77eebf7713b32e5f9945690f60f4f46de] Generate soname dependency metadata (bug 282639) git bisect bad f1c1b8a77eebf7713b32e5f9945690f60f4f46de # good: [06a4e7a09211c4a598a72a97c64daff13cd502ff] setup.py: Revert the select_plugins() changes git bisect good 06a4e7a09211c4a598a72a97c64daff13cd502ff # good: [7fab3aadb4cdca35ce0d81525af1256c745308ff] Add another check for broken /dev/s (bug 538980) git bisect good 7fab3aadb4cdca35ce0d81525af1256c745308ff # first bad commit: [f1c1b8a77eebf7713b32e5f9945690f60f4f46de] Generate soname dependency metadata (bug 282639) (In reply to Karol Herbst from comment #6) > # first bad commit: [f1c1b8a77eebf7713b32e5f9945690f60f4f46de] Generate > soname dependency metadata (bug 282639) The bash changes in this commit are trivial, so I don't understand why they would trigger the "$: bad substitution" messages: http://gitweb.gentoo.org/proj/portage.git/commit/?id=f1c1b8a77eebf7713b32e5f9945690f60f4f46de I cannot reproduce the problem with bash-4.2_p53. Maybe it's a bug in bash-4.3_p33-r2? Comment on attachment 398578 [details]
root's .bashrc
portage doesn't load the user's ~/.bashrc files, so this file shouldn't matter
the trace doesn't seem to be that enlightening ... the errors look like they come up *after* misc-functions.sh finishes fully executing i would try downgrading to some other bash versions in the tree as Zac suggested. I tried downgrading to 4.3_p33-r1, 4.3_p33, and 4.2_p53, with restarting terminal and verifying that `bash --version` checks out, env-update, etc. and re-merging any package. The same "bad substitution" message appeared on each. I'm pretty much lost. I have nothing in p.use for bash so I'm using whatever USE flags it ships with.. same goes for portage. That bisect seems wrong. Are you sure you don't have any malformed files in /etc/portage? (In reply to Michał Górny from comment #11) > That bisect seems wrong. Are you sure you don't have any malformed files in > /etc/portage? I've pored over each file I have in /etc/portage and can't find anything out of place or malformed. I didn't know about repo.postsync.d and it's /usr/bin/q invocation, but given that I've not touched it I doubt it's a problem. It looks like it checks for PORTAGE_QUIET and if it's set, quietly forks. I can attach a file that has the content of all my /etc/portage files if you're interested in looking over it. this also happens for me after whiping out /etc/portage Just to add critical mass to the bug: I confirm that bug too. Although, I catched it only on my laptop, but do not on any of hundred servers. Although, /etc/portage is almost identical between all of them. I'm trying to fix the title so that is turns up in searches for "bad substition", since I had a hard time finding this bug to mark bug 547154 as a duplicate. *** Bug 547154 has been marked as a duplicate of this bug. *** I can observe strange situation: - bug occurs on my desktop systems - bug does not occur on my server systems What differs both: - I have Polish locales on desktop sysmtems - I have systemd on desktop systems - I have openrc on server systems - I have nvidia GPU on desktop systems (yup, it was the cause of one of mysterious bugs in the past!) - I have ATI graphics on server systems I also checked that Python 2.7 is currently set as default python interpreter on both. I'm keeping all my Gentoo boxes up to date, so they share similar versions of everything. Well, I have an nvidia card, using the nouveau driver, en_US.utf8 and no Errors (In reply to Paul Osmialowski from comment #17) > What differs both: > - I have Polish locales on desktop sysmtems > - I have systemd on desktop systems > - I have openrc on server systems > - I have nvidia GPU on desktop systems (yup, it was the cause of one of > mysterious bugs in the past!) > - I have ATI graphics on server systems > > I also checked that Python 2.7 is currently set as default python > interpreter on both. I use openrc, radeon video driver (probably should not matter), Python 3.3 as system default and ru_RU.UTF8 locale and see this issue. I forgot about I guess the most important difference: - I have these server systems for long long years and they still use baselayout1 (make.conf is in /etc) - On desktop systems which I change more frequently, I have baselayout2 (make.conf is in /etc/portage) See https://forums.gentoo.org/viewtopic-t-1014842.html tl;dr: Broken libs have "$$ORIGIN" in runpaths which issues a warning/error since the commit (added varexpand). There's a patch in the following branch: https://github.com/zmedico/portage/tree/bug_542796 I've posted it for review here: https://archives.gentoo.org/gentoo-portage-dev/message/b0ed5c8fd00f88ba4813777d557ac812 I've placed your patch as /etc/portage/patches/sys-apps/portage/2.2.18-bad_substitution.patch and now emerging packages ends like this: >>> Installing (1 of 1) media-video/gxine-0.5.907-r1::gentoo /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution * Updating desktop mime database ... * Updating shared mime info database ... * Updating icons cache ... [ ok ] * Updating desktop mime database ... * Updating shared mime info database ... * Updating icons cache ... [ ok ] /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution >>> Auto-cleaning packages... (In reply to Paul Osmialowski from comment #23) > I've placed your patch as > /etc/portage/patches/sys-apps/portage/2.2.18-bad_substitution.patch and now > emerging packages ends like this: > > >>> Installing (1 of 1) media-video/gxine-0.5.907-r1::gentoo > /var/db/pkg/dev-db/tora-3.0.0_pre20140929/NEEDED.ELF.2: $: bad substitution Please file a new bug for dev-db/tora-3.0.0_pre20140929, and attach that file. In order silence the error message by manually edit the file, and then update the parent directory timestamps to invalidate portage's cache: touch /var/db/pkg/dev-db/tora-3.0.0_pre20140929 \ /var/db/pkg/dev-db \ /var/db/pkg Added bug 547520 This is in the master branch: https://gitweb.gentoo.org/proj/portage.git/commit/?id=a47407e864d9a5a43d6f05ef06b46e34ff0e0c24 Hi, I also have the problem and have found something. In /var/db/pkg/dev-db/tora-9999/NEEDED.ELF.2 is: X86_64;/usr/bin/tora;;$ORIGIN/:$$ORIGIN/;libQtGui.so.4,libQtSql.so.4,libQtNetwork.so.4,libQtCore.so.4,libqscintilla2.so.11,libferrisloki.s o.3,libdl.so.2,libpq.so.5,libstdc++.so.6,libm.so.6,libgcc_s.so.1,libc.so.6 X86_64;/usr/lib64/libtrotl.so;libtrotl.so;$ORIGIN/instantclient/:$$ORIGIN/instantclient/;libclntsh.so.11.1,libstdc++.so.6,libgcc_s.so.1,li bc.so.6 X86_64;/usr/lib64/tora-3.0.0alpha/libporacle.so;libporacle.so;$ORIGIN/:$$ORIGIN/;libclntsh.so.11.1,libQtGui.so.4,libQtCore.so.4,libtrotl.s o,libstdc++.so.6,libgcc_s.so.1,libc.so.6 Look at the "$$ORIGIN". Portage since 2.2.18 looks at this and shows "$: bad substitution" error/warning, nevermind what it merges. After unmerge tora-9999 strange waring disapeared. Merging tora-9999 again or tora-3.0.0_pre20140929 - the same situation Looks like portage has a problem with parsing "$$ORIGIN". In my opinion there should be "$ORIGIN" (In reply to Jacek from comment #27) > After unmerge tora-9999 strange waring disapeared. > Merging tora-9999 again or tora-3.0.0_pre20140929 - the same situation > Looks like portage has a problem with parsing "$$ORIGIN". In my opinion > there should be "$ORIGIN" Something about tora's build system produces the invalid $$ORIGIN thing. See bug 547520. yes, you right. toras code sets that paths. SET(CMAKE_INSTALL_RPATH "$ORIGIN/:$$ORIGIN/") SET(CMAKE_INSTALL_RPATH "$ORIGIN/instantclient/:$$ORIGIN/instantclient/") tnx j These seem to be workarounds. Some buildsystems mess(ed) up the $ORIGIN... http://www.cmake.org/pipermail/cmake/2008-January/019290.html (In reply to Markus from comment #30) > These seem to be workarounds. > Some buildsystems mess(ed) up the $ORIGIN... > http://www.cmake.org/pipermail/cmake/2008-January/019290.html Apparently it's a workaround for a buggy old version of cmake. There are at least two possible ways to fix these packages: 1) Use sed to remove :$$ORIGIN from the source files in src_prepare. 2) Use patchelf to remove :$$ORIGIN from DT_RUNPATH of the binaries in src_install. Update of tora to version dev-db/tora-3.0.0_pre20140929-r1 solves the problem. from what i can tell, there's nothing here related to the shell, so dropping base-system from the list (In reply to Zac Medico from comment #31) > (In reply to Markus from comment #30) > > These seem to be workarounds. > > Some buildsystems mess(ed) up the $ORIGIN... > > http://www.cmake.org/pipermail/cmake/2008-January/019290.html > > Apparently it's a workaround for a buggy old version of cmake. There are at > least two possible ways to fix these packages: > > 1) Use sed to remove :$$ORIGIN from the source files in src_prepare. > 2) Use patchelf to remove :$$ORIGIN from DT_RUNPATH of the binaries in > src_install. Although I think 1 is preferred, in my case that was adobe-air-sdk-bin, which has no "source" :-/ After applying the patch (which is now in the master branch), the culprit was dev-util/adobe-air-runtime-2.6: /var/db/pkg/dev-util/adobe-air-runtime-2.6/NEEDED.ELF.2: $: bad substitution Editing the file with vim (:%s/$$ORIGIN/$ORIGIN/g) and touching the directories fixed the issue for me. yeah, same for me (In reply to Daniel Campbell from comment #35) > After applying the patch (which is now in the master branch), the culprit > was dev-util/adobe-air-runtime-2.6: > > /var/db/pkg/dev-util/adobe-air-runtime-2.6/NEEDED.ELF.2: $: bad substitution > > Editing the file with vim (:%s/$$ORIGIN/$ORIGIN/g) and touching the > directories fixed the issue for me. Actually, I've edited it with vim too, but I think we need to think about patching policy and fix them all then ;) (In reply to Vadim A. Misbakh-Soloviov (mva) from comment #37) > (In reply to Daniel Campbell from comment #35) > > :snip: > > Actually, I've edited it with vim too, but I think we need to think about > patching policy and fix them all then ;) I could cook up a quick script tonight, but I don't think it'd be the preferred way to fix the DB. Fixing it at the ebuild level is probably the most elegant right now. (In reply to Vadim A. Misbakh-Soloviov (mva) from comment #34) > Although I think 1 is preferred, in my case that was adobe-air-sdk-bin, > which has no "source" :-/ It should be possible to use a combination of 'patchelf --print-rpath' and 'patchelf --set-rpath' to fix the binaries. Alternatively, maybe 'patchelf --shrink-rpath' will do the trick. (In reply to Zac Medico from comment #39) > It should be possible to use a combination of 'patchelf --print-rpath' and > 'patchelf --set-rpath' to fix the binaries. Alternatively, maybe 'patchelf > --shrink-rpath' will do the trick. I suggest something like this: find "${ED}" -type f -print0 | while read -r -d '' x; do rpath=$(patchelf --print-rpath "${x}" 2>/dev/null) [[ ${rpath} =~ (.*:)?\$\$ORIGIN[^:]*(:.*)? ]] || continue patchelf --set-rpath "${BASH_REMATCH[1]}${BASH_REMATCH[2]#:}" "${x}" \ || die done (In reply to Zac Medico from comment #40) > (In reply to Zac Medico from comment #39) > > It should be possible to use a combination of 'patchelf --print-rpath' and > > 'patchelf --set-rpath' to fix the binaries. Alternatively, maybe 'patchelf > > --shrink-rpath' will do the trick. > > I suggest something like this: > > find "${ED}" -type f -print0 | while read -r -d '' x; do > rpath=$(patchelf --print-rpath "${x}" 2>/dev/null) > [[ ${rpath} =~ (.*:)?\$\$ORIGIN[^:]*(:.*)? ]] || continue > patchelf --set-rpath "${BASH_REMATCH[1]}${BASH_REMATCH[2]#:}" "${x}" \ > || die > done while read -r -d '' x; do ... done < <(find "${ED}" -type f -print0) Otherwise the loop will be run in subprocess instead of the main process. Possibly stupid question: During last update I've seen many similiar warnings. It's necessary to open a bug for each issued package or it will be better to report it upstream? (In reply to Sergey S. Starikoff from comment #42) > Possibly stupid question: > During last update I've seen many similiar warnings. > It's necessary to open a bug for each issued package or it will be better to > report it upstream? It's probably best to file a gentoo bug, so that the problem can be corrected ASAP. Who knows if upstream still wants to support broken old versions of cmake? (In reply to Zac Medico from comment #43) > Who knows if upstream still wants to support broken old > versions of cmake? Not only cmake. Issue seems to be more common. Working on ebuild for =app-editors/gobby-0.5.0 (see bug #545394) I've seen the same messages. Although it uses cmake only for docs (and I build it with default -doc). Fix from bug #547520 failes because no issued pattern found. Continue testing showed the same issue for =gnome-doc-utils/gnome-doc-utils-0.20.10-r1 (which seems not to use cmake at all). I expect not hundreds, but at least thousands of such packages only in the gentoo overlay. So, opening a bug for each issued package don't looks to be a good idea. Probably we should try to provide some generic check at eclass[es] level. (In reply to Sergey S. Starikoff from comment #44) > I expect not hundreds, but at least thousands of such packages only in the > gentoo overlay. > So, opening a bug for each issued package don't looks to be a good idea. > Probably we should try to provide some generic check at eclass[es] level. I guess we'll have to put the check in portage, since it calls scanelf on all of the files after they are installed into ${D}. We can make it check for invalid substitutions at that time, and trigger a QA notice. Released in portage-2.2.19 |