Bug 781854 – sys-apps/portage: sys-devel/gcc installation fails with ansifilter: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory PORTAGE_LOG_FILTER_FILE_CMD="ansifilter --ignore-clear"

• Status: RESOLVED FIXED
• Product: Portage Development

[Description Thomas Deutschmann (RETIRED) 2021-04-09 17:20:00 UTC]

System has =sys-devel/gcc-10.2.0-r5 installed *and* selected.

PORTAGE_LOG_FILTER_FILE_CMD is set to "ansifilter --ignore-clear".

Upgrading to >=sys-devel/gcc-10.3.0 will fail.

It's hard to get any error message. At first the last line I spotted were

> >>> /usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0/gfortran -> x86_64-pc-linux-gnu-gfortran
> >>> Safely unmerging already-installed instance...
>  *
>  * The following package has failed to build, install, or execute postinst:
>  *
>  *  (sys-devel/gcc-10.3.0:10/10::gentoo, ebuild scheduled for merge), Log file:
>  *   '/var/tmp/portage/sys-devel/gcc-10.3.0/temp/build.log'
>  *

emerge always ended with a non-zero exit code.

There will be "/var/db/pkg/sys-devel/-MERGING-gcc-10.3.0".

`emaint --fix merges` wants to re-emerge gcc-10.3.0 which will fail.

When you force a downgrade to gcc-10.2.0-r5, this will now fail the same.

Getting rid of all gcc:10 versions (I still had gcc:9 installed) allowed me to install gcc:10 again. But upgrading to gcc-10.3.0 with active gcc:10 is always failing.

During my testing I once somehow spotted

> ansifilter: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

as last output from emerge after merging.

Unsetting PORTAGE_LOG_FILTER_FILE_CMD allows me to build & merge sys-devel/gcc again.

From my POV there are two issues:

1) When PORTAGE_LOG_FILTER_FILE_CMD fails at the end, vdb will be left in a broken state. Maybe PORTAGE_LOG_FILTER_FILE_CMD shouldn't be fatal given that emerge successfully merged the package before?

2) I wonder if there is anything we can do to make gcc upgrade of same slot working. I.e. ansifilter will not fail when upgrading gcc-10.2 to gcc-10.3 when gcc:9 is the active gcc version.

[Comment 1 Zac Medico 2021-04-09 17:30:24 UTC]

(In reply to Thomas Deutschmann from comment #0)
> During my testing I once somehow spotted
> 
> > ansifilter: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory
> 
> as last output from emerge after merging.

Maybe there's really nothing portage can do about this except recommend that you use a static executable for PORTAGE_LOG_FILTER_FILE_CMD. You can rule out portage as the culprit if the installation succeeds with PORTAGE_LOG_FILTER_FILE_CMD="cat".

[Comment 2 Zac Medico 2021-04-09 17:34:05 UTC]

There's a fallback if exec fails here:

https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/util/_async/BuildLogger.py?h=portage-3.0.18#n55

However, if it fails after exec, then you're in trouble.

[Comment 3 Zac Medico 2021-04-09 17:44:20 UTC]

(In reply to Zac Medico from comment #2)
> There's a fallback if exec fails here:
> 
> https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/util/_async/
> BuildLogger.py?h=portage-3.0.18#n55
> 
> However, if it fails after exec, then you're in trouble.

I suppose portage could wrap PORTAGE_LOG_FILTER_FILE_CMD with some kind of supervisor that behaves like cat after the subprocess has died and stdin shows that there's something to read.

[Comment 4 Zac Medico 2021-04-09 17:50:29 UTC]

Portage can use an instance of ForkProcess to supervise PORTAGE_LOG_FILTER_FILE_CMD.

[Comment 5 Zac Medico 2021-04-09 17:51:30 UTC]

We can make the supervisor optional, in case anyone wants to avoid the overhead.

[Comment 6 Zac Medico 2021-04-09 17:59:05 UTC]

Since ForkProcess is generally reserved for processes that require a fork, I'll probably use a standalone executable for this. It could be a general purpose program that has nothing to do with portage. It's just a program like cat that can supervise and take over if PORTAGE_LOG_FILTER_FILE_CMD dies while stdin has remaining input.

[Comment 7 Zac Medico 2021-04-09 18:02:11 UTC]

We should be able to use a bash script as the supervisor, and make it exec cat if PORTAGE_LOG_FILTER_FILE_CMD fails.

[Comment 8 Zac Medico 2021-04-09 18:11:50 UTC]

Created attachment 698853 [details]
bash script to supervise PORTAGE_LOG_FILTER_FILE_CMD with fallback to cat

For example, save the attached scrpt as /usr/local/bin/filter-command-supervisor, and then use this setting:

PORTAGE_LOG_FILTER_FILE_CMD="filter-command-supervisor ansifilter --ignore-clear"

[Comment 9 Zac Medico 2021-04-09 18:16:00 UTC]

We can inline the supervisor in portage code as this command:

bash -c '"$@"; exec cat'

[Comment 10 Zac Medico 2021-04-09 18:18:49 UTC]

This does the trick:

PORTAGE_LOG_FILTER_FILE_CMD="bash -c \"ansifilter --ignore-clear; exec cat\""

[Comment 11 Zac Medico 2021-04-09 18:21:22 UTC]

(In reply to Zac Medico from comment #10)
> This does the trick:
> 
> PORTAGE_LOG_FILTER_FILE_CMD="bash -c \"ansifilter --ignore-clear; exec cat\""

Maybe we can just update the make.conf man page to suggest this kind of supervisor usage with ansifilter.

[Comment 12 Zac Medico 2021-04-09 18:46:26 UTC]

Comment on attachment 698853 [details]
bash script to supervise PORTAGE_LOG_FILTER_FILE_CMD with fallback to cat

make.conf.5 patch posted for review:

https://archives.gentoo.org/gentoo-portage-dev/message/637b6d8e19a8221bccfa7d82141d300c
https://github.com/gentoo/portage/pull/695

[Comment 13 Toralf Förster 2021-04-10 07:10:18 UTC]

*** Bug 781713 has been marked as a duplicate of this bug. ***

[Comment 14 Toralf Förster 2021-04-10 07:32:42 UTC]

(In reply to Zac Medico from comment #11)
> (In reply to Zac Medico from comment #10)
> > This does the trick:
> > 
> > PORTAGE_LOG_FILTER_FILE_CMD="bash -c \"ansifilter --ignore-clear; exec cat\""
> 
I tried  that but for some reason here at the tinderbox I got empty log files and non-exit code - might investigate it further ....

[Comment 15 Toralf Förster 2021-04-10 07:45:30 UTC]

(In reply to Toralf Förster from comment #14)
> I tried  that but for some reason here at the tinderbox I got empty log
> files and non-exit code - might investigate it further ....

sry - PEBKAC

[Comment 16 Larry the Git Cow 2021-05-24 06:47:13 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1f3bfcb357c3f262730175f4d59c11e1ed4f8dc3

commit 1f3bfcb357c3f262730175f4d59c11e1ed4f8dc3
Author:     Zac Medico <[e-mail]>
AuthorDate: 2021-04-09 18:26:49 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2021-05-24 06:44:57 +0000

    make.conf.5: Suggest PORTAGE_LOG_FILTER_FILE_CMD supervisor for cat fallback (bug 781854)
    
    If PORTAGE_LOG_FILTER_FILE_CMD fails after exec, then output
    will be lost. Therefore, suggest to use bash as a supervisor,
    with fallback to cat.
    
    Bug: https://bugs.gentoo.org/781854
    Signed-off-by: Zac Medico <[e-mail]>

 man/make.conf.5 | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

[Comment 17 Larry the Git Cow 2021-05-24 09:05:13 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cba21902d9a8d1ac07d2ff5b0d932e71fc1fac67

commit cba21902d9a8d1ac07d2ff5b0d932e71fc1fac67
Author:     Zac Medico <[e-mail]>
AuthorDate: 2021-05-24 07:47:11 +0000
Commit:     Zac Medico <[e-mail]>
CommitDate: 2021-05-24 09:05:08 +0000

    sys-apps/portage: Bump to version 3.0.19
    
     #520378: allow emerge --fetchonly to log to emerge-fetch.log
     #698244: portage(5) document user patch / eapply_user
     #781854: Suggest PORTAGE_LOG_FILTER_FILE_CMD cat fallback
     #782724: sort emerge --unmerge order for determinism
     #783957: lazily evaluate cnf_* variables in tests
     #784566: make emerge insensitive to relative order of optional
                      and positional arguments
     #787545: emerge CTRL C may be ignored when running pkg_pretend
     #787563: ebuild-ipc could handle KeyboardInterrupt
     #788967: emerge --jobs= triggers TypeError
    
    Bug: https://bugs.gentoo.org/785484
    Bug: https://bugs.gentoo.org/788967
    Bug: https://bugs.gentoo.org/787563
    Bug: https://bugs.gentoo.org/787545
    Bug: https://bugs.gentoo.org/784566
    Bug: https://bugs.gentoo.org/783957
    Bug: https://bugs.gentoo.org/782724
    Bug: https://bugs.gentoo.org/781854
    Bug: https://bugs.gentoo.org/698244
    Bug: https://bugs.gentoo.org/520378
    Package-Manager: Portage-3.0.18, Repoman-3.0.3
    Signed-off-by: Zac Medico <[e-mail]>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.19.ebuild | 266 +++++++++++++++++++++++++++++++++
 2 files changed, 267 insertions(+)