Bug 870577 – dev-lang/python: ctypes.util.find_library is broken on pure clang system (dev-util/pkgcheck-0.10.14: NameError: name 'parser' is not defined (ctypes can't find libraries on musl systems))

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description yemou 2022-09-17 00:54:41 UTC]

find_library() from ctypes does not work on musl systems, this causes pkgcheck to error out with the following error when it attempts to find and use tree-sitter-bash:

pkgcheck scan: error: Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/pkgcheck/pipeline.py", line 186, in _run_checks
    if results := sorted(chain.from_iterable(
  File "/usr/lib/python3.10/site-packages/pkgcheck/runners.py", line 67, in run
    for item in self.source.itermatch(restrict):
  File "/usr/lib/python3.10/site-packages/pkgcheck/sources.py", line 316, in itermatch
    yield _ParsedPkg(data, pkg=pkg)
  File "/usr/lib/python3.10/site-packages/pkgcheck/bash/__init__.py", line 117, in __init__
    self.tree = parser.parse(data)
NameError: name 'parser' is not defined



Reproducible: Always

Steps to Reproduce:
1. Have a musl system
2. run `pkgcheck scan`
Actual Results:  
NameError: name 'parser' is not defined

Expected Results:  
pkgcheck completes normally

[Comment 1 yemou 2022-09-17 00:56:40 UTC]

Created attachment 806764 [details]
emerge --info

[Comment 2 Sam James 2022-09-17 01:10:22 UTC]

The solution will be different but just noting that in bug 860831, we also saw the same error message -- we should really make pkgcheck catch it and give a more informative message.

[Comment 3 Michał Górny 2022-09-17 05:54:31 UTC]

Hmm, I thought find_library() didn't work on libraries that didn't have SONAME but libtree-sitter-bash seems to have one.

[Comment 4 Michał Górny 2022-09-17 06:03:41 UTC]

(In reply to Michał Górny from comment #3)
> Hmm, I thought find_library() didn't work on libraries that didn't have
> SONAME but libtree-sitter-bash seems to have one.

Yeah, I've just installed pkgcheck in my musl chroot and it just works.

[Comment 5 yemou 2022-09-17 08:58:19 UTC]

This is probably just an issue on musl/clang installs that don't have gcc and binutils installed. After installing gcc and binutils, pkgcheck worked just fine.

An alternative solution is to symlink clang -> cc and llvm-objdump -> objdump

[Comment 6 Michał Górny 2022-09-19 07:08:36 UTC]

This is actually not a musl-specific problem but a problem on systems without gcc/binutils, and IMO it's our fault.

ctypes.util.find_library() needs "cc" and "objdump" to work.  Our pure clang stages are missing both.  Not sure about objdump but I'd consider a base system without "cc" and "c++" symlinks broken.

I suppose I could have llvm/clang install some useful symlinks into /usr/lib/llvm/*/bin that would naturally work on non-gcc/binutils systems where there are no matching symlinks in /usr/bin without causing collisions.  However, it probably won't help on migrated systems where we'd probably end up with dangling symlinks in /usr/bin.

[Comment 7 Sam James 2022-09-20 17:03:26 UTC]

commit 3ac93781eb20dd9cd6c90cff860fcbf51a81086a
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:15:47 2022 +0200

    sys-devel/clang: Add symlinks PDEP to 15.x

    Signed-off-by: Michał Górny <[e-mail]>

commit d9b80be87be912f0ed7cf46acff1882a477812b7
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:15:21 2022 +0200

    sys-devel/lld: Add symlinks PDEP to 15.x

    Signed-off-by: Michał Górny <[e-mail]>

commit bdf39f35ac968c1f5656096b9c9667a3101f4f2d
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:14:50 2022 +0200

    sys-devel/llvm: Add symlinks PDEP to 15.x

    Signed-off-by: Michał Górny <[e-mail]>

commit 7d393c02475dc00b57390a451e42446fe85c99d4
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:14:09 2022 +0200

    sys-devel/clang-toolchain-symlinks: Add for LLVM 15.x

    Signed-off-by: Michał Górny <[e-mail]>

commit 3d47ca4f047bc0ded5dc9f624beef421e5e0e678
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:13:37 2022 +0200

    sys-devel/lld-toolchain-symlinks: Add for LLVM 15.x

    Signed-off-by: Michał Górny <[e-mail]>

commit 7d2ba3742b95819ff57590238544a7574c66596b
Author: Michał Górny <[e-mail]>
Date:   Tue Sep 20 13:13:06 2022 +0200

    sys-devel/llvm-toolchain-symlinks: Add for LLVM 15.x

    Signed-off-by: Michał Górny <[e-mail]>

With that, I think this is essentially fixed, even though it _should_ function without cc and friends.

[Comment 8 Larry the Git Cow 2022-09-22 23:46:17 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gcc-config.git/commit/?id=1ff1ed38697ed3b376db5c5157fcec24df67aa91

commit 1ff1ed38697ed3b376db5c5157fcec24df67aa91
Author:     Sam James <[e-mail]>
AuthorDate: 2022-09-22 23:41:28 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2022-09-22 23:45:43 +0000

    gcc-config: create ${CTARGET}-cc symlink
    
    We need this because configure scripts may look for ${CTARGET}-cc first,
    and while this wasn't a problem in the past, LLVM installs fallback
    symlinks in /usr/lib/llvm/${SLOT}/bin for the benefit of llvm-only profiles,
    and configure has started finding these rather than falling back to say,
    ${CTARGET}-gcc like it has been all this time.
    
    Bug: https://bugs.gentoo.org/870577
    Bug: https://bugs.gentoo.org/872416
    Signed-off-by: Sam James <[e-mail]>

 gcc-config | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

[Comment 9 Larry the Git Cow 2022-09-22 23:47:57 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cc6a27ec99c1e08ac51c69ff0ab4c2b8a5578e2e

commit cc6a27ec99c1e08ac51c69ff0ab4c2b8a5578e2e
Author:     Sam James <[e-mail]>
AuthorDate: 2022-09-22 23:46:57 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2022-09-22 23:47:27 +0000

    sys-devel/gcc-config: add 2.6
    
    Creates ${CTARGET}-cc symlink.
    
    We need this because configure scripts may look for ${CTARGET}-cc first,
    and while this wasn't a problem in the past, LLVM installs fallback
    symlinks in /usr/lib/llvm/${SLOT}/bin for the benefit of llvm-only profiles,
    and configure has started finding these rather than falling back to say,
    ${CTARGET}-gcc like it has been all this time.
    
    Bug: https://bugs.gentoo.org/870577
    Bug: https://bugs.gentoo.org/872416
    Signed-off-by: Sam James <[e-mail]>

 sys-devel/gcc-config/Manifest               |  1 +
 sys-devel/gcc-config/gcc-config-2.6.ebuild  | 60 +++++++++++++++++++++++++++++
 sys-devel/gcc-config/gcc-config-9999.ebuild |  6 +--
 3 files changed, 64 insertions(+), 3 deletions(-)