Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 721886 - PMS uses 'ar' to unpack .deb and .a files, should use $AR or ${CHOST}-ar (was: app-backup/synology-cloud-station-backup calls ar directly)
Summary: PMS uses 'ar' to unpack .deb and .a files, should use $AR or ${CHOST}-ar (was...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: PMS/EAPI
URL:
Whiteboard:
Keywords:
: 725806 (view as bug list)
Depends on:
Blocks: tc-directly
  Show dependency tree
 
Reported: 2020-05-09 16:46 UTC by Agostino Sarubbo
Modified: 2020-06-06 09:21 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,7.01 KB, text/plain)
2020-05-09 16:47 UTC, Agostino Sarubbo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Agostino Sarubbo gentoo-dev 2020-05-09 16:46:59 UTC
This is an auto-filled bug because app-backup/synology-cloud-station-backup calls ar directly.
The issue was originally discovered on amd64, but it may be reproducible on other arches as well.
If you think that a different summary clarifies the issue better, feel free to change it.
Attached build log and emerge --info.

NOTE:
If you think it doesn't make sense fix these type of issues, I'd like to point out that won't be possible use a different AR implementation (like llvm-ar) by setting the AR variable. So this issue has been reproduced by setting the AR variable to x86_64-pc-linux-gnu-ar and by removing the /usr/bin/ar binary.
Comment 1 Agostino Sarubbo gentoo-dev 2020-05-09 16:47:08 UTC
Created attachment 637076 [details]
build.log

build log and emerge --info
Comment 2 Pacho Ramos gentoo-dev 2020-05-09 17:16:27 UTC
I think it's a false positive.

That call to ar happens when portage tries to unpack the deb file with /usr/lib/portage/python3.7/phase-helpers.sh
Comment 3 Sergei Trofimovich gentoo-dev 2020-05-24 09:10:32 UTC
/usr/lib/portage/python3.7/phase-helpers.sh code is:

"""
  a)
     ar x "${srcdir}${x}" || die "$myfail"
     ;;
  deb)
     if [[ $(ar --version 2>/dev/null) != "GNU ar"* 
         ...
     else
         ar x "$srcdir$x" || die "$myfail"
     fi
     ;;
"""

I think we actually want 'AR' variable to be used here. Or at least fall back to ${CHOST}-ar.
Comment 4 Mike Gilbert gentoo-dev 2020-05-24 15:09:37 UTC
I was going to suggest we drop support for .deb files from the unpack() function and force ebuilds to use unpack_deb from unpacker.eclass.

However, it seems we are mandated to support .ar and .deb by PMS.

https://projects.gentoo.org/pms/7/pms.html#x1-13500012.3.15

```
ar archives (*.a). Ebuilds must ensure that GNU binutils is installed.
deb packages (*.deb). Ebuilds must ensure that the deb2targz program is installed on those platforms where the GNU binutils ar program is not available and the installed ar program is incompatible with GNU archives. Otherwise, ebuilds must ensure that GNU binutils is installed.
```
Comment 5 Mike Gilbert gentoo-dev 2020-05-24 15:11:06 UTC
In unpacker.eclass, we do this:

    local AR=${AR-ar}
    ${AR} x "${deb}" || die
Comment 6 Zac Medico gentoo-dev 2020-05-24 19:01:26 UTC
In PMS, it doesn't say anything about the AR variable or ${CHOST}-ar, so we'll have to extend the specification if we want to support this.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-05-24 19:20:27 UTC
(In reply to Zac Medico from comment #6)
> In PMS, it doesn't say anything about the AR variable or ${CHOST}-ar, so
> we'll have to extend the specification if we want to support this.

But this simply makes no sense.  If we are unpacking generic .ar archives, there's no reason to use enforce a version of 'ar' specific to some toolchain.
Comment 8 Zac Medico gentoo-dev 2020-05-24 19:27:47 UTC
(In reply to Pacho Ramos from comment #2)
> I think it's a false positive.
> 
> That call to ar happens when portage tries to unpack the deb file with
> /usr/lib/portage/python3.7/phase-helpers.sh

Maybe whitelist the src_unpack phase for this ar check.
Comment 9 Ulrich Müller gentoo-dev 2020-05-24 20:16:30 UTC
(In reply to Michał Górny from comment #7)
> (In reply to Zac Medico from comment #6)
> > In PMS, it doesn't say anything about the AR variable or ${CHOST}-ar, so
> > we'll have to extend the specification if we want to support this.
> 
> But this simply makes no sense.  If we are unpacking generic .ar archives,
> there's no reason to use enforce a version of 'ar' specific to some
> toolchain.

I agree, no need to complicate the spec. (Also CHOST looks wrong for an unpack tool running on the build system.)
Comment 10 Arfrever Frehtes Taifersar Arahesis 2020-05-25 22:51:18 UTC
With planned Portage wrappers (bug #684894), configuration will be able to contain:
[ar]
wrapper = ar
target-env-vars = AR
warning = true
warning-phases-excluded = src_unpack
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-05-26 06:45:10 UTC
ar shouldn't be wrapped at all.  I've seen at least one package that fails src_test() because of the silly wrapping attempts.  It is supposed to work on .deb files.
Comment 12 Arfrever Frehtes Taifersar Arahesis 2020-05-26 06:53:14 UTC
Or maybe 'warning = false'?

Does llvm-ar work for these .deb files?

If `ar` wrapper is silent and just redirects to real `ar`, `${CHOST}-ar` or `llvm-ar`, how would src_test() fail?

For unpacking, there may be no difference if `ar` or `${CHOST}-ar` is called, but maybe for some operations with real .o input files using `${CHOST}-ar` is needed.
Comment 13 Arfrever Frehtes Taifersar Arahesis 2020-06-01 21:31:40 UTC
*** Bug 725806 has been marked as a duplicate of this bug. ***
Comment 14 Ulrich Müller gentoo-dev 2020-06-01 21:36:59 UTC
Is there anything here that needs an actual update of PMS?
Otherwise, please reassign.
Comment 15 Arfrever Frehtes Taifersar Arahesis 2020-06-01 22:51:45 UTC
> https://projects.gentoo.org/pms/7/pms.html#x1-13500012.3.15
> 
> ```
> ar archives (*.a). Ebuilds must ensure that GNU binutils is installed.
> deb packages (*.deb). Ebuilds must ensure that the deb2targz program is
> installed on those platforms where the GNU binutils ar program is not
> available and the installed ar program is incompatible with GNU archives.
> Otherwise, ebuilds must ensure that GNU binutils is installed.
> ```

"the installed ar program" is somehow vague.

Does this phrase allow package manager, in situation even when AR variable or another variable (which are unspecified in PMS anyway) is not set, to call a program with functionality equivalent to GNU binutils ar program but different name (e.g. llvm-ar)?

If yes, then it can be construed that package manager is free to implement algorithm which uses first found program from e.g. this list:
- ${AR} (if referring to GNU ar or llvm-ar)
- GNU ar (ar, ${CBUILD}-ar etc.)
- llvm-ar (llvm-ar, ${CBUILD}-ar etc.)
- deb2targz
Comment 16 Ulrich Müller gentoo-dev 2020-06-02 06:38:28 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #15)
> > https://projects.gentoo.org/pms/7/pms.html#x1-13500012.3.15
> > 
> > ```
> > ar archives (*.a). Ebuilds must ensure that GNU binutils is installed.
> > deb packages (*.deb). Ebuilds must ensure that the deb2targz program is
> > installed on those platforms where the GNU binutils ar program is not
> > available and the installed ar program is incompatible with GNU archives.
> > Otherwise, ebuilds must ensure that GNU binutils is installed.
> > ```
> 
> "the installed ar program" is somehow vague.
> 
> Does this phrase allow package manager, in situation even when AR variable
> or another variable (which are unspecified in PMS anyway) is not set, to
> call a program with functionality equivalent to GNU binutils ar program but
> different name (e.g. llvm-ar)?

The wording says that either deb2targz or GNU binutils must be installed, so IMHO a reasonable conclusion is that these are the programs that must be called.

> If yes, then it can be construed that package manager is free to implement
> algorithm which uses first found program from e.g. this list:
> - ${AR} (if referring to GNU ar or llvm-ar)

No (or only if it refers to GNU binutils ar).

> - GNU ar (ar, ${CBUILD}-ar etc.)

Yes.

> - llvm-ar (llvm-ar, ${CBUILD}-ar etc.)

No.

> - deb2targz

Yes.

Note: When looking at the history of the text (see bug 296716), it becomes clear that ar from GNU binutils is meant to be called, except for Prefix where deb2targz can be used as an alternative.
Comment 17 Arfrever Frehtes Taifersar Arahesis 2020-06-02 07:17:45 UTC
(In reply to Ulrich Müller from comment #16)
> (In reply to Arfrever Frehtes Taifersar Arahesis from comment #15)
> > > https://projects.gentoo.org/pms/7/pms.html#x1-13500012.3.15
> > > 
> > > ```
> > > ar archives (*.a). Ebuilds must ensure that GNU binutils is installed.
> > > deb packages (*.deb). Ebuilds must ensure that the deb2targz program is
> > > installed on those platforms where the GNU binutils ar program is not
> > > available and the installed ar program is incompatible with GNU archives.
> > > Otherwise, ebuilds must ensure that GNU binutils is installed.
> > > ```
> > 
> > "the installed ar program" is somehow vague.
> > 
> > Does this phrase allow package manager, in situation even when AR variable
> > or another variable (which are unspecified in PMS anyway) is not set, to
> > call a program with functionality equivalent to GNU binutils ar program but
> > different name (e.g. llvm-ar)?
> 
> The wording says that either deb2targz or GNU binutils must be installed, so
> IMHO a reasonable conclusion is that these are the programs that must be
> called.

Sentence "Ebuilds must ensure that the deb2targz program is
installed on those platforms where the GNU binutils ar program is not
available and the installed ar program is incompatible with GNU archives."
allows for possibility that GNU binutils ar program is not available, but
an installed ar program is compatible with GNU archives.

I claim that llvm-ar program is such an alternative implementation of ar program, from LLVM project, instead of GNU binutils project.

It is also worth mentioning that LLVM build system has option LLVM_INSTALL_BINUTILS_SYMLINKS (disabled by default), which would enable installation of ar symlink to llvm-ar:
https://github.com/llvm/llvm-project/blob/llvmorg-10.0.0/llvm/CMakeLists.txt#L182-L183


(In reply to Arfrever Frehtes Taifersar Arahesis from comment #15)

> - llvm-ar (llvm-ar, ${CBUILD}-ar etc.)

I meant ${CBUILD}-llvm-ar (but it is not implemented and maybe does not matter anyway: bug #617776).
Comment 18 Arfrever Frehtes Taifersar Arahesis 2020-06-02 07:39:28 UTC
I think that these sentences in PMS should be reworded to be more practical and allow package manager to do anything as long as the result of unpacking is the same.
Comment 19 Ulrich Müller gentoo-dev 2020-06-02 07:42:45 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #17)
> Sentence "Ebuilds must ensure that the deb2targz program is
> installed on those platforms where the GNU binutils ar program is not
> available and the installed ar program is incompatible with GNU archives."
"Otherwise, ebuilds must ensure that GNU binutils is installed."

Let's write it in programmatic form, so it will become clearer:

IF (NOT "GNU binutils ar is available")
  AND (NOT "installed ar is compatible with GNU")
THEN
    "Ensure that deb2targz is installed"
ELSE
    "Ensure that GNU binutils is installed"
ENDIF

> allows for possibility that GNU binutils ar program is not available, but
> an installed ar program is compatible with GNU archives.

This would evaluate to "(not false) and (not true)", so the else clause would be taken. Which doesn't make much sense, so maybe we should drop the second part of the condition:

IF (NOT "GNU binutils ar is available")
THEN
    "Ensure that deb2targz is installed"
ELSE
    "Ensure that GNU binutils is installed"
ENDIF
Comment 20 Arfrever Frehtes Taifersar Arahesis 2020-06-02 08:03:52 UTC
Some compromise would be:
"""
・ar archives (*.a). Ebuilds must ensure that program capable of unpacking ar archives and supporting unpacking-related features of GNU binutils ar program is installed.
・deb packages (*.deb). Ebuilds must ensure that program capable of unpacking ar archives and supporting unpacking-related features of GNU binutils ar program is installed. On platforms where no such program is available, ebuilds must ensure that the deb2targz program is installed.
"""
Comment 21 Arfrever Frehtes Taifersar Arahesis 2020-06-02 08:12:42 UTC
If app-arch/lbzip2[symlink], app-arch/pbzip2[symlink] or app-arch/pigz[symlink] is installed, then their symlinks would be used instead of tools from bzip2 / GNU gzip.

So any sentence in PMS about something being merely installed does not guarantee that this something is actually used.
Comment 22 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-06-02 08:18:49 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #20)
> Some compromise would be:
> """
> ・ar archives (*.a). Ebuilds must ensure that program capable of unpacking ar
> archives and supporting unpacking-related features of GNU binutils ar
> program is installed.
> ・deb packages (*.deb). Ebuilds must ensure that program capable of unpacking
> ar archives and supporting unpacking-related features of GNU binutils ar
> program is installed. On platforms where no such program is available,
> ebuilds must ensure that the deb2targz program is installed.
> """

Does that mean that ebuilds are now supposed to magically work when p7zip is installed?
Comment 23 Ulrich Müller gentoo-dev 2020-06-02 08:45:13 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #20)

I disagree. For almost all unpackers there are alternative implementations (for example, you could use app-editors/emacs for unpacking tar archives). Nevertheless, we require specific implementations (e.g., GNU tar for tar archives), and I don't see the point why we should handle this differently for *.a and *.deb. Especially, when sys-devel/binutils is in the system set for most systems.

> ・ar archives (*.a). Ebuilds must ensure that program capable of unpacking ar
> archives and supporting unpacking-related features of GNU binutils ar
> program is installed.

Nope. Allowing arbitrary programs in ebuilds' dependencies simply doesn't work, because the package manager won't know how to call them. We need a specific implementation there. (Or two in case of *.deb, but this already leads to complications as we see in this bug.)
Comment 24 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-06-02 08:48:06 UTC
That said, it might be a good idea to reword the section into naming specific executables to be used and what degree of compatibility we expect.