Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 203891 - [Future EAPI] Allow a more finegrained way to not strip files than on a per package basis
Summary: [Future EAPI] Allow a more finegrained way to not strip files than on a per p...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: PMS/EAPI
URL:
Whiteboard: in-eapi-7
Keywords: PATCH
Depends on:
Blocks: future-eapi ebuild-prep 587290 587292 587294 587296 587298 587300 587316 651462
  Show dependency tree
 
Reported: 2007-12-31 15:05 UTC by Alexis Ballier
Modified: 2018-04-30 22:46 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
First draft of a specification. (dostrip.diff,5.16 KB, patch)
2017-10-24 17:49 UTC, Ulrich Müller
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexis Ballier gentoo-dev 2007-12-31 15:05:53 UTC
As of now, as far as I can see, the only way to not strip a file is to use RESTRICT=strip in an ebuild.
However, portage supports STRIP_MASK to explicitly disallow certain files from being stripped. This is quite handy in some ways, especially with some ocaml packages.

[more or less copy/paste from bug #203887 for justification]
ocamlc -custom uses a nasty trick to have runnable executables withtout needing
an interpreter, some packages use it (see ocamlc manpage for details).
It embeds an interpreter and the bytecode in one file. Stripping them removes the bytecode...

See [1] for a probably better reference.
[1] http://caml.inria.fr/pub/ml-archives/caml-list/2002/08/492887df149fb58b1c21ffa89e6fee66.en.html
[/end of copy paste]

I tend to consider that restricting stripping on the whole package only for this is an overkill, especially with bug #203886, where I'd like to provide native code executables or (interpreter + bytecode) depending on an useflag.
With portage just doing something like: 
use ocamlopt || export STRIP_MASK="*/bin/*"
does what is needed and nothing more (assuming the regexp is correct).

However it doesn't seem well documented, and I'm not sure if it's allowed to do that or not (esp. since paludis still strips the file if I use this ebuild, thus the package becomes unuseable).
If I have a voice, I vote for this to be allowed.
Comment 1 Ciaran McCreesh 2007-12-31 15:08:04 UTC
Do we classify STRIP_MASK as a Portage internal config option, and thus beyond the scope of PMS and ebuilds? I'm inclined to say yes...
Comment 2 Alexis Ballier gentoo-dev 2007-12-31 15:24:10 UTC
(In reply to comment #1)
> Do we classify STRIP_MASK as a Portage internal config option, and thus beyond
> the scope of PMS and ebuilds? I'm inclined to say yes...

Imho that's what it currently is. But then you can see this bug as a feature request to allow this in a more "sane" manner if this one isn't (even if I can't see any problem with it).
grepping the tree shows that glibc ebuilds have been doing this for a while, so imho something has to be done in one way or the other. For the reasons I mentionned, I think such a finegrained way to do it should be allowed, and because portage seems to have supported STRIP_MASK for a while, it could just be allowed without backward compatibilities issues.

[thinking again it seems I misunderstood your comment, but well...]
Comment 3 Ciaran McCreesh 2007-12-31 15:27:27 UTC
Should probably be an EAPI 2 request then.
Comment 4 Petteri Räty (RETIRED) gentoo-dev 2007-12-31 16:22:35 UTC
But do ebuilds need to know about STRIP_MASK?
Comment 5 SpanKY gentoo-dev 2007-12-31 19:48:35 UTC
of course they do ... it's been around since EAPI-0 and their are things relying on it already
Comment 6 Ciaran McCreesh 2007-12-31 19:54:33 UTC
*Should* ebuilds be relying upon it? It looks suspiciously like it's a user config variable like INSTALL_MASK, and thus not something ebuilds should be touching or using.
Comment 7 SpanKY gentoo-dev 2007-12-31 20:02:19 UTC
yes, ebuilds need to set STRIP_MASK so it can execute prepallstrip properly
Comment 8 Ciaran McCreesh 2007-12-31 20:10:40 UTC
Last time I looked, prep* was considered a package manager internal too and not suitable for ebuild use.
Comment 9 SpanKY gentoo-dev 2007-12-31 20:27:13 UTC
your refusal to acknowledge that some prep functions are exported to ebuilds and expected to be used is irrelevant

they are exported, ebuilds are using them, and are part of EAPI-0+
Comment 10 Ciaran McCreesh 2007-12-31 20:32:23 UTC
Are they exported, or do they just happen to be around because of how Portage sets the env up? The fact that people are using them does not imply that people *should* be using them... I mean, do you consider what's being done in bug #167029 to be legitimate?
Comment 11 SpanKY gentoo-dev 2007-12-31 20:54:48 UTC
notice i said "some".  i'm not claiming every prep function deserves should be used by ebuilds, but certainly some are.  what needs to be done by the ruby.eclass is there because Bug 179800 is not implemented yet.
Comment 12 Alexis Ballier gentoo-dev 2008-01-02 16:51:51 UTC
As there does not seem to be any non political reason to not use it, I think I'll use it for ocaml packages. That will greatly help me improving the quality of those packages.
Comment 13 Ciaran McCreesh 2008-01-03 23:13:33 UTC
It's not a political reason, it's a technical one -- the prep* functions are considered to be Portage internals by the Portage people, and are thus subject to arbitrary removal or change.
Comment 14 Alexis Ballier gentoo-dev 2008-03-19 12:42:52 UTC
Considering the summary has just been changed to postpone it to a hypothetical future eapi, let me make a summary for ocaml stuff:

- lots of ebuilds are now using it to not strip files that shouldnt be
- this is needed for bytecode, bytecode is needed for arches that do not have (good) native code support
- portage handles this very well.


I still think postponing this to a future eapi is a political reason; this has nothing to do with prep* functions and isn't more a user defined variable than cflags where afaik its sane to append stuff.
Comment 15 Alexis Ballier gentoo-dev 2013-08-15 22:53:00 UTC
I would like to ask the council to vote on retroactively adding STRIP_MASK to EAPI-0+ or close this bug and let non-portage PMs suffer from it.
Comment 16 Ulrich Müller gentoo-dev 2013-08-16 05:20:52 UTC
Where can I find documentation how STRIP_MASK works? It's neither in Portage man pages nor in the devmanual.
Comment 17 Zac Medico gentoo-dev 2013-08-16 06:26:08 UTC
Here's the code located in /usr/lib/portage/bin/ebuild-helpers/prepstrip:

	# The noglob funk is to support STRIP_MASK="/*/booga" and to keep
	# the for loop from expanding the globs.
	# The eval echo is to support STRIP_MASK="/*/{booga,bar}" sex.
	set -o noglob
	strip_this=true
	for m in $(eval echo ${STRIP_MASK}) ; do
		[[ /${x#${ED}} == ${m} ]] && strip_this=false && break
	done
	set +o noglob
Comment 18 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-16 08:22:24 UTC
Isn't OCaml what should be fixed instead? I doubt it's anywhere close to good design when applications requires to be non-stripped to work properly.
Comment 19 Ulrich Müller gentoo-dev 2013-08-16 09:15:12 UTC
STRIP_MASK was introduced at 20060113:
<http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;f=bin/prepstrip;h=2ef74a4c8dd1ef5dab77264c7a9f2d85a3d5fb19>
This was included with portage-2.1 which went stable (on last arch) at 20060611. So IF we add it to the PMS, it would certainly qualify for EAPI 0.

On the other hand, it's an obscure and undocumented feature that is used by only 22 packages in the tree.

@aballier: I suggest that you post it to the gentoo-dev mailing list, and depending on the outcome of the discussion there the issue can be brought forward to the council.
Comment 20 Ulrich Müller gentoo-dev 2013-08-16 09:26:10 UTC
Here's the list of packages that set STRIP_MASK:
   app-emulation/qemu
   app-misc/geneweb
   app-misc/ledit
   app-misc/wyrd
   app-text/active-dvi
   app-text/htmlc
   dev-lang/ghc
   dev-lang/go
   dev-lisp/clisp
   dev-ml/findlib
   dev-ml/oasis
   dev-ml/ocamlnet
   dev-scheme/schoca
   dev-util/coccinelle
   dev-util/omake
   net-misc/unison
   net-p2p/mldonkey
   sys-boot/grub
   sys-cluster/ceph
   sys-devel/dev86
   sys-freebsd/freebsd-lib
   sys-libs/glibc
Comment 21 Alexis Ballier gentoo-dev 2013-08-16 11:32:21 UTC
(In reply to Michał Górny from comment #18)
> Isn't OCaml what should be fixed instead? I doubt it's anywhere close to
> good design when applications requires to be non-stripped to work properly.

It isn't good design, some apps are avoiding to use that "broken" feature and it is becoming less needed these days, but that's the classical first reaction of an outsider. If you think you can fix it, then please do. Others have tried.

Anyway, this isn't all about ocaml. .o files (think *crt*.o) have no need for being stripped too.


(In reply to Ulrich Müller from comment #19)
> @aballier: I suggest that you post it to the gentoo-dev mailing list, and
> depending on the outcome of the discussion there the issue can be brought
> forward to the council.

If you feel like doing it, then please do. I'm sorry but fixing PMS 6 years later is at the bottom of my priority list :/
Comment 22 Ciaran McCreesh 2013-08-16 11:38:40 UTC
Considering the prep* functions aren't stable or EAPI controlled, it's a bit of a stretch to consider this "fixing PMS"...
Comment 23 Alexis Ballier gentoo-dev 2013-08-16 11:45:32 UTC
(In reply to Ciaran McCreesh from comment #22)
> Considering the prep* functions aren't stable or EAPI controlled, it's a bit
> of a stretch to consider this "fixing PMS"...

This isn't about prep* functions.
Comment 24 Ciaran McCreesh 2013-08-16 11:49:03 UTC
(In reply to Alexis Ballier from comment #23)
> This isn't about prep* functions.

Comment 7 thinks it is.
Comment 25 Alexis Ballier gentoo-dev 2013-08-16 11:56:21 UTC
(In reply to Ciaran McCreesh from comment #24)
> (In reply to Alexis Ballier from comment #23)
> > This isn't about prep* functions.
> 
> Comment 7 thinks it is.

Read the code. glibc does it that way, others do not as far as I've seen.
You should also do some research as of why glibc does it that way.
Comment 26 Ulrich Müller gentoo-dev 2013-08-16 13:47:01 UTC
(In reply to Ciaran McCreesh from comment #24)
> (In reply to Alexis Ballier from comment #23)
> > This isn't about prep* functions.
> 
> Comment 7 thinks it is.

It is about the package manager stripping binaries. The internal Portage function doing this is called prepallstrip, but this doesn't mean that it needs to be called from (or even exposed to) ebuilds. 

Doesn't Paludis also strip binaries after src_install and before merging?


(In reply to Alexis Ballier from comment #21)
> If you feel like doing it, then please do. I'm sorry but fixing PMS 6 years
> later is at the bottom of my priority list :/

Well, then RESO WONT. Obscure and undocumented Portage-only feature.
Comment 27 Ciaran McCreesh 2013-08-16 13:51:13 UTC
(In reply to Ulrich Müller from comment #26)
> Doesn't Paludis also strip binaries after src_install and before merging?

It does, just not using a utility named prep*.

> (In reply to Alexis Ballier from comment #21)
> > If you feel like doing it, then please do. I'm sorry but fixing PMS 6 years
> > later is at the bottom of my priority list :/
> 
> Well, then RESO WONT. Obscure and undocumented Portage-only feature.

Then any uses of it in the tree that are required for correct operation need to be replaced with RESTRICT=strip.
Comment 28 Ulrich Müller gentoo-dev 2013-08-16 13:54:26 UTC
(In reply to Ciaran McCreesh from comment #27)
> Then any uses of it in the tree that are required for correct operation need
> to be replaced with RESTRICT=strip.

Yes, only those uses that are essential for correct operation. Cases where STRIP_MASK is just "nice to have" can stay, IMHO.
Comment 29 Alexis Ballier gentoo-dev 2013-08-16 14:00:27 UTC
(In reply to Ulrich Müller from comment #28)
> Then any uses of it in the tree that are required for correct operation need
> to be replaced with RESTRICT=strip.

start by fixing glibc then; it's either this, an entirely unstripped glibc or bug #46186
Comment 30 Ulrich Müller gentoo-dev 2013-08-16 14:09:10 UTC
(In reply to Alexis Ballier from comment #29)
> start by fixing glibc then; it's either this, an entirely unstripped glibc
> or bug #46186

So to summarise, you want the feature but it's so low on your priority list that you're not willing to champion it in the -dev ML or present it to the council. But others shall do the work and clean up the breakage.
Comment 31 Alexis Ballier gentoo-dev 2013-08-16 14:10:38 UTC
(In reply to Ulrich Müller from comment #28)
> (In reply to Ciaran McCreesh from comment #27)
> > Then any uses of it in the tree that are required for correct operation need
> > to be replaced with RESTRICT=strip.
> 
> Yes, only those uses that are essential for correct operation. Cases where
> STRIP_MASK is just "nice to have" can stay, IMHO.

you know what this implies, right?
either a serious regression by restricting stripping on an entire package or changing those STRIP_MASK uses to RESTRICT=strip + STRIP_MASK + prepallstrip so that there is no regeression on the portage side and other PMs also have working packages, yet totally unstripped.
Comment 32 Ulrich Müller gentoo-dev 2013-08-16 14:26:04 UTC
(In reply to Alexis Ballier from comment #31)
> you know what this implies, right?
> either a serious regression by restricting stripping on an entire package or
> changing those STRIP_MASK uses to RESTRICT=strip + STRIP_MASK + prepallstrip
> so that there is no regeression on the portage side and other PMs also have
> working packages, yet totally unstripped.

As I said, discuss it in the -dev ML, and/or propose it for the next council meeting's agenda (preferably with a PMS patch). And feel free to reopen this bug then.

Otherwise, also the members of the PMS team are volunteers. I can't speak for others, but I'm not willing to do the grunt work for you if the feature is "at the bottom of your priority list".


(In reply to Alexis Ballier from comment #29)
> (In reply to Ulrich Müller from comment #28)
> > Then any uses of it in the tree that are required for correct operation need
> > to be replaced with RESTRICT=strip.

BTW, I haven't written the above sentence that you're attributing to me.
Comment 33 Alexis Ballier gentoo-dev 2013-08-16 14:41:41 UTC
(In reply to Ulrich Müller from comment #30)
> (In reply to Alexis Ballier from comment #29)
> > start by fixing glibc then; it's either this, an entirely unstripped glibc
> > or bug #46186
> 
> So to summarise, you want the feature but it's so low on your priority list
> that you're not willing to champion it in the -dev ML or present it to the
> council. But others shall do the work and clean up the breakage.

Sort of, but I would phrase it differently. See comment #15 for the council part.

I do not _want_ the feature in the usual sense, I claim it is _needed_ and is a PMS deficiency not to document it. I am not involved with PMS. If PMS does not reflect reality and is broken, I consider it fair to ignore it. If PMS people want to impose not well thought rules by forbidding stuff that has always been working fine (for reasons I, moreover, consider doubtful) then I indeed expect those people to provide solutions for the cases where it's needed. And correct solutions, not the RESTRICT=strip bloat.

(In reply to Ulrich Müller from comment #32)
> Otherwise, also the members of the PMS team are volunteers. I can't speak
> for others, but I'm not willing to do the grunt work for you if the feature
> is "at the bottom of your priority list".

Fair enough, the case is closed :) This bug has been ignored for 6 years, bug #46186 was closed in 2004: I understand that as nobody cares and the current status quo is good enough. No need to spend my time on it :)

Note: We have a problem of vocabulary here, what you call a "feature" I consider a "PMS deficiency".
Comment 34 Ciaran McCreesh 2013-08-16 14:49:03 UTC
The phrase "reflect reality" isn't a helpful one to use. By Council decree, "reality" isn't what's allowed in the tree; what PMS specifies is what's allowed in the tree.
Comment 35 Alexis Ballier gentoo-dev 2013-08-16 14:59:43 UTC
(In reply to Ciaran McCreesh from comment #34)
> The phrase "reflect reality" isn't a helpful one to use. By Council decree,
> "reality" isn't what's allowed in the tree; what PMS specifies is what's
> allowed in the tree.

yes, and by this very definition, you have been acting as the council itself in this bug.
Comment 36 Ciaran McCreesh 2013-08-16 15:03:19 UTC
(In reply to Alexis Ballier from comment #35)
> (In reply to Ciaran McCreesh from comment #34)
> > The phrase "reflect reality" isn't a helpful one to use. By Council decree,
> > "reality" isn't what's allowed in the tree; what PMS specifies is what's
> > allowed in the tree.
> 
> yes, and by this very definition, you have been acting as the council itself
> in this bug.

I don't follow. The Council approved the most recent edition of PMS, not me.
Comment 37 Andreas K. Hüttel archtester gentoo-dev 2013-08-16 15:13:22 UTC
OK guys. We're not on the mailing list here. We got your points. All of them. Now may I respectfully ask you to shut up and/or carry your discussion elsewhere.

@ulm: it still is a bug. probably one in reality, which slowly gets distorted here.
Comment 38 Ulrich Müller gentoo-dev 2013-08-16 17:06:01 UTC
I've nothing against including this in PMS. However, STRIP_MASK was proposed previously and the council hasn't accepted it for EAPI 2:
http://archives.gentoo.org/gentoo-dev/msg_6dcca100f593cce5d25870f357bbe318.xml

So please follow regular procedure for EAPI 6, starting with discussion in gentoo-dev.
Comment 39 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-16 17:44:17 UTC
This should probably be done in some kind of 'dostrip' (in consistency with 'docompress').
Comment 40 Ulrich Müller gentoo-dev 2013-08-16 20:53:40 UTC
(In reply to Michał Górny from comment #39)
> This should probably be done in some kind of 'dostrip' (in consistency with
> 'docompress').

docompress accepts one or more plain paths without globbing, whereas STRIP_MASK accepts a single filename pattern. So I wonder if we could have a dostrip with an interface similar to docompress?
Comment 41 Mike Gilbert gentoo-dev 2017-02-06 01:32:28 UTC
(In reply to Ulrich Müller from comment #40)
> (In reply to Michał Górny from comment #39)
> > This should probably be done in some kind of 'dostrip' (in consistency with
> > 'docompress').
> 
> docompress accepts one or more plain paths without globbing, whereas
> STRIP_MASK accepts a single filename pattern.

That is not correct; STRIP_MASK accepts multiple patterns separated by whitespace.

For simple cases, a plain path will suffice (like usr/lib/grub). There may be other cases where more fine-grained pattern matching is necessary.

Are we still looking for a gentoo-dev discussion to move this bug forward? If so, I am happy to start it.
Comment 42 Andreas K. Hüttel archtester gentoo-dev 2017-10-22 16:34:22 UTC
(In reply to Mike Gilbert from comment #41)
> 
> Are we still looking for a gentoo-dev discussion to move this bug forward?
> If so, I am happy to start it.

Please do. 

I don't care how the semantics precisely look, but I do need a way to exempt specific files from stripping, that is guaranteed to work independent of package manager.

I'm fine with adding it to a future EAPI as well, just that *unless* we get this feature I won't be able to remove the remaining calls to prepstrip (see bug 259422).
Comment 43 Andreas K. Hüttel archtester gentoo-dev 2017-10-22 16:36:00 UTC
PS. For obvious reasons this is a feature that 99% of the packages should not and will not need. The rest (boot loaders, glibc, ...) fulfills some critical functionality though.
Comment 44 Ulrich Müller gentoo-dev 2017-10-23 14:55:05 UTC
(In reply to Andreas K. Hüttel from comment #42)
> I don't care how the semantics precisely look, but I do need a way to exempt
> specific files from stripping, that is guaranteed to work independent of
> package manager.

If the answer to the question asked in comment #40 is positive, we can use docompress as a blueprint for a dostrip command in the next EAPI.

> I'm fine with adding it to a future EAPI as well, just that *unless* we get
> this feature I won't be able to remove the remaining calls to prepstrip (see
> bug 259422).

I disagree. The very first council-approved PMS version already said (in 2009) that prep* are internal functions of the PM and that ebuilds may neither use nor rely upon them. So any calls to them are a QA violation and should be removed.
Comment 45 Andreas K. Hüttel archtester gentoo-dev 2017-10-23 19:51:34 UTC
> 
> > I'm fine with adding it to a future EAPI as well, just that *unless* we get
> > this feature I won't be able to remove the remaining calls to prepstrip (see
> > bug 259422).
> 
> I disagree. The very first council-approved PMS version already said (in
> 2009) that prep* are internal functions of the PM and that ebuilds may
> neither use nor rely upon them. So any calls to them are a QA violation and
> should be removed.

Well I give you three options:
* a QA violation that should be removed
* a massive code duplication between portage and e.g. the sys-libs/glibc ebuild (re-implementing the entire stripping procedure)
* or a non-functional gdb in all multithreaded applications

There's a reason why noone fixed this QA violation for 8 years... it's not just "toolchain can't be bothered" here.
Comment 46 Andreas K. Hüttel archtester gentoo-dev 2017-10-23 20:13:28 UTC
(In reply to Ulrich Müller from comment #40)
> (In reply to Michał Górny from comment #39)
> > This should probably be done in some kind of 'dostrip' (in consistency with
> > 'docompress').
> 
> docompress accepts one or more plain paths without globbing, whereas
> STRIP_MASK accepts a single filename pattern. So I wonder if we could have a
> dostrip with an interface similar to docompress?

I need to check in detail still, but on a quick glance the semantics of docompress look useful. I think I can work with that.
Comment 47 Ulrich Müller gentoo-dev 2017-10-24 06:49:44 UTC
(In reply to Andreas K. Hüttel from comment #45)
> Well I give you three options:
> * a massive code duplication between portage and e.g. the sys-libs/glibc
> ebuild (re-implementing the entire stripping procedure)

Why not? This could be in a eclass.

> * or a non-functional gdb in all multithreaded applications

That is what we have now, because prep* is unspecified and we cannot rely doing anything useful.
Comment 48 Alexis Ballier gentoo-dev 2017-10-24 07:45:12 UTC
(In reply to Ulrich Müller from comment #47)
> (In reply to Andreas K. Hüttel from comment #45)
> > Well I give you three options:
> > * a massive code duplication between portage and e.g. the sys-libs/glibc
> > ebuild (re-implementing the entire stripping procedure)
> 
> Why not? This could be in a eclass.

This is not accounting that it bites its own tail by requiring to look at "$FEATURES" to respect FEATURES=nostrip, splitdebug, installsource, etc. I like the idea of moving every bash code out of PM(S) to eclasses, but we're far from it on this side.

> > * or a non-functional gdb in all multithreaded applications
> 
> That is what we have now, because prep* is unspecified and we cannot rely
> doing anything useful.

That is what we have now on the very small minority (I don't have numbers but I would say that's <1%) of users not using portage, yes. Defining STRIP_MASK and prep* would have made this minority happy since almost 10 years now.

Really, if you look at the implications, either the QA violation is OK or those functions need to be added to current EAPIs: EAPI7 will probably come in 2018; I would expect glibc ebuilds with those changes and this EAPI to be stabilized on all arches in 2019 or 2020 if we follow current procedures of not bumping @system packages EAPIs too quickly in order to preserve the upgrade path and give proper testing to our libc. That's a more than 2 years plan.
Comment 49 Ulrich Müller gentoo-dev 2017-10-24 09:17:24 UTC
So, apparently Portage is considered to define the standard, and PMS isn't needed here? Then I suppose this bug can be closed again.
Comment 50 Ulrich Müller gentoo-dev 2017-10-24 17:49:16 UTC
Created attachment 499966 [details, diff]
First draft of a specification.

Attached is a first draft of a specification. For the final version this should presumably be merged with the previous subsection (controllable compression).
Comment 51 Alexis Ballier gentoo-dev 2017-10-24 19:38:52 UTC
(In reply to Ulrich Müller from comment #50)
> Created attachment 499966 [details, diff] [details, diff]
> First draft of a specification.
> 
> Attached is a first draft of a specification. For the final version this
> should presumably be merged with the previous subsection (controllable
> compression).

Thanks.


Some nits:

+stripping of debug symbols, the package manager may compress a subset of the files under the \t{ED}

s/compress/strip/



The current API seems very good to me. I think I saw toolchain using globs (e.g. *crt*.o), hope that works for them.
Comment 52 Mike Gilbert gentoo-dev 2017-10-24 21:11:06 UTC
(In reply to Alexis Ballier from comment #51)
> The current API seems very good to me. I think I saw toolchain using globs
> (e.g. *crt*.o), hope that works for them.

I think that should be fine, with some small ebuild adjustments. If the dostrip call is done after the files are copied to ${D}, the globs can be evaluated by the ebuild rather than the package manager.

For example:

src_install() {
    emake DESTDIR="${D}" install
    cd "${ED}"
    dostrip -x foo/bar/*baz*.o
}
Comment 53 Andreas K. Hüttel archtester gentoo-dev 2017-10-25 18:31:21 UTC
(In reply to Ulrich Müller from comment #50)
> Created attachment 499966 [details, diff] [details, diff]
> First draft of a specification.
> 
> Attached is a first draft of a specification. For the final version this
> should presumably be merged with the previous subsection (controllable
> compression).

Looks good to me (all bugs already found above :).
Comment 54 Ulrich Müller gentoo-dev 2017-10-26 08:04:49 UTC
(In reply to Alexis Ballier from comment #51)
> +stripping of debug symbols, the package manager may compress a subset of
> the files under the \t{ED}
> 
> s/compress/strip/

Corrected.

> The current API seems very good to me. I think I saw toolchain using globs
> (e.g. *crt*.o), hope that works for them.

I'd rather not have the command itself handle wildcards, as none of the other do* commands (especially docompress) does that.
Comment 55 Alexis Ballier gentoo-dev 2017-10-26 09:12:26 UTC
(In reply to Mike Gilbert from comment #52)
> (In reply to Alexis Ballier from comment #51)
> > The current API seems very good to me. I think I saw toolchain using globs
> > (e.g. *crt*.o), hope that works for them.
> 
> I think that should be fine, with some small ebuild adjustments. If the
> dostrip call is done after the files are copied to ${D}, the globs can be
> evaluated by the ebuild rather than the package manager.
> 
> For example:
> 
> src_install() {
>     emake DESTDIR="${D}" install
>     cd "${ED}"
>     dostrip -x foo/bar/*baz*.o
> }


Unless I'm missing something, this will not work or at least is a bit confusing:
We use "/" in the include list to strip everything, this wildcard would expand to e.g. "foo/bar/baz.o", without the leading "/".


(In reply to Ulrich Müller from comment #54)
> > The current API seems very good to me. I think I saw toolchain using globs
> > (e.g. *crt*.o), hope that works for them.
> 
> I'd rather not have the command itself handle wildcards, as none of the
> other do* commands (especially docompress) does that.

I fully agree with that. If the above is a real issue and not a simple misunderstanding on my side, I would rather see dostrip behave exactly the same as docompress for consistency and avoiding any confusion. Moreover, it is easy to write an eutils function expanding a wildcard to match files in $D.
Comment 56 Mike Gilbert gentoo-dev 2017-10-26 14:41:08 UTC
(In reply to Alexis Ballier from comment #55)
> Unless I'm missing something, this will not work or at least is a bit
> confusing:
> We use "/" in the include list to strip everything, this wildcard would
> expand to e.g. "foo/bar/baz.o", without the leading "/".

"Whether an item is to be excluded is determined as follows: For each item in the exclusion list, pretend it has the value of the \t{ED} variable prepended, then: ..."

So, this becomes "${ED}foo/bar/baz.o".

${ED} is guaranteed to always end with a slash. ${ED} generally expands to something like "/tmp/portage/cat/pkg/image/".

So, we end up excluding "/tmp/portage/cat/pkg/image/foo/bar/baz.o".

I don't see why any leading slash would be necessary in the exclusion list.
Comment 57 Ulrich Müller gentoo-dev 2017-10-26 16:04:57 UTC
(In reply to Mike Gilbert from comment #56)
> "Whether an item is to be excluded is determined as follows: For each item
> in the exclusion list, pretend it has the value of the \t{ED} variable
> prepended, then: ..."
> 
> So, this becomes "${ED}foo/bar/baz.o".

Right, the intention is that do{compress,strip} work for arguments beginning with or without a slash.

> ${ED} is guaranteed to always end with a slash. ${ED} generally expands to
> something like "/tmp/portage/cat/pkg/image/".

According to our roadmap, that will no longer be the case in EAPI 7 (see bug 465772). So we will need some adjustment of wording there.

The portage implementation adds any missing / in the docompress call already:
https://gitweb.gentoo.org/proj/portage.git/tree/bin/phase-helpers.sh?id=7d2c4fb609454b76d30c42fc7a0bb720decc39a3#n120
(see f=${f%/} in line 127).
Comment 58 Ulrich Müller gentoo-dev 2017-10-26 16:06:24 UTC
(In reply to Ulrich Müller from comment #57)
> (see f=${f%/} in line 127).

Sorry, I meant line 128 of course:
[[ ${f:0:1} = / ]] || f="/${f}"
Comment 59 Larry the Git Cow gentoo-dev 2018-03-28 05:19:51 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=9aaa652c86b21b925bc56fafd57b561b86ace0f8

commit 9aaa652c86b21b925bc56fafd57b561b86ace0f8
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2018-03-26 19:12:51 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-03-28 05:09:15 +0000

    Add dostrip for EAPI 7
    
    This patch includes the essential parts of the dostrip implementation
    from portage-mgorny. All of the prepstrip code has moved to
    bin/estrip, without any changes except the addition of argument
    parsing for estrip --ignore, --queue, and --deque modes which are
    equivalent to the corresponding ecompressdir modes.
    
    Due to overlap, also ban the non-standard prepstrip and prepallstrip
    helpers in EAPI 7, with a die message suggesting to use 'dostrip'
    instead. Also ignore the non-standard STRIP_MASK variable for EAPI 7.
    
    Bug: https://bugs.gentoo.org/203891

 bin/eapi.sh                     |  4 +++
 bin/ebuild-helpers/prepall      |  2 +-
 bin/ebuild-helpers/prepallstrip |  4 +++
 bin/ebuild-helpers/prepstrip    |  4 +++
 bin/ebuild.sh                   |  2 +-
 bin/estrip                      | 74 +++++++++++++++++++++++++++++++++++++++--
 bin/misc-functions.sh           |  6 ++++
 bin/phase-helpers.sh            | 29 ++++++++++++++++
 bin/save-ebuild-env.sh          |  4 +--
 9 files changed, 122 insertions(+), 7 deletions(-)}
Comment 60 Larry the Git Cow gentoo-dev 2018-04-30 22:14:43 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/pms.git/commit/?id=373ec493c322530b72f7bafb7db846a522704184

commit 373ec493c322530b72f7bafb7db846a522704184
Author:     Ulrich Müller <ulm@gentoo.org>
AuthorDate: 2017-10-29 09:30:43 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2018-03-31 15:30:43 +0000

    EAPI 7 has controllable stripping and dostrip.
    
    Bug: https://bugs.gentoo.org/203891

 dependencies.tex     |  4 +++-
 eapi-differences.tex |  7 ++++++
 pkg-mgr-commands.tex | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 72 insertions(+), 1 deletion(-)}