Bug 470880 – sys-apps/portage: FEATURES=keepwork causes emerge to re-use WORKDIR despite different USE

• Product: Portage Development
• Status: CONFIRMED ---

[Description Roman Žilka 2013-05-21 14:06:58 UTC]

I built sys-apps/grep-2.14 with FEATURES="keepwork" and USE="pcre nls". It compiled and installed cleanly. Then I re-run 'emerge grep' with USE="-pcre -nls". Emerge said:

>>> Emerging (1 of 1) sys-apps/grep-2.14
 * grep-2.14.tar.xz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                     [ ok ]
>>> It appears that 'grep-2.14' is already setup; skipping.
>>> Remove '/boot/tmp/portage/sys-apps/grep-2.14/.setuped' to force setup.
>>> WORKDIR is up-to-date, keeping...
>>> It appears that 'grep-2.14' is already prepared; skipping.
>>> Remove '/boot/tmp/portage/sys-apps/grep-2.14/.prepared' to force prepare.
>>> It appears that 'grep-2.14' is already configured; skipping.
>>> Remove '/boot/tmp/portage/sys-apps/grep-2.14/.configured' to force configuration.
>>> It appears that 'grep-2.14' is already compiled; skipping.
>>> Remove '/boot/tmp/portage/sys-apps/grep-2.14/.compiled' to force compilation.
>>> It appears that grep has already been tested; skipping.
>>> Remove '/boot/tmp/portage/sys-apps/grep-2.14/.tested' to force test.

>>> Install grep-2.14 into /boot/tmp/portage/sys-apps/grep-2.14/image/ category sys-apps

... and started filling BUILDDIR/image with the files already present in BUILDDIR/work from the previous compilation. That, however, was with different USE flags. Emerge should've detected that before the second run and started the compilation a-fresh.

Reproducible: Always

[Comment 1 Zac Medico 2013-05-21 17:52:17 UTC]

(In reply to comment #0)
> ... and started filling BUILDDIR/image with the files already present in
> BUILDDIR/work from the previous compilation. That, however, was with
> different USE flags. Emerge should've detected that before the second run
> and started the compilation a-fresh.

You have to clean it yourself when you use FEATURES=keepwork. Rather than change the behavior, I would prefer to simply remove the feature. I haven't removed it because some people may find it useful as-is.

[Comment 2 Roman Žilka 2013-05-21 19:37:04 UTC]

Well, it's still beter as-is than gone. But it might just wipe the BUILDDIR clean, like it's a regular failed build. I'm guessing keepwork users more often than not want to inspect the unpacked sourcecode and possibly re-compile a package at a later time, not just have the compiled sources there to re-merge them into ROOT repeatedly. But then - the current behavior has been there for some time and those who make use of it may expect it.

[Comment 3 Zac Medico 2013-05-21 19:40:58 UTC]

(In reply to comment #2)
> But then - the current behavior
> has been there for some time and those who make use of it may expect it.

Exactly. For people who use FEATURES=keepwork, cleaning WORKDIR can potentially be a *destructive* operation, because they could have things in there that they expect to be protected.

[Comment 4 Zac Medico 2013-05-21 19:54:55 UTC]

Why do you want to enable FEATURES=keepwork anyway? How is it useful to you?

[Comment 5 Roman Žilka 2013-05-21 20:37:11 UTC]

When there's a problematic pkg, I sometimes want to inspect the source tree, see what went wrong, make a patch etc. Then I either re-compile the sources by hand in the workdir, or I re-emerge - either that's necessary or at least more comfortable. I have to call rm -rf $TMPDIR by hand to avoid the current workdir re-use strategy.

The current keepwork behavior is something I never had any use for. When I know I'm going to need to re-merge (re-fill $ROOT with files) a given pkg repeatedly, I make a tbz2. This happens a few times a year.

[Comment 6 Roman Žilka 2013-05-21 20:44:17 UTC]

What I originally called for, however, is not a droppage of the workdir outright before any new emerge. It's not a bad feature that emerge can detect the pre-compiled sources there. It's just that it should then ideally also detect that now I no longer want to install and use pkg[use1], but pkg[use2] which may have a different ./configure call, different resulting binaries etc. But currently emerge does not detect it and I can't picture anyone who expects this, including those who somehow make use of keepwork for store-and-reuse purposes.

[Comment 7 Roman Žilka 2013-05-21 21:01:57 UTC]

Makes me think again if it really is better as-is than gone... Tough call.

[Comment 8 Paul Varner (RETIRED) 2013-05-22 15:40:36 UTC]

I have no strong opinion either way about the keepwork staying or going.  

For the troubleshooting workflow described, I always use the ebuild command directly instead of emerge. So, as long as you can use the ebuild command, I see no real reason to keep the keepwork feature other than potentially convenience.

[Comment 9 Roman Žilka 2013-05-22 16:29:56 UTC]

Well, FWIW, the ebuild command suffers from the same bug. When the pkg in /work has src_compile() finished, so it seems, then a subsequent call of "ebuild pkg.ebuild compile [install] [...]" performs no compilation despite any USE change. I have to keep in mind that I must call "ebuild ... clean compile" when I change USE.

[Comment 10 Zac Medico 2013-05-22 17:36:38 UTC]

When you run the ebuild command, you should get a warning message like this when it re-uses the existing environments:

>>> Existing ${T}/environment for 'grep-2.14' will be
>>> sourced. Run 'clean' to start with a fresh environment.

I think that warning message is good enough. Cleaning ${WORKDIR} can be a destructive operation, so it's left to the user to decide.

For similar reasons, emerge produces the following warning for FEATURES=keepwork:

 * WARNING: The FEATURES variable contains one or more values that                                                                                                                             
 * should be disabled under normal circumstances: keepwork

I think we can improve the documentation for keepwork, to clarify the reason that it prevents cleaning, and to clarify that things like USE changes are not accounted for.