Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 942760 - sys-apps/portage-3.0.66.1-r1: FEATURES="noclean" deletes source files
Summary: sys-apps/portage-3.0.66.1-r1: FEATURES="noclean" deletes source files
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on: 939444
Blocks:
  Show dependency tree
 
Reported: 2024-11-02 12:48 UTC by Jan Ziak (atomsymbol)
Modified: 2025-01-22 00:30 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Ziak (atomsymbol) 2024-11-02 12:48:25 UTC
Deleting source files contradicts 'man make.conf' which says that 'noclean' means 'Do not delete the source and temporary files after the merge process'.

The deletion means that source code is for example no longer visible in 'gdb /usr/lib64/libc.so.6' with 'disass /s malloc', valgrind or kcachegrind.

----

$ cat /etc/portage/env/noclean 
FEATURES="${FEATURES} noclean"

$ grep libc /etc/portage/package.env/package.env 
sys-libs/glibc  debug noclean

Reproducible: Always
Comment 1 Jan Ziak (atomsymbol) 2024-11-02 12:51:33 UTC
$ grep noclean /var/db/pkg/sys-libs/glibc-2.40-r5/FEATURES
... noclean ...
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-11-02 20:01:50 UTC
(In reply to Jan Ziak (atomsymbol) from comment #0)
> Deleting source files contradicts 'man make.conf' which says that 'noclean'
> means 'Do not delete the source and temporary files after the merge process'.
> 
> The deletion means that source code is for example no longer visible in 'gdb
> /usr/lib64/libc.so.6' with 'disass /s malloc', valgrind or kcachegrind.

But installsources is supposed to be used for that, not noclean. See https://wiki.gentoo.org/wiki/Debugging#Per-package.

> 
> ----
> 
> $ cat /etc/portage/env/noclean 
> FEATURES="${FEATURES} noclean"
> 
> $ grep libc /etc/portage/package.env/package.env 
> sys-libs/glibc  debug noclean
> 
> Reproducible: Always


Anyway, yes, we should look at what's happening here with noclean, just commenting that your use of it sounds unusual.
Comment 3 Jan Ziak (atomsymbol) 2024-11-02 21:25:47 UTC
(In reply to Sam James from comment #2)
> (In reply to Jan Ziak (atomsymbol) from comment #0)
> > Deleting source files contradicts 'man make.conf' which says that 'noclean'
> > means 'Do not delete the source and temporary files after the merge process'.
> > 
> > The deletion means that source code is for example no longer visible in 'gdb
> > /usr/lib64/libc.so.6' with 'disass /s malloc', valgrind or kcachegrind.
> 
> But installsources is supposed to be used for that, not noclean. See
> https://wiki.gentoo.org/wiki/Debugging#Per-package.
> 
> > 
> > ----
> > 
> > $ cat /etc/portage/env/noclean 
> > FEATURES="${FEATURES} noclean"
> > 
> > $ grep libc /etc/portage/package.env/package.env 
> > sys-libs/glibc  debug noclean
> > 
> > Reproducible: Always
> 
> 
> Anyway, yes, we should look at what's happening here with noclean, just
> commenting that your use of it sounds unusual.

'installsources' is unreliable and less general than 'noclean', which is why I am resorting to 'noclean' instead.

Maybe 'installsources' works on your machine, but it doesn't work on my machine. It isn't doing what it is supposed to do. For example, after 'emerge dev-libs/glib':

$ cat /var/db/pkg/dev-libs/glib-2.78.6/FEATURES
... binpkg-dostrip .... installsources ... splitdebug ...

$ grep nostrip /var/db/pkg/dev-libs/glib-2.78.6/FEATURES
(no output)

$ grep -- -g /var/db/pkg/dev-libs/glib-2.78.6/CFLAGS
... -ggdb ...

$ whereis debugedit 
debugedit: /usr/bin/debugedit

$ ls /usr/src/debug
/bin/ls: cannot access '/usr/src/debug': No such file or directory

$ qlist --showdebug dev-libs/glib | grep src
(no output)

The above emerge command didn't print any warning, thus I am assuming that 'installsources' should work, but it doesn't work.

Also: 'man make.conf' doesn't say whether 'installsources' guarantees to install all generated source files.
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-11-02 21:59:02 UTC
Could you file another bug for that please? We should get to the bottom of that too.
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-11-03 05:46:10 UTC
I suspect we broke it with merge-wait cleaning up ${T}. We added skips for keepwork+keeptemp but not noclean. Does it happen with FEATURES="-merge-wait"?
Comment 6 Jan Ziak (atomsymbol) 2024-11-03 14:02:24 UTC
(In reply to Sam James from comment #5)
> I suspect we broke it with merge-wait cleaning up ${T}. We added skips for
> keepwork+keeptemp but not noclean. Does it happen with
> FEATURES="-merge-wait"?

"-merge-wait noclean" resolves the issue, ${WORKDIR} isn't empty.

NOTE: Just out of curiosity, I added misspelled "-merge-wai" to FEATURES on purpose. I expected emerge to abort when starting with an error message saying that there is an unrecognized (unconsumed) identifier in FEATURES, but Portage didn't abort and also didn't print any warning.
Comment 7 Jan Ziak (atomsymbol) 2024-11-03 14:11:37 UTC
Just a note: "Running pre-merge checks for sys-libs/glibc-2.40-r5" deletes /var/tmp/portage/sys-libs/glibc-2.40-r5 before actual emerge of glibc starts.
Comment 8 Zac Medico gentoo-dev 2024-11-04 20:39:21 UTC
(In reply to Jan Ziak (atomsymbol) from comment #6)
> (In reply to Sam James from comment #5)
> > I suspect we broke it with merge-wait cleaning up ${T}. We added skips for
> > keepwork+keeptemp but not noclean. Does it happen with
> > FEATURES="-merge-wait"?
> 
> "-merge-wait noclean" resolves the issue, ${WORKDIR} isn't empty.
> 
> NOTE: Just out of curiosity, I added misspelled "-merge-wai" to FEATURES on
> purpose. I expected emerge to abort when starting with an error message
> saying that there is an unrecognized (unconsumed) identifier in FEATURES,
> but Portage didn't abort and also didn't print any warning.

That sort of warning is currently not implemented for negated FEATURES settings.
Comment 9 Larry the Git Cow gentoo-dev 2024-11-04 20:52:41 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=6648a0d37131534767f15542f1d2efc3d7201f55

commit 6648a0d37131534767f15542f1d2efc3d7201f55
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2024-11-04 20:50:54 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2024-11-04 20:50:54 +0000

    install_qa_check: Fix noclean interaction with merge-wait
    
    Fixes: 8ac72ee300c1 ("install_qa_check: prematurely delete WORKDIR if FEATURES=merge-wait")
    Bug: https://bugs.gentoo.org/942760
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 NEWS                  | 2 ++
 bin/misc-functions.sh | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)
Comment 10 Jan Ziak (atomsymbol) 2024-11-04 21:11:27 UTC
(In reply to Jan Ziak (atomsymbol) from comment #3)
> (In reply to Sam James from comment #2)
> > But installsources is supposed to be used for that, not noclean. See
> > https://wiki.gentoo.org/wiki/Debugging#Per-package.
> > 
> > Anyway, yes, we should look at what's happening here with noclean, just
> > commenting that your use of it sounds unusual.
> 
> 'installsources' is unreliable and less general than 'noclean', which is why
> I am resorting to 'noclean' instead.

It seems I will have to keep resorting to 'noclean' instead of 'installsources splitdebug' in order to get GDB work seamlessly. Another (random) example is app-arch/zstd:

$ gdb /usr/bin/zstd
Reading symbols from /usr/bin/zstd...
Reading symbols from /usr/lib/debug//usr/bin/zstd.debug...
(gdb) disass /s main
Dump of assembler code for function main:
../../programs/zstdcli.c:
warning: 832	../../programs/zstdcli.c: No such file or directory

----

GDB searches paths which look like this:

"/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/build/meson-abi_x86_64.amd64/../../programs/zstdcli.c" = -1 ENOENT (No such file or directory)

"/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/build/meson-abi_x86_64.amd64/../../programs/zstdcli.c" = -1 ENOENT (No such file or directory)

"/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/build/meson-abi_x86_64.amd64/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/build/meson-abi_x86_64.amd64/../../programs/zstdcli.c" -1 ENOENT (No such file or directory)

"/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/build/meson-abi_x86_64.amd64/zstdcli.c" = -1 ENOENT (No such file or directory)

----

but the app-arch/zstd package has installed the following source file:

$ qlist --showdebug zstd | grep zstdcli
/usr/src/debug/app-arch/zstd-1.5.6/zstd-1.5.6/programs/zstdcli.c
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-11-05 02:17:52 UTC
(In reply to Jan Ziak (atomsymbol) from comment #10)
> It seems I will have to keep resorting to 'noclean' instead of
> 'installsources splitdebug' in order to get GDB work seamlessly. Another
> (random) example is app-arch/zstd:

Please file another bug for this so we can investigate it there. We obviously want installsources to work for all cases. Thanks.
Comment 12 hololeap 2024-11-07 03:57:41 UTC
I hit this a few months back but never opened a bug report. Recently, I tried again, follwing instructions in savedconfig.eclass:

> Typically you can create your own configuration files quickly by
> doing:
> 
> 1. Build the package with FEATURES=noclean USE=savedconfig.
> 
> 2. Go into the build dir and edit the relevant configuration system
> (e.g. `make menuconfig` or `nano config-header.h`).  You can look
> at the files in /etc/portage/savedconfig/ to see what files get
> loaded/restored.
> 
> 3. Copy the modified configuration files out of the workdir and to
> the paths in /etc/portage/savedconfig/.
> 
> 4. Emerge the package with just USE=savedconfig to get the custom
> build.

After installing with FEATURES=noclean, the work directory is empty:

$ FEATURES="noclean" emerge -1 gentoo-kernel
...
$ ls -l /var/tmp/portage/sys-kernel/gentoo-kernel-6.6.58-r1/work/
total 0
Comment 13 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-11-07 06:31:30 UTC
(In reply to hololeap from comment #12)
> I hit this a few months back but never opened a bug report. Recently, I
> tried again, follwing instructions in savedconfig.eclass:
> 

But did you try with portage-9999 which has the fix Zac committed just a few days ago?
Comment 14 Larry the Git Cow gentoo-dev 2025-01-22 00:30:21 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=03f41049a0fe0632eabd8cddaaca898e45943201

commit 03f41049a0fe0632eabd8cddaaca898e45943201
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2025-01-22 00:29:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2025-01-22 00:30:02 +0000

    sys-apps/portage: add 3.0.67
    
    Closes: https://bugs.gentoo.org/703520
    Closes: https://bugs.gentoo.org/707980
    Closes: https://bugs.gentoo.org/904702
    Closes: https://bugs.gentoo.org/906044
    Closes: https://bugs.gentoo.org/923530
    Closes: https://bugs.gentoo.org/938164
    Closes: https://bugs.gentoo.org/939299
    Closes: https://bugs.gentoo.org/940120
    Closes: https://bugs.gentoo.org/942512
    Closes: https://bugs.gentoo.org/942760
    Closes: https://bugs.gentoo.org/945382
    Closes: https://bugs.gentoo.org/945861
    Closes: https://bugs.gentoo.org/946326
    Closes: https://bugs.gentoo.org/947822
    Closes: https://bugs.gentoo.org/948067
    Closes: https://bugs.gentoo.org/939444
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.67.ebuild | 231 +++++++++++++++++++++++++++++++++
 2 files changed, 232 insertions(+)