Each procedure of profile update shows me some errors in my system (files, not belonging to any package). For example, on switching to default/linux/amd64/17.1 (stable): orphan dirs/files (not owned by any package) that will be kept in /lib64/: libgcc_s.so.1 and some others. I suggest to make this check regularly, on each system update. Similiarly with reminding to clean obsolete dependencies (emerge --depclean). For example — as option to emerge utility (something like «--search-orphans»). Or as an util in app-portage/gentoolkit.
Like app-portage/portage-utils' `qfile -o <path>`?
I guess OP wants to do a full filesystem search for any file that's not tracked by PM or something. I guess that needs a whitelist, or blacklist of folders not to consider (/home/, /etc, etc.). qfile could possibly be changed to make it recursively scan a directory for orphans (implementing the whitelist approach).
(In reply to Fabian Groffen from comment #2) > I guess OP wants to do a full filesystem search for any file that's not > tracked by PM or something. I guess that needs a whitelist, or blacklist of > folders not to consider (/home/, /etc, etc.). qfile could possibly be > changed to make it recursively scan a directory for orphans (implementing > the whitelist approach). Yes, it is. Just I find black/white lists unnecessary. We just shoul exclude directories, where user (including root) can create files. I.e. /home/, /var and /etc. And properly handle dynamic directories like /dev/ and /run.
(In reply to Jeroen Roovers from comment #1) > Like app-portage/portage-utils' `qfile -o <path>`? Thank you. To check some exact directory it is a solution. But not for complete filesystem check. P.S. Unlike 17.1 profile migration script qfile -o incorrectly handles symbolic links: $ qfile -o /usr/bin/* … /usr/bin/xsubpp … $ equery b /usr/bin/xsubpp * Searching for /usr/bin/xsubpp ... dev-lang/perl-5.28.2-r1 (/usr/bin/xsubpp-3.390.0-perl-5.28.2) $ file /usr/bin/xsubpp /usr/bin/xsubpp: symbolic link to xsubpp-3.390.0-perl-5.28.2 Am I to open a separate bug for this issue?
(In reply to Sergey S. Starikoff from comment #4) > P.S. Unlike 17.1 profile migration script qfile -o incorrectly handles > symbolic links: > $ qfile -o /usr/bin/* > … > /usr/bin/xsubpp > … > $ equery b /usr/bin/xsubpp > * Searching for /usr/bin/xsubpp ... > dev-lang/perl-5.28.2-r1 (/usr/bin/xsubpp-3.390.0-perl-5.28.2) > $ file /usr/bin/xsubpp > /usr/bin/xsubpp: symbolic link to xsubpp-3.390.0-perl-5.28.2 > > Am I to open a separate bug for this issue? qfile intentionally does not resolve symlinks Is this symbolic link present in CONTENTS for any package (/var/db/pkg)?
(In reply to Mike Gilbert from comment #5) > Is this symbolic link present in CONTENTS for any package (/var/db/pkg)? Not present: $ grep xsubpp /var/db/pkg/dev-lang/perl-5.28.2-r1/CONTENTS obj /usr/bin/xsubpp-3.390.0-perl-5.28.2 42e9b054d01ebd11edad5c436db10bc2 1559044837 obj /usr/lib64/perl5/5.28.2/ExtUtils/xsubpp 8a3672b8fa0d614fd698e16204a318a4 1559044836 obj /usr/share/man/man1/xsubpp-3.390.0-perl-5.28.2.1.bz2 82c07a7fd257608fc40777c6357dee02 1559044865 Symlink may point anywhere in writable area. So it's resolution is the thing should be done. Of course with detecting broken symbolic links.
(In reply to Mike Gilbert from comment #5) > (In reply to Sergey S. Starikoff from comment #4) > > P.S. Unlike 17.1 profile migration script qfile -o incorrectly handles > > symbolic links: > > $ qfile -o /usr/bin/* > > … > > /usr/bin/xsubpp > > … > > $ equery b /usr/bin/xsubpp > > * Searching for /usr/bin/xsubpp ... > > dev-lang/perl-5.28.2-r1 (/usr/bin/xsubpp-3.390.0-perl-5.28.2) > > $ file /usr/bin/xsubpp > > /usr/bin/xsubpp: symbolic link to xsubpp-3.390.0-perl-5.28.2 > > > > Am I to open a separate bug for this issue? > > qfile intentionally does not resolve symlinks hmmm... qfile doesn't really bother about what it is, if it's in CONTENTS, it matches against the name listed there. It lists symlinks for me too. > Is this symbolic link present in CONTENTS for any package (/var/db/pkg)? (In reply to Sergey S. Starikoff from comment #6) > (In reply to Mike Gilbert from comment #5) > > Is this symbolic link present in CONTENTS for any package (/var/db/pkg)? > > Not present: > > $ grep xsubpp /var/db/pkg/dev-lang/perl-5.28.2-r1/CONTENTS if the symlink isn't recorded, there's no way qfile can find it. I don't know what equery b is doing to match the symlink to an owned file, perhaps it is resolving it because nothing else matched. This is disputable behaviour though. > Symlink may point anywhere in writable area. > So it's resolution is the thing should be done. > Of course with detecting broken symbolic links. If you add a symlink to something installed by Portage, it doesn't mean that symlink becomes "owned" by the owning package of the object it points to. In fact, that symlink really is an orphan. Note that for this scenario, things like eselect, gcc-config, binutils-config, java-config, all are problems, because they sometimes create files and symlinks in the system that are nowhere recorded as owned by their packages. Removing them, though, would yield in interesting, sometimes disastrous situations (like libgcc_s.so.1).
(In reply to Fabian Groffen from comment #7) > (In reply to Mike Gilbert from comment #5) > > qfile intentionally does not resolve symlinks > > hmmm... qfile doesn't really bother about what it is, if it's in CONTENTS, > it matches against the name listed there. It lists symlinks for me too. What I meant is that qfile treats symlinks as a distinct object, and doesn't really care what the symlink points at. We're on the same page.
(In reply to Mike Gilbert from comment #8) > (In reply to Fabian Groffen from comment #7) > > (In reply to Mike Gilbert from comment #5) > > > qfile intentionally does not resolve symlinks > > > > hmmm... qfile doesn't really bother about what it is, if it's in CONTENTS, > > it matches against the name listed there. It lists symlinks for me too. > > What I meant is that qfile treats symlinks as a distinct object, and doesn't > really care what the symlink points at. > > We're on the same page. After re-reading, I realised I misread your response at first. You were completely correct.
+1 as Portage won't remove files that were modified - when updating the whole system it is very easy to miss these "mtime" lines between hundreds of updated files. There are some third-party solutions for this (not reviewed by me) [1], [2], [3], but it would be good to have an officially reviewed and maintained tool for doing that instead. [1]: https://github.com/gcarq/portage-lostfiles [2]: https://github.com/rtgill82/gentoo-cruft [3]: https://github.com/vaeth/find_cruft/
(In reply to Maciej S. Szmigiero from comment #10) > +1 as Portage won't remove files that were modified - when updating the > whole system it is very easy to miss these "mtime" lines between hundreds of > updated files. > But Portage will clobber files if they're owned by a package, even if you've touched them manually since? $ emerge -v1 app-misc/hello [...] $ sha256sum /bin/hello 65d27b1286826c2800b5c20934903ffb5ab46f83e4937eaf01b51e99498a77df /bin/hello $ echo junk >> /bin/hello $ sha256sum /bin/hello b2bc71cfb8df31354973155cdd2dfd0a446d25dfec216ffc4e9d9588f607bca2 /bin/hello $ emerge -v1 app-misc/hello $ sha256sum /bin/hello 65d27b1286826c2800b5c20934903ffb5ab46f83e4937eaf01b51e99498a77df /bin/hello If we look at just removing files instead: $ echo junk >> /bin/hello $ emerge -C hello $ sha256sum /bin/hello sha256sum: /bin/hello: No such file or directory What situation are you referring to? Do you mean things which are CONFIG_PROTECTed?
> Do you mean things which are CONFIG_PROTECTed? Yes, CONFIG_PROTECTed stuff and also generic orphan files in the system - not necessarily installed by Portage but generated by packaged software themselves (possibility leftovers from older versions).
