Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 593672 - app-portage/gentoolkit-0.3.2-r1: always wants to rebuild dev-util/android-sdk-update-manager
Summary: app-portage/gentoolkit-0.3.2-r1: always wants to rebuild dev-util/android-sdk...
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2016-09-13 15:32 UTC by Jason A. Donenfeld
Modified: 2017-02-03 02:03 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch to fix filename/dir masking matching (0001-revdep-rebuild:-Fixfilenamematchingfordirectories.patch,3.25 KB, patch)
2016-09-13 23:53 UTC, Brian Dolbec
Details | Diff
version 2 patch (0001-revdep-rebuild:-Fixfilenamematchingfordirectories.patch,3.47 KB, patch)
2016-09-14 14:53 UTC, Brian Dolbec
Details | Diff
v3 adds testing realpaths as well (0001-revdep-rebuild:-Fixfilenamematchingfordirectories.patch,3.60 KB, patch)
2016-09-14 17:45 UTC, Brian Dolbec
Details | Diff
adds realpath on the dirs (0001-revdep-rebuild:-Userealpathonthemaskdirectoriestoo(Bug593672).patch,872 bytes, patch)
2016-09-14 19:51 UTC, Brian Dolbec
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2016-09-13 15:32:46 UTC
No matter how many times I run it, it still wants to emerge it again. This might be a search path mask issue, since /opt/android-sdk-update-manager is already a part of SEARCH_DIRS_MASK. The problem isn't there in revdep-rebuild.sh.

# revdep-rebuild -v -- -a
 * This is the new python coded version
 * Please report any bugs found using it.
 * The original revdep-rebuild script is installed as revdep-rebuild.sh
 * Please file bugs at: https://bugs.gentoo.org/
 * Collecting system binaries and libraries
 * Collecting dynamic linking informations
 * Scanning files
 * Checking dynamic linking consistency
 * Broken files that requires: libncurses.so.5 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so
 * Broken files that requires: libpulse.so.0 (32 bits)
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64
 * Broken files that requires: libXrender.so.1 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/monitor-x86/libcairo-swt.so
 * Broken files that requires: libXfixes.so.3 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
 * Broken files that requires: libXdamage.so.1 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
 * Broken files that requires: libncurses.so.5 (64 bits)
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libosmesa.so
 * Assign files to packages
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/monitor-x86/libcairo-swt.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libosmesa.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so.1 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64 -> dev-util/android-sdk-update-manager-24.4.1

emerge -a --verbose --oneshot --complete-graph=y dev-util/android-sdk-update-manager:0

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-util/android-sdk-update-manager-24.4.1::gentoo  0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] 

$ cat /etc/revdep-rebuild/80android-sdk-update-manager 
SEARCH_DIRS_MASK="/opt/android-sdk-update-manager"
Comment 1 Zac Medico gentoo-dev 2016-09-13 16:34:33 UTC
(In reply to Jason A. Donenfeld from comment #0)
>  * Broken files that requires: libncurses.so.5 (32 bits)
>         * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
>         * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
>         * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so

This indicates a missing dependency on sys-libs/ncurses:5/5[abi_x86_32], and missing dependencies like this will trigger @preserved-rebuild loops if you manage to have libncurses.so.5 preserved via preserve-libs.
Comment 2 Paul Varner (RETIRED) gentoo-dev 2016-09-13 19:24:25 UTC
This is definitely a bug with the handling of masking.

The offending code is this in analyze.py

if filename in self.all_masks:
    self.logger.debug('\tFile %s ignored as it is masked' % filename)
    continue

The problem is self.all_masks contains just the names of masked directories and files and a full filename is never going to be in that set.  We need to change the logic to see if the full filename matches an entry in the allmasks set.
Comment 3 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2016-09-13 22:45:19 UTC
As requested, here's the --debug output.

zx2c4@thinkpad ~ $ sudo revdep-rebuild -v --debug -- -p
 * This is the new python coded version
 * Please report any bugs found using it.
 * The original revdep-rebuild script is installed as revdep-rebuild.sh
 * Please file bugs at: https://bugs.gentoo.org/
        main(), _libs_to_check = set()
        all_masks:

                /lib/modules
                /lib32/modules
                /lib64/modules
                /opt/android-sdk-update-manager
                /opt/oracle-jdk-bin-1.8.0.102
                libjava.so
                libjawt.so
                libjvm.so
                libodbc.so
                libodbcinst.so
 * Found a valid cache, skipping collecting phase
        analyse(), Found 33984 libraries (+6932 symlinks) and 23014 binaries
 * Scanning files
        scan(), scanelf params = ['-BF', '%F;%f;%S;%n;%M'], # files: 36794
        scan(), final output length: 29204
        scan_files(); total time to get scanelf data is 698 milliseconds
        scan_files(); total filenames found: 16665 in 777 milliseconds
 * Checking dynamic linking consistency
        analyse(), Searching for 36794 libs, bins within 40916 libraries and links
        LibCheck.__init__(), new searchlibs: ['32', '64']
        LibCheck.search(), Checking for broken 32 bit libs
        LibCheck.search(); FOUND: 32bit, libXdamage.so.1, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        LibCheck.search(); FOUND: 32bit, libXfixes.so.3, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        LibCheck.search(); FOUND: 32bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        LibCheck.search(); FOUND: 32bit, libpulse.so.0, /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64
        LibCheck.search(); FOUND: 32bit, libXrender.so.1, /opt/android-sdk-update-manager/tools/lib/monitor-x86/libcairo-swt.so
        LibCheck.search(); FOUND: 32bit, libpulse.so.0, /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64
        LibCheck.search(); FOUND: 32bit, libpulse.so.0, /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el
        LibCheck.search(); FOUND: 32bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so
        LibCheck.search(); FOUND: 32bit, libXdamage.so.1, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
        LibCheck.search(); FOUND: 32bit, libXfixes.so.3, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
        LibCheck.search(); FOUND: 32bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
        LibCheck.search(), Checking for broken 64 bit libs
        LibCheck.search(); FOUND: 64bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so.1
        LibCheck.search(); FOUND: 64bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libosmesa.so
        Library libjawt.so ignored as it is masked
        LibCheck.search(); FOUND: 64bit, libncurses.so.5, /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so
        LibCheck.search(); Total found: 6 libs, 14 files in 3451 milliseconds
 * Broken files that requires: libpulse.so.0 (32 bits)
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64
 * Broken files that requires: libXrender.so.1 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/monitor-x86/libcairo-swt.so
 * Broken files that requires: libXdamage.so.1 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
 * Broken files that requires: libXfixes.so.3 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
 * Broken files that requires: libncurses.so.5 (32 bits)
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so
 * Broken files that requires: libncurses.so.5 (64 bits)
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libosmesa.so
        LibCheck.process_results(); total filepaths found: 14 in 1 milliseconds
 * Assign files to packages
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libosmesa.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so.1 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/gles_mesa/libGL.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib/monitor-x86/libcairo-swt.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libosmesa.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so.1 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/lib64/gles_mesa/libGL.so -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64 -> dev-util/android-sdk-update-manager-24.4.1
        assign_packages(); assigned 1 packages, 0 orphans in 642 milliseconds

emerge -p --verbose --oneshot --complete-graph=y dev-util/android-sdk-update-manager:0

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-util/android-sdk-update-manager-24.4.1::gentoo  0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB
        rebuild(); emerge call for 1 ebuilds took: 20.681 seconds
rebuild return code = 0


Paul's analysis seems absolutely correct.
Comment 4 Brian Dolbec gentoo-dev 2016-09-13 23:53:34 UTC
Created attachment 445628 [details, diff]
Patch to fix filename/dir masking matching

Please try this patch.  It should fix the issue, plus another one I spotted in the code while fixing it.

My system is consistent atm, so not as easy to test thoroughly.
Comment 5 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2016-09-14 09:49:38 UTC
While this might work for most cases, the code is fundamentally buggy:

+	def is_masked(self, filename):
+		for m in self.masked_dirs:
+			if filename.startswith(m):
+				return True
+		return False
+

The reason is that masked_dirs often contains entries that do not end with a '/'. So, file /a/b-c would match directory /a/b which simply isn't correct.
Comment 6 Brian Dolbec gentoo-dev 2016-09-14 14:53:06 UTC
Created attachment 445696 [details, diff]
version 2 patch

You are quite right.  It was bugging me that that previous fix was a little too quick, I might have missed something.

This updated patch should ensure the matching is done at the path boundaries.
Comment 7 Paul Varner (RETIRED) gentoo-dev 2016-09-14 15:24:26 UTC
Patch looks good to me.
Comment 8 Zac Medico gentoo-dev 2016-09-14 15:58:24 UTC
Not that an alternative to string comparison is to stat the corresponding directories and compare the st_dev and st_ino attributes (like os.path.samefile does), which is more robust be cause it handles differences in paths due to symlinks (which never works with path comparison unless you use realpath).
Comment 9 Brian Dolbec gentoo-dev 2016-09-14 17:45:01 UTC
Created attachment 445706 [details, diff]
v3 adds testing realpaths as well

Test realpaths as well
Comment 10 Zac Medico gentoo-dev 2016-09-14 19:29:38 UTC
You should probably also realpath each of masked_dirs for the comparison to be really meaningful.
Comment 11 Brian Dolbec gentoo-dev 2016-09-14 19:36:40 UTC
(In reply to Zac Medico from comment #10)
> You should probably also realpath each of masked_dirs for the comparison to
> be really meaningful.

I thought about that, but isn't that a corner case... and shouldn't people be putting in the real path in those masks to begin with?  

It'll be easy enough to realpath the dir too.
Comment 12 Zac Medico gentoo-dev 2016-09-14 19:44:18 UTC
(In reply to Brian Dolbec from comment #11)
> (In reply to Zac Medico from comment #10)
> > You should probably also realpath each of masked_dirs for the comparison to
> > be really meaningful.
> 
> I thought about that, but isn't that a corner case... and shouldn't people
> be putting in the real path in those masks to begin with?  

Sounds like an unnecessary assumption. Why burden the user with stuff like that?

> It'll be easy enough to realpath the dir too.
Comment 13 Brian Dolbec gentoo-dev 2016-09-14 19:51:49 UTC
Created attachment 445730 [details, diff]
adds realpath on the dirs
Comment 14 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2016-09-15 03:45:52 UTC
Seems to work well. Thanks for drawing that up so quickly. Ship it!
Comment 15 urcindalo 2016-12-17 16:40:41 UTC
After the recent app-portage/gentoolkit-0.3.2-r1 stabilization on amd64, I've been hit by this bug.

Running revdep-rebuild has re-emerged android-sdk-update-manager (twice in a row):
===============

Calculating dependencies... done!
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.
 * This is the new python coded version
 * Please report any bugs found using it.
 * The original revdep-rebuild script is installed as revdep-rebuild.sh
 * Please file bugs at: https://bugs.gentoo.org/
 * Collecting system binaries and libraries
 * Collecting dynamic linking informations
 * Scanning files
 * Checking dynamic linking consistency
 * Broken files that requires: libpulse.so.0 (32 bits)
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64
 * Broken files that requires: libpulse.so.0 (64 bits)
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-aarch64
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-mips64el
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-x86_64
 * Assign files to packages
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-x86_64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-aarch64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86/qemu-system-mips64el -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-x86_64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-aarch64 -> dev-util/android-sdk-update-manager-24.4.1
        * /opt/android-sdk-update-manager/tools/qemu/linux-x86_64/qemu-system-mips64el -> dev-util/android-sdk-update-manager-24.4.1

emerge  --verbose --oneshot --complete-graph=y dev-util/android-sdk-update-manager:0
Comment 16 Fabian Henze 2017-01-31 08:31:04 UTC
any news on this? for two months revdep-rebuild keeps rebuilding the android-sdk-update-manager for no reason...
Comment 17 Brian Dolbec gentoo-dev 2017-02-03 02:03:11 UTC
This has been released in gentoolkit-0.3.3