The attempt to launch duplicity results in the following error: $ duplicity Traceback (most recent call last): File "/usr/bin/duplicity", line 40, in <module> import fasteners ModuleNotFoundError: No module named 'fasteners' $ equery l duplicity * Searching for duplicity ... [IP-] [ ] app-backup/duplicity-0.8.09-r1:0 $ equery u duplicity [ Legend : U - final flag setting for installation] [ : I - package is installed with flag ] [ Colors : set, unset ] * Found these USE flags for app-backup/duplicity-0.8.09-r1: U I + + python_targets_python2_7 : Build with Python 2.7 + + python_targets_python3_6 : Build with Python 3.6 - - s3 : Support for backing up to the Amazon S3 system - - test : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
See also https://bugs.gentoo.org/705726 . This type of bug should be detected during testing, should it not?
(In reply to Ralph Seichter from comment #1) > See also https://bugs.gentoo.org/705726 . This type of bug should be > detected during testing, should it not? Indeed, thanks for testing!
Actually, this is a bit bizarre. You do have dev-python/fasteners installed, right? It is actually a dependency in the package, so emerge shouldn't install duplicity unless you have it installed. Of course, you could install it manually in some way and skip dependency resolution, in which case it would break. Feel free to re-open with this info.
(In reply to Richard Freeman from comment #2) > Indeed, thanks for testing! If by "testing" you mean "having the updated ebuild automatically installed and later realising that system backups no longer work", you're welcome. ;-) Seriously though: Is there no test process involved, CI-based or other, that catches these obvious problems? If I, as a proxy maintainer, wanted to add changes with unmet dependencies to the Gentoo tree, somebody would probably notice (and possibly give me a slap on the wrist).
(In reply to Richard Freeman from comment #3) > You do have dev-python/fasteners installed, right? Looks like it: $ equery l dev-python/fasteners * Searching for fasteners in dev-python ... [IP-] [ ] dev-python/fasteners-0.14.1-r1:0 $ equery u dev-python/fasteners [ Legend : U - final flag setting for installation] [ : I - package is installed with flag ] [ Colors : set, unset ] * Found these USE flags for dev-python/fasteners-0.14.1-r1: U I + + python_targets_python2_7 : Build with Python 2.7 + + python_targets_python3_6 : Build with Python 3.6 + + python_targets_python3_7 : Build with Python 3.7 - - test : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled independently)
(In reply to Ralph Seichter from comment #4) > (In reply to Richard Freeman from comment #2) > > > Indeed, thanks for testing! > If by "testing" you mean "having the updated ebuild automatically installed > and later realising that system backups no longer work", you're welcome. ;-) > > Seriously though: Is there no test process involved, CI-based or other, that > catches these obvious problems? If I, as a proxy maintainer, wanted to add > changes with unmet dependencies to the Gentoo tree, somebody would probably > notice (and possibly give me a slap on the wrist). So, in this case there are no unmet dependencies that I'm aware of. However, in general I'm not aware of any tools that detect unspecified python dependencies. If you're aware of any I'm all ears. I've certainly seen scripts that detect unspecified linked dependencies for binaries and do use them routinely. What we do have is a workflow where only users who volunteer to test packages get exposed to new packages, as you did by accepting the ~arch keyword. :) In any case - if there is something wrong with the ebuild I'm more than happy to fix it. Certainly I wouldn't like broken backups either, and I rely on this package daily myself.
(In reply to Ralph Seichter from comment #5) > (In reply to Richard Freeman from comment #3) > > > You do have dev-python/fasteners installed, right? > Looks like it: > Interesting. Not sure what is going on there. Seems like both packages were built for python 2.7/3.6.
(In reply to Richard Freeman from comment #6) > in general I'm not aware of any tools that detect unspecified > python dependencies. If you're aware of any I'm all ears. In case that sounded odd: I had no intention of chastising you. I am genuinely interested in whether or not there are CI processes that can pick up these issues. I write a lot of software in Python and have just today opened a pull request for a new package (app-antivirus/fangfrisch). I rely on Larry the Cow and the proxy maintainers' watchful eyes to pick up possible errors in ebuilds.
No offense taken, and if you do learn of some tool I'm all ears. Auditing python dependencies is certainly something that would be useful. What would help is if we had a container/jail/etc-based build system that prevented packages from accessing non-deps during build. Though that wouldn't help with a runtime issue which a python issue is more likely to be. Something like this for runtime wouldn't hurt. In this case though I have no idea what is going on, as you have the package installed and it is listed as a dep. Obviously there is some sort of issue. I'm not running into it personally. Maybe I'll try to play around in a minimal container with it...
The error is still present with app-backup/duplicity-0.8.11.1596:0
Ok, please report whether this persists with 0.8.12.1612.
Sadly, the problem persists. $ equery l duplicity * Searching for duplicity ... [IP-] [ ] app-backup/duplicity-0.8.12.1612:0 $ duplicity Traceback (most recent call last): File "/usr/bin/duplicity", line 39, in <module> import fasteners ModuleNotFoundError: No module named 'fasteners'
The problem is that duplicity executables are not wrapped with python-exec: $ head -n1 /usr/bin/duplicity /usr/bin/rdiffdir ==> /usr/bin/duplicity <== #!/usr/bin/env python3 ==> /usr/bin/rdiffdir <== #!/usr/bin/env python3 So if selected python version is not in PYTHON_TARGETS duplicity was built with, it will not work. I see two possible solutions: 1. Support only single python target (DISTUTILS_SINGLE_IMPL) and fix shebangs (python_fix_shebang()). 2. Wrap executables with python-exec (python_doexe()). But do we really need to support multiple python targets in duplicity?
Created attachment 636696 [details] duplicity-0.8.12.1612.ebuild.patch OK, the easiest solution seems to just wrap all executables. Please try attached patch.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=d554635fb1e0683395b6cf1c8ac205e8c9bc0327 commit d554635fb1e0683395b6cf1c8ac205e8c9bc0327 Author: Richard Freeman <rich0@gentoo.org> AuthorDate: 2020-05-07 16:30:52 +0000 Commit: Richard Freeman <rich0@gentoo.org> CommitDate: 2020-05-07 16:52:46 +0000 app-backup/duplicity: setup.py fix to allow python-exec wrapping Thanks to Alexander who discovered the root cause of this problem. Credit: Alexander Tsoy Closes: https://bugs.gentoo.org/710500 Package-Manager: Portage-2.3.99, Repoman-2.3.22 Signed-off-by: Richard Freeman <rich0@gentoo.org> .../duplicity/duplicity-0.8.12.1612-r1.ebuild | 50 ++++++++++++++++++++++ .../files/duplicity-0.8.12.1612-fix-docs-cmd.patch | 29 +++++++++++++ 2 files changed, 79 insertions(+)
Please test this to confirm that it fixes your issue. I think you correctly identified the root cause, but better fix was to patch setup.py, which apparently was designed to block shebang editing/etc. Let me know if this works though, as I'd like to get this stabilized as it is blocking the python migration. I did testing and confirm that it works, but I wasn't the one with the problem...
Thanks! It fixed my issue. I didn't notice that it was a problem in setup.py.
(In reply to Alexander Tsoy from comment #17) > Thanks! It fixed my issue. I didn't notice that it was a problem in setup.py. Yeah, floppym gets the credit for spotting that. However, I figured from reading the distutils-r1 docs that something was going wrong with setup.py since that is supposed to wrap everything already. There are no other packages using distutils which use doexe either.
