Running duplicity "generates" the following error: # /usr/bin/duplicity Traceback (most recent call last): File "/usr/bin/duplicity", line 31, in <module> from future import standard_library ModuleNotFoundError: No module named 'future' I found bug #705726 but it did not help to re-emerge dev-python/future (or app-backup/duplicity). Multiple python versions are installed: # eselect python list Available Python interpreters, in order of preference: [1] python3.7 [2] python3.6 [3] python2.7 (fallback) Selecting python3.6 produces an other error: " TypeError: a bytes-like object is required, not 'str'" Reproducible: Always
Created attachment 616704 [details] Output from "emerge --info"
Thank you for the report. Please recompile and *attach* the logfiles as described on https://wiki.gentoo.org/wiki/Attach_the_logs_to_the_bug_ticket We need to have all information at hand before ticket assignment, so that the maintainer can look after it in one session with minimal number of callbacks.
Created attachment 616706 [details] Duplicity build log
Created attachment 616708 [details] Future build log
I'll see what I can figure out with this. It seems likely to be related to: https://bugs.gentoo.org/710500
I was able to make duplicity work by explicitly invoking it as `python3.6 /usr/bin/duplicity`. I'm not familiar with the python ecosystem nor how it specifically works on Gentoo, but if anyone has further ideas for testing I'm happy to try things out.
Starting duplicity with python3.6 /usr/bin/duplicity "solved" the "future" error here as well, BUT now I get another error: Example: # /usr/bin/python3.6 /usr/bin/duplicity collection-status --gpg-options='--pinentry-mode loopback' --encrypt-sign-key=xxxxxxx --s3-use-new-style --s3-european-buckets --s3-use-multiprocessing --name=ABCD --archive-dir=/var/duplicity s3+http://xxxx.zzzz.yyyy Traceback (innermost last): File "/usr/bin/duplicity", line 101, in <module> with_tempdir(main) File "/usr/bin/duplicity", line 87, in with_tempdir fn() File "/usr/lib64/python3.6/site-packages/duplicity/dup_main.py", line 1525, in main action = commandline.ProcessCommandLine(sys.argv[1:]) File "/usr/lib64/python3.6/site-packages/duplicity/commandline.py", line 1174, in ProcessCommandLine globals.backend = backend.get_backend(args[0]) File "/usr/lib64/python3.6/site-packages/duplicity/backend.py", line 225, in get_backend obj = get_backend_object(url_string) File "/usr/lib64/python3.6/site-packages/duplicity/backend.py", line 211, in get_backend_object return factory(pu) File "/usr/lib64/python3.6/site-packages/duplicity/backends/_boto_multi.py", line 94, in __init__ BotoSingleBackend.__init__(self, parsed_url) File "/usr/lib64/python3.6/site-packages/duplicity/backends/_boto_single.py", line 173, in __init__ self.resetConnection() File "/usr/lib64/python3.6/site-packages/duplicity/backends/_boto_single.py", line 199, in resetConnection location=self.my_location) File "/usr/lib64/python3.6/site-packages/boto/s3/connection.py", line 619, in create_bucket data=data) File "/usr/lib64/python3.6/site-packages/boto/s3/connection.py", line 671, in make_request retry_handler=retry_handler File "/usr/lib64/python3.6/site-packages/boto/connection.py", line 1071, in make_request retry_handler=retry_handler) File "/usr/lib64/python3.6/site-packages/boto/connection.py", line 913, in _mexe self.is_secure) File "/usr/lib64/python3.6/site-packages/boto/connection.py", line 705, in get_http_connection return self.new_http_connection(host, port, is_secure) File "/usr/lib64/python3.6/site-packages/boto/connection.py", line 747, in new_http_connection connection = self.proxy_ssl(host, is_secure and 443 or 80) File "/usr/lib64/python3.6/site-packages/boto/connection.py", line 796, in proxy_ssl sock.sendall("CONNECT %s HTTP/1.0\r\n" % host) TypeError: a bytes-like object is required, not 'str' BUT your solution brought me an other idea. If I start duplicity with python 2.7 like this "/usr/bin/python2.7 /usr/bin/duplicity ..." it works. I will implement this as a workaround for now. -- Dan
The Workaround worked well until this morning when duplicity was reemerged with "(-python2_7)". Trying to put "app-backup/duplicity PYTHON_TARGETS: -* python2_7 PYTHON_SINGLE_TARGET: -* python2_7" in package.use now results in: "The ebuild selected to satisfy "app-backup/duplicity" has unmet requirements." And running duplicity with python3.6 results with the error in https://bugs.gentoo.org/711158#c7 (:-( Suggestions? -- Dan
Looks like python 2 support was removed as part of the general removal of python 2 from the repo - that was part of my motivation for getting duplicity updated. I'm not getting this error, so it is hard to say what is causing it. You're running 0.8.09-r1 which doesn't even support python 3.7 so it is hard to see why you'd need to explicitly launch 3.6, or why it would have boto errors since building duplicity with 3.6 support should force boto to be built with 3.6 support. You might want to check your versions - are you actually running the version of duplicity listed in your build log?
I wonder if involving upstream might help. According to [1], "Anything that doesn't work with Python 3 is a bug that should definitely be fixed.", and the latest version's setup.py (0.8.12.1596, released 2020-03-19) contains the following line: python_requires=u">2.6, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, <4" Duplicity should therefore work with any Python version in Gentoo's repo tree, as far as I can tell. [1] https://answers.launchpad.net/duplicity/+question/687576
In between I have "unmasked" app-backup/duplicity-0.8.11.1596-r1 but I still had the same issues. Starting without /usr/bin/python3.6 it still errors out with "ModuleNotFoundError: No module named 'future'". I have been searching the net and found two "know" issues with duplicity with s3 backend. First a problem with boto (duplicity does not yet use boto3): https://github.com/boto/boto/pull/3713/commits/a4fcfb3bf9fb6a3f6f6b79a96ac79253b60d207b And second a problem is in duplicity itself: https://bugs.launchpad.net/duplicity/+bug/1865648 (this is fixed in duplicity 0.8.12) After manually patching these files I can run duplicity again. I know manually patching "random" files could have other side effects - but for me it is OK in this case as I only use boto with duplicity. -- Dan
I was successful with the following workaround: * python3.8 -m venv venv ; source venv/bin/activate * Download and unpack the latest official duplicity tarball. * python setup.py sdist * pip install dist/duplicity-0.8.12.0.tar.gz In my wrapper script, I use duplicity from that virtual environment. So far, it seems to be working.
(In reply to Dan Johansson from comment #11) > In between I have "unmasked" app-backup/duplicity-0.8.11.1596-r1 but I still > had the same issues. That would be an important detail. > Starting without /usr/bin/python3.6 it still errors out with > "ModuleNotFoundError: No module named 'future'". That makes sense. Have you tried just using python 3.6? And you can't do that by just launching it with python3.6 - you need to build it with python 3.6 support and not python 3.7 support, or manually make sure that boto has 3.6 support or else boto will probably fail the way it did above. > (this is fixed in duplicity 0.8.12) I'll see about getting 0.8.12 in the tree - that is on my todo list...
(In reply to Richard Freeman from comment #13) > (In reply to Dan Johansson from comment #11) > > In between I have "unmasked" app-backup/duplicity-0.8.11.1596-r1 but I still > > had the same issues. > > That would be an important detail. I did that that just about 1h before I wrote the comment... > Have you tried just using python 3.6? And you can't do that by just > launching it with python3.6 - you need to build it with python 3.6 support > and not python 3.7 support, or manually make sure that boto has 3.6 support > or else boto will probably fail the way it did above. No, that I have not yet tried. > > (this is fixed in duplicity 0.8.12) > > I'll see about getting 0.8.12 in the tree - that is on my todo list... Would be great, but no stress. I appreciate the work you put into this. -- Dan
Ok, please report whether this persists with 0.8.12.1612.
(In reply to Richard Freeman from comment #15) > Ok, please report whether this persists with 0.8.12.1612. Emerged 0.8.12.1612 today and running this version "prefixed" with /usr/bin/python3.6 works, but without it still fails with: Traceback (most recent call last): File "/usr/bin/duplicity", line 30, in <module> from future import standard_library ModuleNotFoundError: No module named 'future' This probably comes from the fact that my default python3 is set to 3.7: # eselect python list Available Python interpreters, in order of preference: [1] python3.7 [2] python3.6 [3] python2.7 (fallback) and duplicity only gets installed (as I read the output from eix) for 3.6: [I] app-backup/duplicity Available versions: 0.8.09-r1^t ~0.8.11.1596^t ~0.8.11.1596-r1^t (~)0.8.12.1612^t {s3 test PYTHON_TARGETS="python3_6 python3_7"} Installed versions: 0.8.12.1612^t(06:34:15 PM 03/30/2020)(s3 -test PYTHON_TARGETS="python3_6 -python3_7") But I have to live with the patched boto file until duplicity 0.9 (which is supposed to have boto3, aka. python 3.x, support. Regards, -- Dan
(In reply to Richard Freeman from comment #5) > It seems likely to be related to: > https://bugs.gentoo.org/710500 Yeah, I think this is a dupe of bug #710500
I'm going to close this again pending a confirmation that this still occurs in 0.8.12.1612-r1. There was an issue with python-exec that could have caused this issue. If it is still happening with this version please let me know, especially since we would like to stabilize it soon (and the current stable version will probably go away soon).
This now works for me.
