Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 426912 - net-p2p/deluge unicode casting problems with net-libs/rb_torrent 0.16.x
Summary: net-p2p/deluge unicode casting problems with net-libs/rb_torrent 0.16.x
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL:
Whiteboard:
Keywords:
: 431782 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-07-16 21:45 UTC by Micah Bucy
Modified: 2016-01-30 18:03 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (emerge_info.log,5.29 KB, text/plain)
2012-07-16 21:45 UTC, Micah Bucy
Details
ebuild patch (file_426912.txt,532 bytes, text/plain)
2012-07-16 21:48 UTC, Micah Bucy
Details
deluge 1.3.5 patch (deluge-1.3.5-unicode.patch,1.41 KB, patch)
2012-07-17 03:24 UTC, Micah Bucy
Details | Diff
deluge 9999 patch (deluge-9999-unicode.patch,1.37 KB, patch)
2012-07-17 03:28 UTC, Micah Bucy
Details | Diff
deluge 1.3.5 ebuild patch (deluge-1.3.5.ebuild.patch,256 bytes, patch)
2012-07-17 03:31 UTC, Micah Bucy
Details | Diff
deluge 9999 ebuild patch (deluge-9999.ebuild.patch,254 bytes, patch)
2012-07-17 03:31 UTC, Micah Bucy
Details | Diff
deluge 1.3.5 patch (deluge-1.3.5-unicode.patch,1.42 KB, patch)
2012-11-10 02:21 UTC, Fernando
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Micah Bucy 2012-07-16 21:45:55 UTC
Created attachment 318378 [details]
emerge --info

Starting in version 0.16, rasterbar libtorrent makes use of #if defined UNICODE for checking for the need of std::wstring vs std::string (line 362 in http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/tags/libtorrent_0_16_2/include/libtorrent/config.hpp?revision=7204&view=markup).  This shows itself in my case when using the python bindings (http://libtorrent.svn.sourceforge.net/viewvc/libtorrent/tags/libtorrent_0_16_2/bindings/python/src/torrent_handle.cpp?revision=7204&view=markup) and move_storage gets called by for example deluge.  I have unicode as a global use flag.  I don't know if it's an issue with my system, but UNICODE is not defined, so when move_storage gets called from python libtorrent throws an error, expecting type std::string while getting type unicode.  What I did to fix this was to add unicode to IUSE for the ebuild.  I then use cppflags_append to add -DUNICODE if use unicode.  I don't know if this is the proper approach but it works.

I'm attaching my emerge --info.  I will make another post with a patch for the ebuild I am currently using following this.
Comment 1 Micah Bucy 2012-07-16 21:48:53 UTC
Created attachment 318380 [details]
ebuild patch
Comment 2 Rafał Mużyło 2012-07-16 22:27:38 UTC
Add more info about your usecase - UNICODE in this package is meant only for Windows. On Linux, it simply uses utf8.
Comment 3 Micah Bucy 2012-07-17 01:47:11 UTC
Use case is simple.  Install one of the rb_libtorrent 0.16.x releases.  Install deluge (1.3.5 and 9999 were tested).  After a download completes, right click the listing and select "Move Storage."  Nothing happens. (Automatic move upon finishing does the same thing if set.)  

If debug logging is enabled for deluged, a simple Error is displayed.  I chatted with andar on the #libtorrent channel at irc.freenode.net and he walked me through programming deluge to log more information on the exception.  I did so and the traceback showed that libtorrent was expecting std::string to be passed to move_storage, but was getting unicode instead.

I tracked down what could have changed in move_storage to cause this, and it was the lack of UNICODE causing TORRENT_USE_WSTRING to be false thus only allowing std::string usage with move_storage.  Passing -DUNICODE to the cppflags allows TORRENT_USE_WSTRING to be true and libtorrent allows for unicode to be passed to move_storage.

Like I said, I wasn't sure this was the proper method for resolving the issue, but it is now working for me.  If this package was only intending UNICODE for windows, then there's a bug upstream somewhere, but someone with more knowlege should report that.
Comment 4 Micah Bucy 2012-07-17 02:00:52 UTC
Actually, just took another look at deluge.  They intentionally cast the destination path as unicode for move_storage.

        # Attempt to convert utf8 path to unicode
        # Note: Inconsistent encoding for 'dest', needs future investigation
        try:
            dest_u = unicode(dest, "utf-8")
        except TypeError:
            # String is already unicode
            dest_u = dest

So it would appear this is in fact a deluge bug.  Not sure exactly what deluge needs to do instead and maintain cross platform consistency.
Comment 5 Micah Bucy 2012-07-17 03:22:19 UTC
http://git.deluge-torrent.org/deluge/commit/deluge/core/torrent.py?id=417a9f6e63e75e5c8b5c2cd9e922aea1723c50de is where that code got added.  Apparently a windows issue.  Until deluge upstream is working, should just unapply this patch.  Will be attaching patch files for 1.3.5 and 9999 respectively.
Comment 6 Micah Bucy 2012-07-17 03:24:07 UTC
Created attachment 318396 [details, diff]
deluge 1.3.5 patch
Comment 7 Micah Bucy 2012-07-17 03:28:16 UTC
Created attachment 318398 [details, diff]
deluge 9999 patch
Comment 8 Micah Bucy 2012-07-17 03:31:03 UTC
Created attachment 318400 [details, diff]
deluge 1.3.5 ebuild patch
Comment 9 Micah Bucy 2012-07-17 03:31:51 UTC
Created attachment 318402 [details, diff]
deluge 9999 ebuild patch
Comment 10 Micah Bucy 2012-07-18 05:10:26 UTC
Turns out my inital attempt only resolves the issue for manually moving data in deluge.  When set to automatically move when a download finishes, the issue crops up again.  Need to investigate further.
Comment 11 Alex 2012-07-31 15:49:46 UTC
I have a bug related with rb_libtorrent-0.16 on deluge-1.3.5 too

Deluge just do not work. Very slowly download and almost do not upload.
Comment 12 Rafał Mużyło 2012-08-21 16:42:10 UTC
*** Bug 431782 has been marked as a duplicate of this bug. ***
Comment 13 Rafał Mużyło 2012-08-21 19:25:40 UTC
I suspect part of the answer is here (http://code.google.com/p/libtorrent/issues/detail?id=173):
> unfortunately, in 0.15 and earlier, the torrent creation is an exception from
> the rest of the API in that it doesn't re-encode strings for filenames.
>
> You may want to try to use the wide string version of add files,
> especially on windows.

However, the problem deluge has is that it needs to handle Windows and Unix-like differently. Actually, it's (arguably) a little design flaw of libtorrent...

Well, not quite. It's more that libtorrent_rasterbar was written in a way that it could be built/work on Windows versions that don't support Unicode. That was a feature back when Win9x was common, now is a misfeature.
On the other hand, even these days there are Windows programs released not built with wide API :roll:.
Comment 14 Torbjörn Lönnemark 2012-11-04 19:24:05 UTC
Upstream bug report: http://dev.deluge-torrent.org/ticket/2140
Comment 15 Fernando 2012-11-10 02:21:10 UTC
Created attachment 329054 [details, diff]
deluge 1.3.5 patch

Fixes "move completed to" when download finishes.
Comment 16 Heiss 2013-01-26 06:22:19 UTC
Deluge git patch for that bug is here:
http://git.deluge-torrent.org/deluge/commit/?h=1.3-stable&id=7d88cb1850eaf4d15646bdbf919b4451de0cbfea

It looks a bit more extensive than the patch attached to this bug.
Comment 17 Diogo Pereira 2015-10-20 21:26:49 UTC
This was fixed years ago. Please close.