aterm no longer pastes high ascii properly For example, pasting the word "fran
aterm no longer pastes high ascii properly For example, pasting the word "français" (8 characters) into aterm causes the literal "fran\x{00E7}ais" (15 characters) to be pasted instead It seems to be some silly UTF8 forced-representation bug in one of the patches in files/ Reproducible: Always Steps to Reproduce: 1. Copy to clipboard: français 2. Paste into aterm Actual Results: fran\x{00E7}ais was pasted instead Expected Results: Pasted français as-is This bug affects ebuilds 0.4.2-r8 and 0.4.2-r7 Downgrading to 0.4.2-r6 fixes that problem, but loses some of the important pasting patches, such as the large-paste patch (INCR support)
I have narrowed the problem down to files/aterm-0.4.2-copynpaste-r2.patch Specifically, to this chunk in the file: --- cut here --- @@ -2787,6 +2854,9 @@ selection_request(Time tm, int x, int y) { Atom prop; + Atom xa; + + xa = XInternAtom(Xdisplay, "COMPOUND_TEXT", False); if (x < 0 || x >= TermWin.width || y < 0 || y >= TermWin.height) return; /* outside window */ @@ -2797,7 +2867,7 @@ selection_paste(Xroot, XA_CUT_BUFFER0, False); } else { prop = XInternAtom(Xdisplay, "VT_SELECTION", False); - XConvertSelection(Xdisplay, XA_PRIMARY, XA_STRING, prop, TermWin.vt, + XConvertSelection(Xdisplay, XA_PRIMARY, xa, prop, TermWin.vt, tm); } } --- cut here --- If I remove that chunk from the patch and re-emerge, my problem is fixed.
Could you please try -r9? I experience no such problems with it. But then, I didn't have them with -r8 and -r7, either. If the problem is still there with -r9, please tell me your LANG and LC_ALL settings, and paste the output of `emerge info`.
I've tried -r9 and it has the same problem. My environment vars LANG and LC_ALL are unset by default forcing them for aterm -r9 such as: $ LANG=C LC_ALL=C aterm or $ LANG=en_US LC_ALL=en_US aterm Does not affect anything - problem still persists. Here's my `emerge info` output: Portage 2.0.50-r3 (default-x86-1.4, gcc-3.3.2, glibc-2.3.2-r9, 2.6.3-gentoo-r1) ================================================================= System uname: 2.6.3-gentoo-r1 i686 Pentium III (Coppermine) Gentoo Base System version 1.4.3.13 Autoconf: sys-devel/autoconf-2.58-r1 Automake: sys-devel/automake-1.8.3 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-O3 -march=pentium3 -fprefetch-loop-arrays -funroll-loops -pipe" CHOST="i686-pc-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=pentium3 -fprefetch-loop-arrays -funroll-loops -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache sandbox" GENTOO_MIRRORS="http://gentoo.mirrors.pair.com/ ftp://gentoo.mirrors.pair.com/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X aalib apache2 apm avi berkdb crypt cups encode esd foomaticdb gdbm gif gnome gpm gtk gtk2 imlib java jpeg ldap libg++ libwww mad mikmod mmx motif mozilla moznocompose moznoirc mpeg msn mysql ncurses nls oci8 oggvorbis opengl oscar oss pam pdflib perl png python qt quicktime readline sdl slang spell sse ssl svga tcpd tetex truetype x86 xml2 xmms xv zlib"
Could you check if similar problems occur with rxvt and xterm?
No it does not. rxvt and xterm both behave very well. Like I showed in http://bugs.gentoo.org/show_bug.cgi?id=47018#c1 , the bug is caused by that chunk in the patch file. I removed that chunk from the patch file and re-emerged aterm and my problem is fixed. And so at this point I do not have a problem as per my above fix. I don't know exactly what that chunk in the patch does but if it's not useful it might be a good idea to take it out.
Removing the lines you pointed out is, unfortunately, out of the question. These lines add support for pasting non-ASCII characters (eg. national characters from different languages). I'll see what I can do now that I know rxvt works fine.
I had a look at both aterm's and rxvt's pasting mechanism and well - I can't see any relevant differences. I must be missing something here. Could you please try to do: export DEBUG="-DDEBUG_SELECT" emerge rxvt then start rxvt from another terminal window (so that you can see the debug messages), try to paste the 'fran
I had a look at both aterm's and rxvt's pasting mechanism and well - I can't see any relevant differences. I must be missing something here. Could you please try to do: export DEBUG="-DDEBUG_SELECT" emerge rxvt then start rxvt from another terminal window (so that you can see the debug messages), try to paste the 'français' word and post the rxvt's output here? You should get something like this: rxvt_selection_request(13849417, 0, 0) rxvt_selection_request_other: pasting SECONDARY rxvt_selection_property(03600006, 340) rxvt_selection_paste(03600006, 340, 1), wait= 1 rxvt_selection_paste: bytes written: 11
Bizarre. I could've sworn RXVT behaved well when I tried it based on your earlier request, but I might have made a mistake somewhere. Anyways, rxvt (now) exhibits the same problem. Here's the debug output on pasting the word fran
Bizarre. I could've sworn RXVT behaved well when I tried it based on your earlier request, but I might have made a mistake somewhere. Anyways, rxvt (now) exhibits the same problem. Here's the debug output on pasting the word français: rxvt_selection_request(8935679, 223, 90) rxvt_selection_request_other: pasting SECONDARY rxvt_selection_property(01400006, 409) rxvt_selection_paste(01400006, 409, 1), wait= 1 rxvt_selection_paste: bytes written: 15 The actual word pasted is "fran\x{00E7}ais" (15 characters) Maybe the aterm ebuild could detect some USE flag (utf8/whatever) and based on that include or not include the troublesome patch ? Just my $0.02
What version of x11 are you using? It appears the problem here might be caused the X server failing to convert the selection to COMPOUND_TEXT properly. And BTW, does this happen when pasting from a particular app (such as Mozilla) or just from any app?
I'm using the latest stable X in portage: x11-base/xfree-4.3.0-r5 Testing from different copy sources: 1. From mozilla == problem 2. From gvim (gui) == problem 3. From aterm/rxvt/xterm == no problem 4. From an "xmessage" window == no problem Let me know if you'd like me to try any tests/patches.
Could you try it with some non-GTK apps (like xedit, nedit) and QT apps (for example licq)? Now that I look at the list of apps that work / don't work I'm under the impression that it might have something to do with GTK. Let's see if that impression is correct :)
Your hunch might be warranted: xedit == no problem tora (only qt app I have) == no problem qtconfig (qt config program - I assume it itself is a qt app) == no problem
Created attachment 29955 [details, diff] A patch to test selection conversion. Could you please compile aterm with the attached patch applied (either add an epatch line to the ebuild, or do: ebuild /usr/portage/x11-terms/aterm/aterm-0.4.2-r9.ebuild unpack; <copy patch to /var/tmp/portage/aterm-0.4.2-r9/work/aterm-0.4.2/ and apply with patch -p1 selection_test.patch, and then: ebuild <path> install, ebuild <path> qmerge) and then check its output when pasting data from GTK apps? it should look similar to this: Supported targets: TIMESTAMP TARGETS MULTIPLE _VIMENC_TEXT _VIM_TEXT UTF8_STRING COMPOUND_TEXT Converting to COMPOUND_TEXT. Could you also try pasting some non-iso-8859-1 characters, smth like: "żołądź" and check if they appear as \u{xxxx} as well?
Blah, it seems bugzilla didn't like these characters :) You can find some here: http://wiem.onet.pl/wiem/00edc2.html.
The patch didn't fix the problem, but here's the output (all copying done from mozilla) : When pasting the word "fran
The patch didn't fix the problem, but here's the output (all copying done from mozilla) : When pasting the word "français" (8 characters) : Supported targets: TIMESTAMP TARGETS MULTIPLE text/html text/_moz_htmlcontext text/_moz_htmlinfo UTF8_STRING COMPOUND_TEXT Converting to COMPOUND_TEXT. Actual pasted output: fran\x{00E7}ais When pasting the word "edycję" (6 characters - copied from http://wiem.onet.pl/wiem/00edc2.html): Supported targets: TIMESTAMP TARGETS MULTIPLE text/html text/_moz_htmlcontext text/_moz_htmlinfo UTF8_STRING COMPOUND_TEXT Converting to COMPOUND_TEXT. Actual pasted output: edycj\x{0119}
I'm not 100% familiar with Unicode, but it seems to me that the first 256 bytes of unicode correspond to iso-8859-1 - for example, the "
I'm not 100% familiar with Unicode, but it seems to me that the first 256 bytes of unicode correspond to iso-8859-1 - for example, the "ç" is 00E7. E7 == 231 decimal which corresponds to the ascii value of "ç" I don't know if the feasibility of this idea or it's cleanliness, but if all the bits in a pasted unicode char minus 8 least significant (right side) are all 0s, maybe strip it off and treat it as simple iso-8859-1 ? Also maybe the libiconv could be of help ? Just my $0.02
You're right about the Unicode, but that is not where the root of the problem lies. The thing is - when a conversion to COMPOUND_TEXT is requested, the resulting string should be encoded in the current locale. The locale being iso8859-1, ascii characters should remain untouched. I'll try to talk with someone with a better knowledge of GTK than mine.
What version of x11-libs/gtk+ are you using?
I've got both 1 and 2 installed: blackcorner root # qpkg -I -v x11-libs/gtk+ x11-libs/gtk+-1.2.10-r10 * x11-libs/gtk+-2.2.4-r1 * Mozilla seems to use version 2.x: blackcorner root # ldd /usr/lib/mozilla/mozilla-bin | grep gtk libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0x45595000) blackcorner root # ls -la /usr/lib/libgtk-x11-2.0.so.0 lrwxrwxrwx 1 root root 25 Nov 7 06:21 /usr/lib/libgtk-x11-2.0.so.0 -> libgtk-x11-2.0.so.0.200.4 blackcorner root # ls -la /usr/lib/libgtk-x11-2.0.so.0.200.4 -rwxr-xr-x 1 root root 2837032 Nov 7 06:21 /usr/lib/libgtk-x11-2.0.so.0.200.4
Would it be possible for you to upgrade to GTK-2.4.0-r1 (at least temporarily) to see if the problems occurs with never versions of GTK? If you decide to do so, you can test everything using the `gtk-demo` program, which is installed with GTK. I'll be away till the 8th of May, so there will probably be no progress on this bug, unless you find a solution or someone else decides to work on it. Please be patient :)
I will try it once I get home and post the result. Also I'm not in a hurry to fix the bug, I fixed it temporarily by removing the offending chunk from the patch file like I described earlier whenever I compile my aterm.
Things have taken a weird turn. Now running: x11-libs/gtk+-2.4.0-r1 Copying the word "fran
Things have taken a weird turn. Now running: x11-libs/gtk+-2.4.0-r1 Copying the word "français" from mozilla and trying to paste in aterm no longer pastes anything ! Normal English words paste fine. Again, yanking out that problematic chunk from files/aterm-0.4.2-copynpaste-r2.patch and re-emerging aterm fixes that problem. (I know this is not the ideal solution, but it's what works for me).
OK, one more thing to check - what about GTK1 apps (xmms is a good example of those)?
Worked a-okay copying the word "fran
Worked a-okay copying the word "français" from the XMMS file selector dialog.
I doubt much has changed, but is it any better with gtk-2.4.3?
I believe I have the same problem with aterm-0.4.2-r10 and gtk+-2.4.4. If I access the following site in firefox [ http://www.phpbuilder.com/annotate/message.php3?id=1018951 ] and paste any of the lines including [ var $_delimiters = " \"'<>;-.!`#+*~
I believe I have the same problem with aterm-0.4.2-r10 and gtk+-2.4.4. If I access the following site in firefox [ http://www.phpbuilder.com/annotate/message.php3?id=1018951 ] and paste any of the lines including [ var $_delimiters = " \"'<>;-.!`#+*~´&\$\\?,:(){}[]%/"; ] and then attempt to paste into a file edited by vim (input mode), nothing gets pasted. If I perform the operation in an xterm or in konsole, the paste works as expected. Note that the character following the ~ (tilde) is outside the plain ascii character set.
Again, I can paste it without any problems :/ Just to make sure it's the same problem as the one described earlier: - does pasting this string into rxvt work? - does pasting this string from a non-GTK app to aterm work? (you can try pasting to an xterm window, and the reselecting it and trying to paste it into aterm; but please make sure that the selection owner is xterm and not mozilla)
1. Same failure pasting from firefox -> rxt vim window 2. no failure pasting from konqueror (qt) -> aterm/rxvt vim window 3. no failure pasting from aterm/rxvt -> aterm/rxvt vim window 4. no failure pasting from firefox -> beaver (gui editor) window Note that beaver isplays the B4 (acute) character properly; vim does not (a line drawing character appears).
OK, this definitely is _the_ bug. Still, I don't know how to solve the problem. Frankly speaking, I don't even know where the problem is. Is it aterm/rxvt that is broken or is it GTK or is it something else? Why does it happen only on some systems (ie. what causes the thing to happen at all)? I'm sorry, but I can't promise any progress on that one in the nearest future.
aterm-1.0 has the same problem... (pasting stuff like
aterm-1.0 has the same problem... (pasting stuff like ç from gtk apps) Michal, did you try to contact upstream? (sasha@aftercode.net) Eterm works fine. Maybe try looking at eterm's code (screen.c i think)
(In reply to comment #30) > aterm-1.0 has the same problem... (pasting stuff like
(In reply to comment #30) > aterm-1.0 has the same problem... (pasting stuff like ç from gtk apps) Indeed, and I believe this is a gtk2 specific bug. > Michal, did you try to contact upstream? (sasha@aftercode.net) Unless I'm mistaken, Sasha has gone through one round of fighting this bug thusfar... but he can't reproduce it, and his tester bailed on him. He's unfortunately on vacation this week, we've had a few complaints about it. I can reproduce the "doesn't paste at all from firefox to aterm 1" right now... so I'll be able to help with that, at least in theory. I hadn't noticed this bug report before, unfortunately, it had become quiet before I had become the upstream liason to gentoo. :D It might be handy for anyone who wants to be involved with fixing this bug to contact me on freenode, in #afterstep.
Fixed and tested in upstream cvs. bigpaste was broken too, and is fixed. will be included in aterm1.1 release, coming soon. We're good to close this bug.
(In reply to comment #42) > Fixed and tested in upstream cvs. bigpaste was broken too, and is fixed. > will be included in aterm1.1 release, coming soon. > > We're good to close this bug. OK, it's been a year. Is there really a 1.1 release coming? Is anybody maintaining aterm? I realize 8 bugs reported in the past 3 years isn't that bad of a record, but they're all still "NEW"...
could someone verify this is fixed in 1.0.1 and close this bug?
I observe apparently the same bug with x11-terms/aterm-1.0.1. I can't copy Russian text (locale ru_RU.KOI8-R) from other terminal window or other x11 application. For example, if I copy text from firefox to aterm window, instead of my text I get something like "\u0440\u0443\u0441\u0441\u043a\u0438\u0439 \u0442\u0435\u043a\u0441\u0442" But coping Russian text from inside the same aterm window works fine as always! x11-terms/aterm-1.0.0 don't have this problem.
(In reply to comment #45) > I observe apparently the same bug with x11-terms/aterm-1.0.1. > I can't copy Russian text (locale ru_RU.KOI8-R) from other terminal window > or other x11 application. > For example, if I copy text from firefox to aterm window, instead of my text > I get something like > "\u0440\u0443\u0441\u0441\u043a\u0438\u0439 \u0442\u0435\u043a\u0441\u0442" > But coping Russian text from inside the same aterm > window works fine as always! > x11-terms/aterm-1.0.0 don't have this problem. > > You might want to retest aterm-1.0.0 i believe you're not pasting high-ascii but unicode. aterm does not support unicode.
(In reply to comment #46) > You might want to retest aterm-1.0.0 i believe you're not pasting > high-ascii > but unicode. aterm does not support unicode. aterm-1.0.0 works fine, actually I'm using it right now. String "\u0440\u0443\u0441\u0441\u043a" makes me think about unicode too, but it can't be the case, because I have UNICODE="no" in /etc/rc.conf and I have even recompiled world with "-unicode" USE flag yesterday, while trying to solve the Russian copy-paste problem...
<x11-terms/aterm-1 no longer in tree. x11-terms/aterm-1.0.1 seems to fix this problem
