Bug 125288 - Broken encoding in ChangeLog
Bug#: 125288 Product:  Gentoo Linux Version: unspecified Platform: All
OS/Version: Linux Status: RESOLVED Severity: normal Priority: P1
Resolution: INVALID Assigned To: tgall@gentoo.org Reported By: tove@gentoo.org
Component: Ebuilds
URL: 
Summary: Broken encoding in ChangeLog
Keywords:  
Status Whiteboard: 
Opened: 2006-03-06 12:49 0000
Description:   Opened: 2006-03-06 12:49 0000
Your commit broke the encoding of these documents:
http://www.gentoo.org/cgi-bin/viewcvs.cgi/app-admin/skey/ChangeLog?r1=1.36&r2=1.37
http://www.gentoo.org/cgi-bin/viewcvs.cgi/sys-fs/cryptsetup/ChangeLog?r1=1.30&r2=1.31
http://www.gentoo.org/cgi-bin/viewcvs.cgi/sys-fs/e2fsprogs/ChangeLog?r1=1.79&r2=1.80

If you want i can fix it.

------- Comment #1 From Torsten Veller 2006-03-07 13:03:34 0000 -------
I've fixed it in the tree.

If you need help with your system, someone to test,..., don't hesitate and ask.
Thanks.

------- Comment #2 From Torsten Veller 2006-06-25 12:25:23 0000 -------
Again:
http://sources.gentoo.org/viewcvs.py/gentoo-x86/sci-geosciences/gpsd/ChangeLog?r1=1.14&r2=1.15

------- Comment #3 From Matthias Schwarzott 2006-08-31 04:04:51 0000 -------
fixed changelog of sci-geosciences/gpsd

------- Comment #4 From Torsten Veller 2006-09-12 11:59:33 0000 -------
Tom, your commits are constantly breaking the encoding of the ChangeLog, if you
need help to fix this problem please ask.
I think it's unacceptable to deface other contributor's names.

------- Comment #5 From Jakub Moc (RETIRED) 2006-12-03 06:49:02 0000 -------
No idea why is this still open.

------- Comment #6 From Torsten Veller 2007-01-01 15:58:37 0000 -------
(In reply to comment #5)
> No idea why is this still open.

It is not fixed:
http://sources.gentoo.org/viewcvs.py/gentoo-x86/mail-client/mozilla-thunderbird/ChangeLog?r1=1.153&r2=1.154
http://sources.gentoo.org/viewcvs.py/gentoo-x86/x11-plugins/enigmail/ChangeLog?r1=1.42&r2=1.43

Tom, i know it only happens sometimes. Maybe it helps you to find the reason
for it.

------- Comment #7 From Jakub Moc (RETIRED) 2007-01-02 07:29:49 0000 -------
(In reply to comment #6)
> It is not fixed:

So make repoman detect those garbled logs?

------- Comment #8 From Torsten Veller 2007-01-03 02:58:36 0000 -------
A new one:
http://sources.gentoo.org/viewcvs.py/gentoo-x86/www-client/mozilla-firefox/ChangeLog?r1=1.232&r2=1.233

Tom Gall, you don't care, do you?



(In reply to comment #7)
> So make repoman detect those garbled logs?

Have an idea how to do it?

------- Comment #9 From Torsten Veller 2007-01-06 19:30:43 0000 -------
http://sources.gentoo.org/viewcvs.py/gentoo-x86/www-client/mozilla-firefox/ChangeLog?r1=1.237&r2=1.238
http://sources.gentoo.org/viewcvs.py/gentoo-x86/app-portage/ufed/ChangeLog?r1=1.54&r2=1.55

------- Comment #10 From Jakub Moc (RETIRED) 2007-01-21 19:43:42 0000 -------
CCing portage folks, repoman really should check for this...

@tgall - could you damn post some response on this bug finally?

------- Comment #11 From Marius Mauch (RETIRED) 2007-01-22 11:50:52 0000 -------
(In reply to comment #10)
> CCing portage folks, repoman really should check for this...

Sure, let us know how.

------- Comment #12 From Jakub Moc (RETIRED) 2007-03-09 21:40:48 0000 -------
tgall - are you breaking this on purpose or WTH? Could you kindly respond here
finally?! This is getting absurd...

<snip>
21:29:28 <+CIA-1> tgall * gentoo-x86/dev-ruby/fusefs/ (ChangeLog
fusefs-0.6.0.ebuild): 
21:29:28 <+CIA-1> added ~ppc64                                                  
21:29:28 <+CIA-1> (Portage version: 2.1.2-r9)                                   
21:35:10 <+CIA-1> tove * gentoo-x86/dev-ruby/fusefs/ChangeLog:                  
21:35:10 <+CIA-1> fixed broken encoding in ChangeLog                     
</snip>

------- Comment #13 From Jakub Moc (RETIRED) 2007-03-10 23:04:02 0000 -------
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/facets/ChangeLog?r1=1.12&r2=1.13
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/facets/ChangeLog?r1=1.12&r2=1.13
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/dpklib/ChangeLog?r1=1.5&r2=1.6
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/devel-logger/ChangeLog?r1=1.10&r2=1.11

etc. etc. etc.

------- Comment #14 From Jakub Moc (RETIRED) 2007-03-15 10:32:27 0000 -------
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/sqlite-ruby/ChangeLog?r1=1.29&r2=1.30
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/misen/ChangeLog?r1=1.4&r2=1.5
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/ncurses-ruby/ChangeLog?r1=1.21&r2=1.22
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/nora/ChangeLog?r1=1.8&r2=1.9
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/ruby-fcgi/ChangeLog?r1=1.18&r2=1.19
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/ruby-xslt/ChangeLog?r1=1.7&r2=1.8
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/narray/ChangeLog?r1=1.27&r2=1.28
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/needle/ChangeLog?r1=1.14&r2=1.15
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/mysql-ruby/ChangeLog?r1=1.32&r2=1.33
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/nano-methods/ChangeLog?r1=1.3&r2=1.4
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/og/ChangeLog?r1=1.10&r2=1.11
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/libxml/ChangeLog?r1=1.10&r2=1.11
http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-ruby/nimage/ChangeLog?r1=1.2&r2=1.3

------- Comment #15 From Tom Gall 2007-03-15 13:50:05 0000 -------
then as now, the system I'm using is a ppc64 box, vim 7.0.174. There's really
nothing magical about the system at all. 

I'm not breaking anything on purpose. 

> vim foo-whatever.ebuild

usual vim commands here ending with :w, :x

> repoman scan
> repoman commit

Has anything thought about putting together a test data file with DB data in it
and testing it with vim on ppc64?

pop into #gentoo-ppc64 if you'd like to check out that route.

adding ppc64@gentoo.org on the CC as should be the case for all ppc64 related
bugs. 

------- Comment #16 From Chris Gianelloni (RETIRED) 2007-03-15 14:27:28 0000 -------
Are you using "echangelog" for your ChangeLog entries?  If not, why not?  That
might solve the issue for you, as echangelog is UTF8-aware.

------- Comment #17 From Tom Gall 2007-03-15 16:44:35 0000 -------
No never have. 

Shouldn't matter. vim *SHOULD* be utf-8 aware. Did a UTF8 bug get missed in
vim?

------- Comment #18 From Ciaran McCreesh 2007-03-15 17:18:22 0000 -------
Vim is only utf-8 aware if you tell it to be. Vim's documentation goes into
this in great depth.

------- Comment #19 From Tom Gall 2007-03-15 21:00:00 0000 -------
given that bit of info, shouldn't
http://devmanual.gentoo.org/appendices/editor-configuration/vim/index.html be
updated to include the gory details like has been done for emacs at : 

http://devmanual.gentoo.org/appendices/editor-configuration/emacs/index.html ?

Have a deeper link into the vim documentation Ciaran?

Thanks

------- Comment #20 From Ciaran McCreesh 2007-03-15 21:12:33 0000 -------
Probably, if anyone's maintaining the devmanual these days. Having said that,
you're supposed to use echangelog, not vim.

------- Comment #21 From Brent Baude 2007-03-15 21:25:18 0000 -------
Tom, good point here to use echangelog.  echangelog seems to use nano but for
what it is worth, if the variable EDITOR is set to /usr/bin/vim then echangelog
will invoke vim.  And we're back to Tom's problem. 

Tom do you know what to fix now?

------- Comment #22 From Chris Gianelloni (RETIRED) 2007-03-16 17:02:08 0000 -------
I was thinking more like:

echangelog "This is my ChangeLog message.  There are many like it, but this one
is mine."

;]

------- Comment #23 From Paul Varner 2007-03-26 22:16:47 0000 -------
I just hit this myself. When editing a Changelog, vim shows:
fileencoding=utf-8

Any changes I make will cause the UTF-8 encoding to be broken.  However, if I
manually set "fileencoding=latin1" then it doesn't break. This doesn't seem to
be inline with vim's documentation and I find it to be confusing.

------- Comment #24 From Ciaran McCreesh 2007-03-26 22:21:00 0000 -------
fileencoding in vim contains what the file is *now*, not what it was when it
was opened. If vim can't figure out the file encoding on its own, you need to
open a file using the ++enc option.

------- Comment #25 From Tom Gall 2007-04-01 02:58:31 0000 -------
Getting back to the original bug (as I'm back from a business trip and plenty
of real life crap)

so are you saying Ciaran that perhaps vim's ability to detect the encoding of
the file might be buggy or the files are mislabeled? 

Guess I'd like to understand a bit more how vim does this encoding detection
magic. 

------- Comment #26 From Ciaran McCreesh 2007-04-01 13:51:41 0000 -------
Vim's detection isn't buggy. It's just that it's impossible to determine
encoding automatically in many situations because most documents are valid in
most encodings. Like I said, Vim's documentation explains all.

------- Comment #27 From Tom Gall 2007-04-01 21:54:53 0000 -------
adding the vim team onto the cc. 

The little bit about GLEP31 and gentoo-syntax changes to utf-8 being the
default seems to be the interesting bit of trivia.

In review: 

1) vim and ppc64 and amd64 seems to have issues when the ebuild/changelog have
utf8 data in them. 

2) Comment 23 from Paul seems to suggest that fileencoding=latin1 avoid the
problem. (And I've confirmed on ppc64)

I'm going to rebuild vim on my box with the ignore-glep31 USE flag and see if
that has any effect. 

Given that this is affecting two gentoo architectures, seems like the ownership
of this bug probably need to fall into the hands of those that implemented
glep31 or the vim team.

------- Comment #28 From Tom Gall 2007-04-01 21:59:53 0000 -------
rebuilding gentoo-synxtax with ignore-glep31 fixes this bug.

------- Comment #29 From Ciaran McCreesh 2007-04-01 22:02:09 0000 -------
This has nothing to do with architecture, and using latin1 will lead to broken
display and broken files. This is user error and nothing more.

------- Comment #30 From Paul Varner 2007-05-22 21:17:11 0000 -------
How do we get this fixed? In order for me to not break the ChangeLog, I have to
do a set fileencoding=latin1. I have read through the documentation and I can
not find how I have vim misconfigured that doing the above causes it to work
correctly.

------- Comment #31 From Ciaran McCreesh 2007-05-22 21:22:18 0000 -------
You need to make sure that fileencoding is set to utf-8 *before* you open up
the ChangeLog file. The easiest way to do this is using one of the ++enc
options, which are described at length in the documentation.

------- Comment #32 From Jakub Moc (RETIRED) 2007-07-12 13:37:53 0000 -------
Uh. Seriously, a developer unable to set up $EDITOR properly is not a bug;
closing this, as in "read the fine manual". :P

------- Comment #33 From Mark Loeser 2008-01-18 02:04:43 0000 -------
Ignore me...

------- Comment #34 From Mark Loeser 2008-01-18 02:05:55 0000 -------
Nothing to see here...just fixing the resolution (if invalid could be called a
fix...its definitely not a "remind" though :) )