I'm not perfectly sure if this is an upstream bug or a Gentoo specific problem, but: Since I upgraded to Lilypond 2.19.80-r1, it doesn't handle non-ASCII characters correctly anymore. Lilypond uses UTF-8, and up to version 2.18.2-r3, everything was fine: If I wrote e. g. German Umlauts in the sources, they were displayed as such. With version 2.19.80-r1, I get a "?" character in my output instead of the non-ASCII character. Unfortunately, I can't downgrade Lilypond anymore, because the older version pulls <=dev-scheme/guile-2, which collides with my GnuCash installation, but as said: as long as I used the older version, everything was fine. I also get "?" characters on the console when using a non-C locale, e. g.: Interpretation der Musik... Vorverarbeitung der grafischen Elemente... Musik wird auf eine Seite angepasst... Systeme erstellen... Layout nach »/tmp/lilypond-25RULQ« ausgeben... Konvertierung nach ??1. Alt-Saxophon in Es.pdf??... L??schen von ??/tmp/lilypond-25RULQ??... Kompilation erfolgreich beendet Any idea how to track this down?
I just tried to install the binary version from http://lilypond.org/unix.de.html manually ("GNU/Linux 64: LilyPond 2.18.2-1"). I can run it (apparently, there's no guile conflict with this package) and I don't get output errors. I also don't get those double ?? in the console output: Analysieren... Interpretation der Musik... Vorverarbeitung der grafischen Elemente... Musik wird auf eine Seite angepasst... Systeme erstellen... Layout nach »1. Alt-Saxophon in Es.ps« ausgeben... Konvertierung nach »./1. Alt-Saxophon in Es.pdf«... Kompilation erfolgreich beendet So probably, something goes wrong compiling the newer version on Gentoo?!
Apparently, guile 2 is the problem. This (German) forum thread https://lilypondforum.de/index.php/topic,355.0.html links to a GNU mailing list post: http://lists.gnu.org/archive/html/bug-lilypond/2018-06/msg00011.html which includes a patchset to make Lilypond somewhat work with guile 2. I could apply all those patches, with the exception of patch #13. Building it with this patchset does at least solve the Locale problem, I get my special chars again. But apparently, Lilypond isn't meant to be used with guile 2 at all yet! So I think it would be meaningful to remove the guile2 USE flag by default. This would lead to a slot conflict problem, at least for people also using GnuCash (like me), which blocks guile <2. So perhaps, one should enable slots for guile, so that one can have an <2 version in parallel to v2? After all, I don't want/need to use the guile2 USE flag, but I can't build lilypond without as long as I have GnuCash installed. Would be nice if this could be solved! Thanks in advance for all efforts!
(In reply to Tobias Leupold from comment #2) > This (German) forum thread https://lilypondforum.de/index.php/topic,355.0.html links to a GNU mailing list post: http://lists.gnu.org/archive/html/bug-lilypond/2018-06/msg00011.html which includes a patchset to make Lilypond somewhat work with guile 2. Those are the relevant patches originally from the "guile-v2-work" branch in the lilypond git repository. > I could apply all those patches, with the exception of patch #13. That one doesn't apply because it does the same as the existing Gentoo patch lilypond-2.19.80-remove-ly-protect.patch > Would be nice if this could be solved! Thanks in advance for all efforts! Seconded, and thanks to you for finding that patchset. Extracting those in userpatches and removing #13 did result in a lilypond binary that processes at least the utf-8 lyrics correctly.
I just got additional upstream information from the lilypond forum. I'll cite (translated): > Jouni Kosonen is only partially right: The patches 13-15 from my link are not > part of this branch. I already merged some reworked patches from the "guile-v2- > work" branch to master. The "guile-v2-work" branch should be updated soonish > [...] > > Apparently, Gentoo already addressed the problem from patch-13 themselves. > patch-14 only turns off unneeded warnings. > But patch-15 ist important if one uses file names with special characters. I can confirm lilypond also not to handle non-ASCII characters in filenames ("File not found" or whatever) without patch #15.
(In reply to Tobias Leupold from comment #2) > But apparently, Lilypond isn't meant to be used with guile 2 at all yet! So > I think it would be meaningful to remove the guile2 USE flag by default. I just saw that actually, guile2 is disabled by default (it really should be!). But, as said, I could not update my system anymore without it due to GnuCash blocking depending on >=dev-scheme/guile-2.0.0:12 and dev-scheme/guile-2.0.14-r3 does not allow to be dev-scheme/guile-1.8.8-r3 to be installed at the same time. So probably, we should split this to two bugs? One: "Essential fixes for Lilypond with USE=guile2 set" and "Please change slotting for dev-scheme/guile"?
I see this bug is still marked as "unconfirmed"; I'm experiencing this exact problem some months later. (I probably was at the time, but I didn't have non-ASCII characters in my input.)
Current situation (at least at here) is, that either lilypond is compiled with USE="-guile2" or lilypond won't handle non-ASCII characters at all, both in text handling and it's own (diagnostic or progress) output. So it seems to at least partially be related to some change in guile. Would it be possible to find (and probably fix) the occansins where that happens? Unfortunately I'm far from being a Scheme/Lisp expert...
Created attachment 574910 [details, diff] Patch for lilypond-guil This is a patch I cherry picked from patches upstream last year when I encountered this bug. I really don't recount how it works specifically, it was more or less try and error but it does the job for me. Also, it's reasonable short so if you're fluent you'll hopefully be able to grasp the essence.
Was supposed to be "Patch for lilypond/guile2 with German umlauts"
Works for me (and applies without problems), thanks a lot!
There are comments from 2018 that some patches are already merged. Another says that guile2 is disabled by default, and another to compile lilypond with USE=-guile2. The latter is actually not possible (anymore): USE=-guile2 emerge -pv lilypond [ebuild R ] media-sound/lilypond-2.19.80-r1::gentoo USE="(guile2) ..." If there is a working patch from at least May, could this please be put into portage at some point (or is that rather an upstream issue)? Greetings.
I've had these in userpatches when I build with USE="emacs guile2" since Sep -18: 0001-Initialize-locale-in-GUILEv2.patch 0002-Source_file-init_port-Keep-GUILEv2-from-redecoding-s.patch 0003-Update-changes-from-commit-122525f-Keep-GUILEv2-from.patch 0004-XXX-Avoid-the-lockup-in-ly_scm_write_string.patch 0005-Fix-the-encoding-of-the-postscript-output.patch 0006-Fix-wide-characters-support-in-embedded-SCM-with-gui.patch 0007-Use-the-UTF-8-encoding-for-SVG-files.patch 0008-Fix-formatting-floating-point-variables-with-guile-2.patch 0009-Fix-ly_scm2string-to-work-with-guile-2.0.patch 0010-XXX-add-support-for-itexi-files-to-the-vim-config.patch 0011-XXX-don-t-override-LANG-globally-in-the-build-proces.patch 0012-XXX-reset-the-locale-when-building-index.html.patch 0014-Suggestions-by-Arne.patch 0015-Initialize-locale-in-lily.scm.patch They are exactly the 15 patches referred to in comment 2, less patch #13 that Gentoo already handles.
can you please test with 2.19.83?
Same problems with 2.19.83
can you please test whether the issue still exists in 2.21.6? and if it is still there, whether the attached patch still applies and fixes the issue?
For me, -2.21.6 seems to work without the need for patching.
(In reply to hangglider from comment #16) > For me, -2.21.6 doesn't work without the need for patching (First test case was wrong, so I incorrectly reported it to work before).
Problem not fixed: $ lilypond àéè.ly GNU LilyPond 2.21.6 warning: cannot find file: `??????.ly' fatal error: failed files: "??????.ly"
Same problem. Patch didn't apply here to lilypond-2.21.6, trying to manually patch files didn't work well either and result is the same.
