Bugzilla Bug 163174

gimp-help-0.11-r1 fails to build during the copying images phase.  This is
quite strange as gimp-help-0.11 builds fine.  The only difference in the
ebuilds is that a few languages are added and a patch is applied.

Reproducible: Always

Steps to Reproduce:
With USE="" and LINGUAS="en":
# emerge gimp-help
Actual Results:  
Output from 'make':
mkdir -p -- stamps
mkdir -p -- html/en
mkdir -p -- pdf/en
*** Creating directories ***
Language: en
mkdir -p -- html/./images/.
...

*** Creating link in PDF directory to images ***
mkdir -p -- xml
-- Profiling XML for en
-- Validating XML for en        number of errors: 0
*** Validating profiled XML ***
Log files of the validation process have been generated in the xml directory.
Please correct any errors before writing new content.
*** Copying images ***
Language: en
make: *** [stamps/images.stamp] Error 1


$ emerge --info
Portage 2.1.1-r2 (default-linux/x86/2006.1/desktop, gcc-4.1.1, glibc-2.4-r4,
2.6.19-gentoo-r4 i686)
=================================================================
System uname: 2.6.19-gentoo-r4 i686 Intel(R) Pentium(R) M processor 1.73GHz
Gentoo Base System version 1.12.6
Last Sync: Mon, 22 Jan 2007 04:50:01 +0000
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: 1.3.7, 2.0.31-r3
dev-lang/python:     2.5-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium-m -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/qmail/alias
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf
/etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo
/etc/texmf/web2c"
CXXFLAGS="-O2 -march=pentium-m -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="ftp://ftp.ussg.iu.edu/pub/linux/gentoo
ftp://gentoo.chem.wisc.edu/gentoo/ ftp://ftp.ndlug.nd.edu/pub/gentoo/
http://prometheus.cs.wmich.edu/gentoo ftp://lug.mtu.edu/gentoo
ftp://gentoo.cites.uiuc.edu/pub/gentoo/
ftp://distro.ibiblio.org/pub/linux/distributions/gentoo/
http://distro.ibiblio.org/pub/linux/distributions/gentoo/"
LINGUAS="en"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="x86 X a52 aac aalib acpi acpm aim alsa alsa_cards_hda-intel
alsa_pcm_plugins_adpcm alsa_pcm_plugins_alaw alsa_pcm_plugins_asym
alsa_pcm_plugins_copy alsa_pcm_plugins_dmix alsa_pcm_plugins_dshare
alsa_pcm_plugins_dsnoop alsa_pcm_plugins_empty alsa_pcm_plugins_extplug
alsa_pcm_plugins_file alsa_pcm_plugins_hooks alsa_pcm_plugins_iec958
alsa_pcm_plugins_ioplug alsa_pcm_plugins_ladspa alsa_pcm_plugins_lfloat
alsa_pcm_plugins_linear alsa_pcm_plugins_meter alsa_pcm_plugins_mulaw
alsa_pcm_plugins_multi alsa_pcm_plugins_null alsa_pcm_plugins_plug
alsa_pcm_plugins_rate alsa_pcm_plugins_route alsa_pcm_plugins_share
alsa_pcm_plugins_shm alsa_pcm_plugins_softvol apache2 apm arts audiofile
bash-completion berkdb bitmap-fonts blas bzip2 cairo cdparanoia cdr cli
cracklib crypt css cups dbus dhcp divx4linux dlloader doc dri dts dv dvd dvdr
dvdread eds elibc_glibc emacs emboss encode esd fam fame fbcon ffmpeg fftw flac
font-server fortran ftp gcj gif ginac glut gmp gnutls gpm gs gstreamer gtk gtk2
hal howl hpn iconv icq ieee1394 imap imlib input_devices_joystick
input_devices_keyboard input_devices_mouse input_devices_synaptics ipv6 isdnlog
ithreads java javascript jce jpeg jpeg2k kde kerberos kernel_linux krb4 lapack
lcd_devices_bayrad lcd_devices_cfontz lcd_devices_cfontz633 lcd_devices_glk
lcd_devices_hd44780 lcd_devices_lb216 lcd_devices_lcdm001 lcd_devices_mtxorb
lcd_devices_ncurses lcd_devices_text ldap libcaca libedit libg++ linguas_en lzo
mad mikmod mime mmx mmx2 mng mp3 mpeg mplayer msn nas ncurses network nfs nls
nntp nptl nptlonly nsplugin ogg oggvorbis openal opengl oss pam pcmcia pcre pdf
perl png posix ppds pppd prelude python qt3 qt4 quicktime rar readline real
reflection rtc ruby samba sasl sdl seamonkey session sockets socks5 spell spl
sse sse2 ssl subtitles svg svga sysvinc tcl tcltk tcpd tetex tga theora threads
tiff tk truetype truetype-fonts type1-fonts udev unicode usb userland_GNU v41
vcd video_cards_nvidia video_cards_vga vim-with-x vorbis wifi win32codecs
wxwindows xine xinerama xinetd xml xorg xpm xprint xscreensaver xv xvid yahoo
zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS,
PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

There's nothing useful in the output you've posted; try w/ MAKEOPTS="-j1" and
post  the output here.

I encounter the same error (only with de included in ALL_LINGUAS as well).

(In reply to comment #1)
> There's nothing useful in the output you've posted; try w/ MAKEOPTS="-j1" and
> post  the output here.

There isn't anything more to see. It's not a parallel build issue. It really is
the stamps/images.stamp target that fails because a command returns 1, and all
the commands for that target are preceded by @ so you won't get the command
from a normal build log. I removed the @ and found it's the large shell loop in
there that is exiting with an error.

This loop comes from gimp-help-2.0.11-image-honour-ALL_LINGUAS.patch, so it's a
Gentooish kind of error. Simple workaround would be to have make ignore those
errors, i.e. replace "@for" by "@-for" in the patch. Even nicer of course would
be some mechanism that causes make to fail when the copy fails and not
otherwise. Right now the loop does not do any error checking of its own, but
neither does the upstream one.

(In reply to comment #2)
> There isn't anything more to see. It's not a parallel build issue.

I wasn't saying it's a parallel make issue, just that the output is pretty
unhelpful :)

(In reply to comment #1)
> There's nothing useful in the output you've posted; try w/ MAKEOPTS="-j1" and
> post  the output here.
> 

I got the same error, however, gimp-help-0.11-r1 also failed to build with
MAKEOPTS="-j1".  portage/app-doc/gimp-help-0.11-r1 /temp/build.log does not
contain any further or different information.

*** Creating link in PDF directory to images ***
mkdir -p -- xml
-- Profiling XML for de
-- Validating XML for de        number of errors: 0
-- Profiling XML for en
-- Validating XML for en        number of errors: 0
*** Validating profiled XML ***
Log files of the validation process have been generated in the xml directory.
Please correct any errors before writing new content.
*** Copying images ***
Language: de
Language: en
make: *** [stamps/images.stamp] Error 1

!!! ERROR: app-doc/gimp-help-0.11-r1 failed.
Call stack:
  ebuild.sh, line 1611:   Called dyn_compile
  ebuild.sh, line 968:   Called qa_call 'src_compile'
  environment, line 3460:   Called src_compile
  gimp-help-0.11-r1.ebuild, line 57:   Called die

!!! emake failed
!!! If you need support, post the topmost build error, and the call stack if
rel
evant.
!!! A complete build log is located at ' /portage/app-doc/gimp-help-0.11-r1
/temp/build.log'.

(In reply to comment #3)
> I wasn't saying it's a parallel make issue,

I know. But your request suggested it might be, and now I say it definitely is
not, so we now know better. :)

> just that the output is pretty unhelpful :)

That's why I hate shell scripts in make files. And that in turn is the main
reason why I did not yet write a better patch. If I did things my way, I'd have
make have the stamp depend on all the installed files, and the installed files
depend on the source files by a simple cp rule. This would be far away from the
upstream approach, which is not nice for patch maintainability.

I just added "set -x;" to the shell script and found out that the "failing"
command is this one here: "test -f './images/using/*.jpg'". Ignoring that one
seems all right.

It also looks like this problem should have been perfectly reproducible, which
makes me wonder if Peter tried the -r1 ebuild before comitting it as a solution
for bug 148880.

Created an attachment (id=107785) [details]
gimp-help-2.0.11-image-honour-ALL_LINGUAS.patch Version 2

This is a replacement for gimp-help-2.0.11-image-honour-ALL_LINGUAS.patch that
does some advanced find magic to choose images. Especially english images are
identified by their depth in the directory tree. I handle examples as another
language here, which has the added benefit that the examples are installed even
when en is not in LINGUAS. The patch might have reduced compatibility to
different find versions, but on Gentoo I'd guess there should be no problem.

Thank you all for report. Fix in CVS. Just wait a an hour and try again.

Martin von Gagern: Thank you that you CC'ing me to this bug. But next time try
to keep closer to topic. Also of course fix was tested and was not applied
UNTIL upstream told me this patch is appropriate to be included in THEIR CVS
also.

Well... Enjoy :)

Martin: Well. The idea behind echo is that for English language we should avoid
subdirectories like de, ru, es inside for IMAGE_DIRS. Thus I just use echo to
list files in predefined image directories. Of course there could be better
solutions if en images were in en subdirectory... 

This find:

find $(top_srcdir)/images/ \
    -wholename "$(top_srcdir)/images/*/*" -a -type d -prune \
    -o \( -name "*.png" -o -name "*.jpg" \) -print

stops working in case somebody creates lang subdirectory in images directory
itself.

Of course better solution is to organize things somehow better in gimp help,
but this should be upstream decision.

Created an attachment (id=107801) [details]
gimp-help-2.0.11-image-honour-ALL_LINGUAS.patch Version 3

(In reply to comment #9)
> This find: [...]
> stops working in case somebody creates lang subdirectory in images directory
> itself.

OK, in case we get language dirs directly inside images, my approach would no
longer work. On the other hand, in case new subdirectories were added to
images, your approach would fail, as upstream doesn't seem to maintain the
IMAGE_DIRS list; that's why your patch had to fix that while mine does not.

To sum it up: both our approaches may break for some likely upstream
modification, and for both there is some fix to solve the problem if that
should happen. For now, both our approaches will work (or can be made to work)
with the current setup. The error handling from my solution could be included
in your solution, given some work. On the other hand, my solution is nearer to
the original code, less lines to diff, and more likely to copy too many files
instead of too few. Therefore I'd prefer my solution to yours.

Another idea I just had is this:

find images -name "*.png" -o -name "*.jpg" | \
egrep -v "/($(grep 'ALL_LINGUAS="' configure | cut -d\" -f2 | tr ' ' '|'))/"

This lists all image files that are not specific to any language. I guess many
of the images that are not in some language subdir are in fact not english but
language neutral. The above statement could list them all, copy as the original
makefile did, and then we'd have a loop over the languages including english,
so we are even ready for when they choose to make subdirs for en as well.

I just rewrote the patch to include this command. Seems to work well enough.

Of course for upstream it would be nicer to have two variables in make, one
with the list of ALL AVAILABLE linguas and one with the list of lunguas TO BE
INSTALLED. Then the first can be used to construct the grep expression (in make
instead of shell) and the second as ALL_LINGUAS is used now to build the loop.
Or of course some better directory layout, say images/en/... images/default/...
and so on. Has somebody already discussed this issue with upstream people?