While trying to emerge dev-libs/icu-4.2.1 get an error generating out/tmp/icudata.lst (list of data files) LD_LIBRARY_PATH=../stubdata:../tools/ctestfw:../lib:$LD_LIBRARY_PATH MAKEFLAGS= ../bin/pkgdata -O ../data/icupkg.inc -q -c -s /home/tmp/portage/dev-libs/icu-4.2.1/work/icu/source/data/out/build/icudt42l -d ../lib -e icudt42 -T ./out/tmp -p icudt42l -L sicudata -m static ./out/tmp/icudata.lst pkgdata: i686-pc-linux-gnu-gcc -D_REENTRANT -O3 -march=nocona -pipe -fpeel-loops -finline-functions -ftracer -ffinite-math-only -fpeephole2 -fno-thread-jumps -fstrength-reduce -fregmove -freorder-blocks -fschedule-insns2 -fdelete-null-pointer-checks -fcaller-saves -frerun-cse-after-loop -fcse-skip-blocks -fno-trapping-math -fsched-spec -falign-functions -fno-defer-pop -fno-if-conversion -mno-push-args -fno-merge-constants -Wall -ansi -pedantic -Wshadow -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -Wno-long-long -c-I../common -I../common -DPIC -fPIC -I../common -I../common -DPIC -fPIC -o ./out/tmp/icudt42l_dat.o ./out/tmp/icudt42l_dat.s i686-pc-linux-gnu-gcc: unrecognized option '-c-I../common' /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/../../../crt1.o: In function `_start': (.text+0x18): undefined reference to `main' collect2: ld returned 1 exit status -- return status = 256 Error generating assembly code for data. make[1]: *** [packagedata] Segmentation fault make[1]: Leaving directory `/home/tmp/portage/dev-libs/icu-4.2.1/work/icu/source/data' make: *** [all-recursive] Error 2 * * ERROR: dev-libs/icu-4.2.1 failed. * Call stack: * ebuild.sh, line 49: Called src_compile * environment, line 2479: Called _eapi2_src_compile * ebuild.sh, line 634: Called die * The specific snippet of code: * emake || die "emake failed" * The die message: * emake failed * * If you need support, post the topmost build error, and the call stack if relevant. * A complete build log is located at '/home/tmp/portage/dev-libs/icu-4.2.1/temp/build.log'. * The ebuild environment file is located at '/home/tmp/portage/dev-libs/icu-4.2.1/temp/environment'. * >>> Failed to emerge dev-libs/icu-4.2.1, Log file: >>> '/home/tmp/portage/dev-libs/icu-4.2.1/temp/build.log' Reproducible: Always Steps to Reproduce: 1. emerge icu 2. get an error uname -a Linux tyr 2.6.23-gentoo-r3 #1 SMP Fri Dec 21 16:17:49 EET 2007 i686 Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz GenuineIntel GNU/Linux emerge --info Portage 2.1.6.13 (default/linux/x86/2008.0, gcc-4.1.2, glibc-2.9_p20081201-r2, 2.6.23-gentoo-r3 i686) ================================================================= System uname: Linux-2.6.23-gentoo-r3-i686-Intel-R-_Core-TM-2_Duo_CPU_E6550_@_2.33GHz-with-gentoo-1.12.11.1 Timestamp of tree: Sun, 20 Sep 2009 13:45:01 +0000 ccache version 2.4 [enabled] app-shells/bash: 4.0_p28 dev-lang/python: 2.5.4-r3, 2.6.2-r1 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.7.9-r1, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O3 -march=nocona -pipe -fpeel-loops -finline-functions -ftracer -ffinite-math-only -fpeephole2 -fno-thread-jumps -fstrength-reduce -fregmove -freorder-blocks -fschedule-insns2 -fdelete-null-pointer-checks -fcaller-saves -frerun-cse-after-loop -fcse-skip-blocks -fno-trapping-math -fsched-spec -falign-functions -fno-defer-pop -fno-if-conversion -mno-push-args -fno-merge-constants" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/X11/xkb /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d" CXXFLAGS="-O2 -march=i686 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" LDFLAGS="-Wl,-O1" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/storage/SYS/overlay" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="acl apache2 berkdb bzip2 cli cracklib crypt cups dri extensions fortran gd gdbm gpm iconv ipv4 isdnlog jpeg mbox mmx mudflap ncurses nls nptl nptlonly ntpl ntplonly openmp pam pcre perl png pppd python readline reflection session slang spl sse ssl svg sysfs tcpd unicode x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 204721 [details] build.log of icu-4.2.1.
Use CFLAGS="-O3 -march=nocona -pipe".
(In reply to comment #2) > Use CFLAGS="-O3 -march=nocona -pipe". I meant CFLAGS="-O2 -march=nocona -pipe".
I can CONFIRM the bug to be perfectly VALID. It's not caused by any particular CFLAG (or combination of such) but just by generating a buffer overflow in the program pkgdata. A quick and simple fix (which I would recommend to add to the portage tree immediatelly) would be to --- icu/source/tools/pkgdata/pkgdata.cpp +++ icu/source/tools/pkgdata/pkgdata.cpp @@ -103,1 +103,1 @@ -#define SMALL_BUFFER_MAX_SIZE 512 +#define SMALL_BUFFER_MAX_SIZE 2048 which should give enough room for even very long command lines. The true and clean fix would be to get rid of SMALL_BUFFER_MAX_SIZE (and any other #define like that too) all together, but that would be a job for upstream, because it will require a redesign of (hopfully) internal API. BTW: Just tested the change - now ico can be emerged without any problems. @arfrever: After illegally resolving this bug as invalid while it was perfectly VALID, I leave it up to you to inform upstream and push them to fix their code. Regards, Bodo
(In reply to comment #4) There wasn't any proof that this bug is valid. Please report it to upstream.
*** Bug 318011 has been marked as a duplicate of this bug. ***
(In reply to comment #6) > *** Bug 318011 has been marked as a duplicate of this bug. *** > I was typing a response to Bug 318011 until I received an email saying that it had been marked invalid. Here is what I was going to say: > The problem is at pkgdata.cpp:488: > > ... > #define SMALL_BUFFER_MAX_SIZE 512 > ... > static int runCommand(const char* command, UBool specialHandling) { > char cmd[SMALL_BUFFER_MAX_SIZE]; > ... > } else { > normal_command_mode: > sprintf(cmd, "%s", command); // line 488 > } > ... > > When the supplied command line ("x86_64-pc-linux-gnu-gcc -D_REENTRANT -O2 > -pipe ...") is too long, buffer overflow occurs. > > (I have to say, it surprises me that sprintf/strcat is used everywhere in the > code, without any bounds checking) > The file extension implies that it is written in C++, but the code appears to be entirely written in C. If this was C++ code like the file extension had suggested, I would say that they should have used the string and stringstream classes for this, but since it is C, they probably should use something like this: int strncat_fixed ( char * dest, char * src, size_t size ) { return strncat(dest, src, size - strlen(dest) - 1); } In any case, I have filed a ticket with upstream to inform them of this problem: http://bugs.icu-project.org/trac/ticket/7749 Until they fix it, the file could be patched to increase the small and large buffer sizes by a factor 8, which should workaround this for people until upstream hopefully rewrites their code to not rely on fixed-sized buffers for this kind of data in the first place. By the way, if the rest of ICU is written like this and the code interacts with stuff from the internet, which I imagine it does, we could be looking at a much more profound problem, because using these functions without proper bounds checking in a major system component exposed to the internet is a hacker's dream come true.
Richard, It would be helpful to us to add such comments to the upstream bug system so that fixes can be made there. We strive to be a responsive (and responsible) project so that things don't need to be 'hacked around' downstream. We do run static and dynamic analysis especially on the core ICU libraries, this aspect in the tool was missed- thank you for finding it. Steven R. Loomis, ICU for C/C++ Technical Lead (In reply to comment #7) > .... > In any case, I have filed a ticket with upstream to inform them of this > problem: > > http://bugs.icu-project.org/trac/ticket/7749 > > Until they fix it, the file could be patched to increase the small and large > buffer sizes by a factor 8, which should workaround this for people until > upstream hopefully rewrites their code to not rely on fixed-sized buffers for > this kind of data in the first place. > > By the way, if the rest of ICU is written like this and the code interacts with > stuff from the internet, which I imagine it does, we could be looking at a much > more profound problem, because using these functions without proper bounds > checking in a major system component exposed to the internet is a hacker's > dream come true. >
It seems that the issue is fixed in 4.5.2 and the latest version in portage is 4.4.1. Is it possible to backport the upstream patch to the older versions of icu?
Fixed in dev-libs/icu-4.4.1.
