*** t6026-merge-attr.sh *** * ok 1: setup * ok 2: merge * ok 3: check merge result in index * ok 4: check merge result in working tree * ok 5: custom merge backend * FAIL 6: custom merge backend git reset --hard anchor && git config --replace-all \ merge.custom.driver "./custom-merge %O %A %B 1" && git config --replace-all \ merge.custom.name "custom merge driver for testing" && if git merge master then echo "Eh? should have conflicted" false else echo "Ok, conflicted" fi && cmp binary union && sed -e 1,3d text >check-1 && o=$(git-unpack-file master^:text) && a=$(git-unpack-file anchor:text) && b=$(git-unpack-file master:text) && sh -c "./custom-merge $o $a $b 0" && sed -e 1,3d $a >check-2 && cmp check-1 check-2 && rm -f $o $a $b * failed 1 among 6 test(s) make[1]: *** [t6026-merge-attr.sh] Error 1 make[1]: Leaving directory `/mnt/alt/portage-tmp/portage/dev-util/git-1.5.2.5/work/git-1.5.2.5/t' make: *** [test] Error 2 Tested with these USE flag configurations (failing at the same error message): [ebuild U ] dev-util/git-1.5.2.5 [1.5.1.6] USE="bash-completion curl doc emacs gtk mozsha1 perl tk webdav (-ppcsha1)" 0 kB [ebuild U ] dev-util/git-1.5.2.5 [1.5.1.6] USE="-bash-completion* -curl* -doc* -emacs* -gtk* -mozsha1* -perl* (-ppcsha1) -tk* -webdav*" 0 kB
Let's try to get more info about that failure, try this src_test: --- 8< --- src_test() { cd "${S}" emake ${MY_MAKEOPTS} DESTDIR="${D}" GIT_TEST_OPTS="--verbose --debug" -C t t6026-merge-attr.sh || die "test failed" } --- 8< --- That will give us more information about what failed in that && chain. - ferdy
* ok 5: custom merge backend * expecting success: git reset --hard anchor && git config --replace-all \ merge.custom.driver "./custom-merge %O %A %B 1" && git config --replace-all \ merge.custom.name "custom merge driver for testing" && if git merge master then echo "Eh? should have conflicted" false else echo "Ok, conflicted" fi && cmp binary union && sed -e 1,3d text >check-1 && o=$(git-unpack-file master^:text) && a=$(git-unpack-file anchor:text) && b=$(git-unpack-file master:text) && sh -c "./custom-merge $o $a $b 0" && sed -e 1,3d $a >check-2 && cmp check-1 check-2 && rm -f $o $a $b HEAD is now at 03d8861... Side Merging HEAD with master Merging: 03d8861 Side 5054610 Master found 1 common ancestor(s): 5f96ff8 Initial Auto-merged binary Auto-merged text merging text using custom merge driver for testing fatal: Failed to execute internal merge Merge with strategy recursive failed. Ok, conflicted cmp: EOF on union * FAIL 6: custom merge backend git reset --hard anchor && git config --replace-all \ merge.custom.driver "./custom-merge %O %A %B 1" && git config --replace-all \ merge.custom.name "custom merge driver for testing" && if git merge master then echo "Eh? should have conflicted" false else echo "Ok, conflicted" fi && cmp binary union && sed -e 1,3d text >check-1 && o=$(git-unpack-file master^:text) && a=$(git-unpack-file anchor:text) && b=$(git-unpack-file master:text) && sh -c "./custom-merge $o $a $b 0" && sed -e 1,3d $a >check-2 && cmp check-1 check-2 && rm -f $o $a $b * failed 1 among 6 test(s) make: *** [t6026-merge-attr.sh] Error 1
Easy to reproduce, too: jeroen@elmer /dev/shm/portage/dev-util/git-1.5.2.5/work/git-1.5.2.5/t/trash $ cmp binary union cmp: EOF on union
In that same directory, there should be a custom-driver script. Can you paste it here along with its current permissions ? For some reason, it is not executing it properly. This is how it looks like here: --- 8< --- [ $ ~/git/git-master/t(master) ] cat trash/custom-merge #!/bin/sh orig="$1" ours="$2" theirs="$3" exit="$4" ( echo "orig is $orig" echo "ours is $ours" echo "theirs is $theirs" echo "=== orig ===" cat "$orig" echo "=== ours ===" cat "$ours" echo "=== theirs ===" cat "$theirs" ) >"$ours+" cat "$ours+" >"$ours" rm -f "$ours+" exit "$exit" [ $ ~/git/git-master/t(master) ] ls -l trash/custom-merge -rwxr-xr-x 1 ferdy users 293 2007-09-20 19:52 trash/custom-merge [ $ ~/git/git-master/t(master) ] ls -l /bin/sh lrwxrwxrwx 1 root root 4 2007-08-08 18:19 /bin/sh -> bash [ $ ~/git/git-master/t(master) ] --- 8< --- - ferdy
jeroen@elmer /dev/shm/portage/dev-util/git-1.5.2.5/work/git-1.5.2.5/t/trash $ ls -l total 28 -rw-r--r-- 1 jeroen users 20 2007-09-20 19:22 binary -rw-r--r-- 1 jeroen users 13 2007-09-20 19:22 binary-orig -rw-r--r-- 1 jeroen users 77 2007-09-20 19:22 check-1 -rw-r--r-- 1 jeroen users 77 2007-09-20 19:22 check-2 -rwxr-xr-x 1 jeroen users 293 2007-09-20 19:22 custom-merge -rw-r--r-- 1 jeroen users 13 2007-09-20 19:22 text -rw-r--r-- 1 jeroen users 13 2007-09-20 19:22 union jeroen@elmer /dev/shm/portage/dev-util/git-1.5.2.5/work/git-1.5.2.5/t/trash $ cat custom-merge #!/bin/sh orig="$1" ours="$2" theirs="$3" exit="$4" ( echo "orig is $orig" echo "ours is $ours" echo "theirs is $theirs" echo "=== orig ===" cat "$orig" echo "=== ours ===" cat "$ours" echo "=== theirs ===" cat "$theirs" ) >"$ours+" cat "$ours+" >"$ours" rm -f "$ours+" exit "$exit" jeroen@elmer /dev/shm/portage/dev-util/git-1.5.2.5/work/git-1.5.2.5/t/trash $ ls -l /bin/sh lrwxrwxrwx 1 root root 4 2007-07-22 16:18 /bin/sh -> bash
@ferdy - While I am at it... Could you explain what this means? It doesn't appear to be proper English. 's|with|from|'? dev-util/git:mozsha1 - Make use of a bundled routine commit with Mozilla that should be fast on non-x86 archs
That doesn't look like English to me either. That USE flag makes git use an optimized SHA1 routine from Mozilla that should be fast on non-x86 machines. - ferdy PS: I'm still trying to figure out what's going on with that test. Unfortunately, I'm not able to reproduce the error, so I have to make guesses. You should probably try instrumenting merge-recursive.c to see what codepath is the testcase hitting. (Or run it under gdb, whatever you are more comfortable with).
(In reply to comment #7) > That doesn't look like English to me either. That USE flag makes git use an > optimized SHA1 routine from Mozilla that should be fast on non-x86 machines. Thanks. Fixed that in CVS.
Btw, this is the only test in the suite that fails.
Did you find something 'interesting' in your instrumentation/debugging of merge-recursive.c ? I still can't figure out what might be going wrong. - ferdy
hppa: please retest with 1.5.3.7-r1, that i'm filing a stablereq for shortly.
(In reply to comment #11) > hppa: please retest with 1.5.3.7-r1, that i'm filing a stablereq for shortly. The very same test fails because nothing has been done yet.
jer: please run t6026-merge-attr.sh, and then follow my command examples here: (ls -l trash/{binary,union} ; cmp trash/{binary,union} ; echo $? ; xxd <trash/binary ; trash/union) This works on x86, amd64, ppc64-32ul in my testing. root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # ls -l trash/{binary,union} -rw-r--r-- 1 root root 20 2007-12-25 02:20 trash/binary -rw-r--r-- 1 root root 20 2007-12-25 02:20 trash/union root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # cmp trash/{binary,union} root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # echo $? 0 root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # xxd <trash/{binary,union} -bash: trash/{binary,union}: ambiguous redirect root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # xxd <trash/binary 0000000: 496e 6974 6961 6c0a 5369 6465 0a4d 6173 Initial.Side.Mas 0000010: 7465 720a ter. root@grubbs-int:/dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # xxd <trash/union 0000000: 496e 6974 6961 6c0a 5369 6465 0a4d 6173 Initial.Side.Mas 0000010: 7465 720a ter.
elmer /dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t # ls -l trash/{binary,union} ; cmp trash/{binary,union} ; echo $? ; xxd <trash/binary ; xxd <trash/union -rw-r--r-- 1 root root 20 2007-12-25 15:18 trash/binary -rw-r--r-- 1 root root 13 2007-12-25 15:18 trash/union cmp: EOF on trash/union 1 0000000: 496e 6974 6961 6c0a 5369 6465 0a4d 6173 Initial.Side.Mas 0000010: 7465 720a ter. 0000000: 496e 6974 6961 6c0a 5369 6465 0a Initial.Side. elmer /dev/shm/portage/dev-util/git-1.5.3.7-r1/work/git-1.5.3.7/t #
Created attachment 139288 [details] diff -ur of t/trash/.git directories between x86 and hppa Maybe this sheds light on where in the process git goes wrong.
Created attachment 139323 [details, diff] 20071225-t6026-amd64-x86.diff Attached is my comparison between amd64/x86 for the same thing. In yours, which side was x86 and hppa? Besides t6026, do any other tests fail? You may have to remove t6026 to see. On your HPPA machine, grab the last line of trash/.git/logs/HEAD, and run 'git show COMMITSH' for the ID that is NOT 03d886186d9cafce1df533a8b9b579c8a71a1acf. 918bdc850518cc71129dd394256290af19d3d527 and 8365483bb24ae05801aa7d18ec7074503be8e07e are the COMMITSH values in your patch data above, not sure which one was the hppa box. Please post the output of the 'git show COMMITSH >file' as an attachment. If you compare against x86 (I will too), there should be difference in the 'text' file, as well as the first 'commit' line, and the 'index' line for the 'text' file. However, I expect your HPPA box to show at least one other change. Either there's a lurking bug that is specific to HPPA, or your glibc is doing something weird compared to other architectures.
jer: the HPPA kernel has a bug. I googled for other occurances of the bug, and found it at debian. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=427907 Which leads to: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=427398 They have a kernel patch there, but also note that the HPPA kernel was getting rid of the buggy code at some point in the first place. So please test with a newer kernel, or patch the kernel yourself.
Built with -O0, git passes 6026 fine (no other tests fail, btw). I don't use hppa-sources at this moment so I cannot test the validity of that kernel bug - I can see that the patch has not been applied in the latest hppa-sources, though, so I will patch that, rebuild and see if we can do without -O0 that way. Wed Dec 26 07:01:11 CET 2007 Portage 2.1.4_rc11 (default-linux/hppa/2007.0, gcc-4.1.2, glibc-2.7-r1, 2.6.23-gentoo-r5-JeR parisc) ================================================================= System uname: 2.6.23-gentoo-r5-JeR parisc PA8700 (PCX-W2) Timestamp of tree: Wed, 26 Dec 2007 05:16:01 +0000 distcc 2.18.3 hppa2.0-unknown-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [disabled] app-shells/bash: 3.2_p17 dev-lang/python: 2.4.4-r6 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 sys-apps/baselayout: 2.0.0_rc6 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r1 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10 sys-devel/binutils: 2.18-r1 sys-devel/gcc-config: 1.3.16 sys-devel/libtool: 1.5.24 virtual/os-headers: 2.6.23-r2 ACCEPT_KEYWORDS="hppa" CBUILD="hppa2.0-unknown-linux-gnu" CFLAGS="-O2 -pipe -mschedule=8000 -march=2.0 -ggdb -Wall" CHOST="hppa2.0-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config /var/bind /var/spool/torque /var/www/localhost/htdocs/wordpress/wp-config.php" CONFIG_PROTECT_MASK="/etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -mschedule=8000 -march=2.0 -ggdb -Wall" DISTDIR="/keeps/gentoo/distfiles" FEATURES="autoaddcvs buildpkg cvs distlocks fixpackages notitles sandbox sfperms splitdebug strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://mirror.muntinternet.net/pub/gentoo/ http://gentoo.tiscali.nl/" LC_ALL="en_US.UTF-8" LINGUAS="en nl he" MAKEOPTS="-j4" PKGDIR="/keeps/gentoo/packages/elmer" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*" PORTAGE_TMPDIR="/mnt/alt/portage-tmp" PORTDIR="/keeps/gentoo/portage" PORTDIR_OVERLAY="/keeps/gentoo/local" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="7zip X Xaw3d a52 aac aalib accessibility ads alsa amr amrnb amrwb ao aoss apache2 ares arts asf async audiofile automount avfs bash-completion berkdb bidi bitmap-fonts bittorrent bl bzip2 c++ cairo caps catalogs cdb cddb cdparanoia cdr chardet cjk cli cpudetection cracklib crypt cups curl custom-cflags dbtool dbus device-mapper dga dia directfb djbfft domainkeys dts dv dvd dvdr dvdread dxr3 edl elf emacs enca encode esd examples exif expat fam fame fastbuild fastcgi fbcon ffmpeg filter flac fontconfig foomaticdb fortran ftp gadu gd gdbm geoip ggi gif gimp gimpprint glep glib glut gmp gnome gnutls gphoto2 gpm gs gsl gtk gtk2 gtkhtml hal hesiod hppa ical icecast iconv idea idn imagemagick imlib immqt-bc inquisitio ipv6 isdnlog jack javascript jingle jpeg jpeg2k kde kerberos lapack lcms ldap leim libcaca libnotify libsamplerate libwww logrotate logwatch lua lzo mad matroska memcache mhash midi mikmod mmap mng modplug motif mozbranding mp3 mssql mudflap musepack mysql nas ncurses netpbm network-cron nfconntrack nfs nls nntp nptl nptlonly nsplugin offensive ogg openexr opengl openmp oss ots overlays pam pango pbs pch pcre pdf pdo-external perl php pic plotutils plugins png portage portaudio postgres povray ppds pppd pulseaudio python pyzord qdbm qt3 qt3support quotas raw readline recode reflection rpc rrdtool rtc ruby samba sasl scanner scim sdl seamonkey session sid slang slp sndfile snmp soundex speex spell spl sqlite ssl startup-notification suhosin svg swat sysfs talkfilters tcl tcpd tga theora threads thunar-vfs tidy tiff timidity tk tools truetype truetype-fonts twolame type1-fonts udev unicode unzip urandom usb userlocales utempter utf v4l v4l2 vanim vcd vidix vim-syntax vorbis wavpack webdav webinstall winbind wlan wma wmf xanim xchattext xcomposite xface xml xml2 xmpi xorg xpm xrandr xscreensaver xsettings xulrunner xv xvid xvmc zip zip-external zlib" ALSA_CARDS="ad1889 usb-audio" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en nl he" USERLAND="GNU" VIDEO_CARDS="stifb fbdev matrox" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LDFLAGS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
The kernel bug mentioned in comment #17 is unrelated. It's a compiler optimisation problem.
Hmm, so for HPPA, should we force -O0 for now? For all of Git, or just for merge-recursive.c? Here's a another Debian bug for GCC messing up Git. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=426302
(In reply to comment #20) > Hmm, so for HPPA, should we force -O0 for now? For all of Git, or just for > merge-recursive.c? I'd rather like to try to solve the problem. As git-1.5.1.6 doesn't exhibit the problem of this bug, and git-1.5.2.5 does, I diffed the two and found that the MSB macro (which appears to find the most significant bit) was added between those versions: diff -ur git-1.5.1.6/work/git-1.5.1.6/git-compat-util.h git-1.5.2.5/work/git-1.5.2.5/git-compat-util.h --- git-1.5.1.6/work/git-1.5.1.6/git-compat-util.h 2007-05-20 10:21:54.000000000 +0200 +++ git-1.5.2.5/work/git-1.5.2.5/git-compat-util.h 2007-08-16 00:57:15.000000000 +0200 @@ -13,6 +13,14 @@ #define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0])) +#ifdef __GNUC__ +#define TYPEOF(x) (__typeof__(x)) +#else +#define TYPEOF(x) +#endif + +#define MSB(x, bits) ((x) & TYPEOF(x)(~0ULL << (sizeof(x) * 8 - (bits)))) + /* Approximation of the length of the decimal representation of this type. */ #define decimal_length(x) ((int)(sizeof(x) * 2.56 + 0.5) + 1) This macro is used in a few files: jeroen@astrid /var/tmp/portage/dev-util $ grep -lr MSB git-1.5.2.5/work/git-1.5.2.5/ [...] git-1.5.2.5/work/git-1.5.2.5/builtin-pack-objects.c git-1.5.2.5/work/git-1.5.2.5/builtin-unpack-objects.c [...] git-1.5.2.5/work/git-1.5.2.5/index-pack.c git-1.5.2.5/work/git-1.5.2.5/sha1_file.c
From the other debian bug, they think it's this code in merge-recursive.c: status = run_command(&child); if (status < -ERR_RUN_COMMAND_FORK) ; /* failure in run-command */ else status = -status; Where the empty ; section is, can you replace it with: { fprintf(stderr, "Failed to run llmerge cmdbuf=%s\n", cmdbuf); } and see if the error persists while a non-zero -O level is used?
Ok, I did try the replacement running on the hake.hppa.dev box, and confirm that it makes the problem seem to vanish. I don't know HPPA asm, and it seems nobody was home in #gentoo-hppa. I did some interesting testing however. I applied test.patch (attachment here). And compared the output on x86/amd64/ppc vs. the hppa box. amd64/ppc/x86 output: """"""" merging text using custom merge driver for testing 00 status=-1 01 status=1 CONFLICT (content): Merge conflict in text """"""" hppa output: """"""" merging text using custom merge driver for testing 00 status=-1 01 status=-1 fatal: Failed to execute internal merge """"""" So GCC on HPPA is emitting ASM that takes the wrong branch, or it's emitting code that fucks up the delay slot usage.
Created attachment 139480 [details, diff] Add debugging output right around the broken if statement.
Here is the smallest GCC testcase I could come up with that exhibits the issue: int ret1() { return -1 }; int main(int argc, char **argv) { int i = ret1(); if(i >= -10000) { i = -i; } return i+10; } And I used it as follows: for i in 0 1 2 3 ; do hppa2.0-unknown-linux-gnu-gcc -o test2 -save-temps -Wall test2.c -O${i} && cp test2.s test2-O${i}.s && ./test2 ; echo $? ; done ; 11 9 9 11 '11' is the correct value. -O3 gives the correct value only because it converts the entire program to 'return 11';
Since this looks like a miscompilation, the best we can do in git is forcing -O0 on hppa. Patching that particular chunk to workaround a gcc bug looks like asking for trouble somewhere else. - ferdy
There is still no revision bump or other mechanism to ensure that an appropriately patched gcc is used to build dev-util/git.
toolchain: any progress on your side? Is there some version spec we can depend on to make sure we have a good gcc?
The current gcc has been fixed half a year ago and dev-util/git-1.5.6.4 is now stable for HPPA.
Just to make it clear, the fix for bug 193134 hasn't been integrated in 4.1.2 but in 4.2.4. And unfortunately, 4.2.4 exhibit other breakage :-/
