I kept seeing the message "Binary file $whatever_file matches" during emerging of packages after upgrading to grep 2.13. And after hours of trying I also found out that it was this reporting of normal text files as binary, several packages crashed during compilation, most notably xen 4.1.2, spl 0.6.0_rc9 and zfs 0.6.0_rc9. Reproducible: Always Steps to Reproduce: 1. Have grep 2.13 installed 2. Emerge for instance gcc-4.7.1 Actual Results: Some random messages of "Binary file matches" keep popping up when grep is running during emerge Expected Results: Text files should not be reported as binary files. This emerge --info is from a chroot running the Gentoo LiveDVD 12.0 - kernel sources is 3.4.4. (chroot) Gentoo-2012 ~ # emerge --info Portage 2.2.0_alpha116 (default/linux/amd64/10.0, gcc-4.7.1, glibc-2.15-r2, 3.3.0-gentoo-r1 x86_64) ================================================================= System uname: Linux-3.3.0-gentoo-r1-x86_64-AMD_FX-tm-8150_Eight-Core_Processor-with-gentoo-2.1 Timestamp of tree: Tue, 10 Jul 2012 07:30:02 +0000 app-shells/bash: 4.2_p29 dev-lang/python: 2.7.3-r2, 3.2.3-r1 dev-util/cmake: 2.8.8-r3 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.10.5 sys-apps/sandbox: 2.6 sys-devel/autoconf: 2.69 sys-devel/automake: 1.12.1 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.5.3-r2, 4.7.1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.4 (virtual/os-headers) sys-libs/glibc: 2.15-r2 Repositories: gentoo fem-overlay ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA Oracle-BCLA-JavaSE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=native -mno-avx -mno-sse4.1 -mno-sse4.2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /usr/share/themes/oxygen-gtk/gtk-2.0" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=native -mno-avx -mno-sse4.1 -mno-sse4.2 -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles news parallel-fetch parse-eapi-ebuild-head preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en en_US" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/fem-overlay" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow 3dnowext acl aes-ni amd64 avx bzip2 caps caps-ng cli cracklib crypt cups cxx dri fortran gdbm gpm hvm iconv icu ipv6 lzma mmx mmxext modules mudflap multilib ncurses nls nptl openmp openvswitch pam pcre pppd readline session sse sse2 sse3 sse4 sse4_1 sse4_2 sse4a sse5 ssl ssse3 tcpd threads unicode xattr xen xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_US" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" QEMU_SOFTMMU_TARGETS="x86_64 i386" QEMU_USER_TARGETS="x86_64 i386" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Created attachment 317782 [details] ebuild /usr/portage/sys-devel/gcc/gcc-4.7.1.ebuild unpack --debug You can see an example of this message in this emerge log in line 39332-39335: +++ grep -e '^[[:space:]]*VERSION=' /var/tmp/portage/sys-devel/gcc-4.7.1/work/gcc-4.7.1/ltmain.sh ++ eval Binary file /var/tmp/portage/sys-devel/gcc-4.7.1/work/gcc-4.7.1/ltmain.sh matches +++ Binary file /var/tmp/portage/sys-devel/gcc-4.7.1/work/gcc-4.7.1/ltmain.sh matches /var/tmp/portage/sys-devel/gcc-4.7.1/temp/environment: line 238: Binary: command not found
Workaround is to downgrade to grep-2.12.
Could you attach i.e. that /var/tmp/portage/sys-devel/gcc-4.7.1/work/gcc-4.7.1/ltmain.sh from the *failed* build ? grep -e '^[[:space:]]*VERSION=' /usr/share/libtool/config/ltmain.sh seems to work correctly with grep 2.13.
I think I tested that too, running that command standalone and getting it to work. I think it might be related to environment variables or maybe some weird race condition.... I'll recreate the environment tomorrow to see what happens both standalone and in the build.
Installed grep 2.13 again: # locale LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 LC_NUMERIC=nb_NO.UTF-8 LC_TIME=nb_NO.UTF-8 LC_COLLATE=nb_NO.UTF-8 LC_MONETARY=nb_NO.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=nb_NO.UTF-8 LC_NAME=nb_NO.UTF-8 LC_ADDRESS=nb_NO.UTF-8 LC_TELEPHONE=nb_NO.UTF-8 LC_MEASUREMENT=nb_NO.UTF-8 LC_IDENTIFICATION=nb_NO.UTF-8 LC_ALL= # grep -e '^[[:space:]]*VERSION=' /usr/share/libtool/config/ltmain.sh VERSION=2.4.2 # LC_ALL=C grep -e '^[[:space:]]*VERSION=' /usr/share/libtool/config/ltmain.sh VERSION=2.4.2 ebuild /usr/portage/sys-devel/gcc/gcc-4.7.1.ebuild unpack ... output ... * updating multilib directories to be: ../lib64 ../lib32 * Running elibtoolize in: gcc-4.7.1/ * Applying portage/2.2 patch ... /var/tmp/portage/sys-devel/gcc-4.7.1/temp/environment: line 221: Binary: command not found * Applying sed/1.5.6 patch ... * Applying as-needed/2.2.6 patch ... * Using GNU config files from /usr/share/gnuconfig So it only happens during special circumstances. This is the same error appearing in that long debug-log I pasted earlier.
Any chance it's a filesystem issue ? That is in regard of what filesystem your WORKDIR uses (or perhaps sparse files).
That might be a valid hypothesis, yes.. It happened on a zfs file system. Though, it is consistent that it happens with 2.13 and not with 2.12... Could try it with tmpfs later today and see what happens.
I was bitten by this today too. I have just started to install a server with ZFS and glib was refusing to compile because the version script had been garbled by grep. I put /var/tmp/portage on a tmpfs and all of a sudden it worked again.
So, the change http://git.savannah.gnu.org/cgit/grep.git/commit/?id=582cdfacf297181c2c5ffec83fd8a3c0f6562fc6 seems to be not quite correct for zfs. Does anyone see why ?
Would you try reverting the following commit when building grep? It looks like it might cause this. http://git.savannah.gnu.org/cgit/grep.git/commit/?id=582cdfacf297181c2c5ffec83fd8a3c0f6562fc6(In reply to comment #9) > So, the change > http://git.savannah.gnu.org/cgit/grep.git/commit/ > ?id=582cdfacf297181c2c5ffec83fd8a3c0f6562fc6 seems to be not quite correct > for zfs. > > Does anyone see why ? Have you confirmed that reverting that change makes this problem go away?
I can confirm that reverting that change makes the problem go away. I tried it today and glib that before failed for me due to grep worked without any problems.
Created attachment 318106 [details, diff] Patch that reverts the new behaviour in grep that makes spares file be parsed as binary This patch reverts the commit that added the new behaviour for grep. However the issue seem to lie in how ZFS reports it's metadata or something because as far as I know this issue doesn't come up with other filesystems like ext*, xfs or tmpfs
This is a bug in the ZFS POSIX Layer. I have filed an upstream bug and I am reassigning it to myself, with base-system on CC.
that seems to apply to older btrfs implementations as well, kernel 3.4 works fine, 3.1 does not, didn't test intermediate versions, though.
(In reply to comment #14) > that seems to apply to older btrfs implementations as well, kernel 3.4 works > fine, 3.1 does not, didn't test intermediate versions, though. What are the exact version numbers that you tested?
(In reply to comment #15) > What are the exact version numbers that you tested? 3.1.5-gentoo and 3.4.4-gentoo
(In reply to comment #16) > (In reply to comment #15) > > What are the exact version numbers that you tested? > > 3.1.5-gentoo and 3.4.4-gentoo My time to work on this is currently rather limited. It would help me greatly if someone would test sys-kernel/gentoo-sources-3.2.1-r2 and sys-kernel/gentoo-sources-3.2.21 to see if btrfs is affected there.
The btrfs fix will need to be backported once it is identified, so I am CCing the kernel team on this.
unless i missed something, this isn't a bug in grep ...
*** Bug 428000 has been marked as a duplicate of this bug. ***
(In reply to comment #19) > unless i missed something, this isn't a bug in grep ... This one bites me too. http://lists.gnu.org/archive/html/bug-grep/2012-07/msg00016.html http://git.savannah.gnu.org/cgit/grep.git/commit/?id=2f0255e9f4cc5cc8bd619d1f217902eb29b30bc2 Can we get a backport, please?
I have been hit by this when installing dev-texlive/texlive-fontsextra. My filesystem is btrfs, running on vanilla-sources-3.5.0.
*** Bug 429924 has been marked as a duplicate of this bug. ***
(In reply to comment #22) > I have been hit by this when installing dev-texlive/texlive-fontsextra. > My filesystem is btrfs, running on vanilla-sources-3.5.0. Does the following patch fix it for you? http://git.savannah.gnu.org/cgit/grep.git/commit/?id=2f0255e9f4cc5cc8bd619d1f217902eb29b30bc2
(In reply to comment #24) > (In reply to comment #22) > > I have been hit by this when installing dev-texlive/texlive-fontsextra. > > My filesystem is btrfs, running on vanilla-sources-3.5.0. > > Does the following patch fix it for you? > > http://git.savannah.gnu.org/cgit/grep.git/commit/ > ?id=2f0255e9f4cc5cc8bd619d1f217902eb29b30bc2 I applied the patch on top of grep-2.13, but it didn't fix the problem. grep-2.12 is fine.
(In reply to comment #25) grep-2.14 is in the tree. please verify that fixes things.
(In reply to comment #26) > (In reply to comment #25) > > grep-2.14 is in the tree. please verify that fixes things. Thanks for posting in the bug. I had some fairly high priority things to do offline that caused me to neglect this until now. With that said, ZFSOnLinux upstream and numerous people in IRC have verified that grep 2.14 fixes this issue. That implies that this is a regression in sys-apps/grep-2.13. https://github.com/zfsonlinux/zfs/issues/829 =sys-apps/grep-2.13 is now masked in tree.
