Bugzilla Bug 23116

Many Makefiles use the coreutils 'head' and 'tail'.
In the new version they require that the option '-1' is replaced with '-n 1'.
This breaks lots of builds.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.




I guess it would be nice to have something like a 'sed' line that replaces '-1'
with '-n 1' after 'head' and 'tail' as part of ebuilding a package.

can you list some which are broken?

Broken builds: (I'll add as I come across them):
1. kde-base/kdeadmin-3.1.2

2. dev-util/arch-1.0_pre17

3. dev-lang/squeak-3.4.1
4. kernel 2.5.73-mm1: tail bails twice, but compile finishes and kernel works ;(

5. app-games/enemy-territory-2.55
6. dev-haskell/hmake-3.08

I've looked at coreutils info page - there are different posix standards for
many commands.  
Try setting:  
export _POSIX2_VERSION=199209  

Wow, that solves the problem, perhaps it should be set as default in coreutils
for now.

just to note another one is blackdown-jdk which prevents mod_php and other
biggies.
doesnt seem to happen with gcc-2.x only gcc-3.x

Seemant, what is the status on this (interest sake)?

Az, as far as I know, John is working on patching the offending makefiles

7. kde-base/arts-1.1.3

Created an attachment (id=15882) [details]
fixheadtails

This is a quick bash script to replace all those evil occurances of head -X and
tail -X with newer POSIX compliant ones.
This should probably be in distfiles and ran from offending ebuilds, I am ok to
do this.

Anyone have any objections to this method of working?

Created an attachment (id=15883) [details]
fixheadtails

Created an attachment (id=15886) [details]
list of offending files in gentoo-x86 (some can be safely ignored)

just to update you, I have written an eclass and am now testing it with other
ebuilds to see how we get on.
I suspect this should be in good working order sometime after work, when i get
chance to sit down properly with it.

Let's just add Enemy Territory to the list of stuff that needs fixing. 

actually enemy-territory isnt broken, the unpack_makeself in eutils.eclass is
... i'll fix it shortly

and let's add Arch as well...

sun-jdk ebuilds appear in the list of files offending but theyre working ok, i 
have tested with coreutils-5.0-r1,r3. 
 
Any ideas on this ?? 

fixheadtails is now in eclasses.
usage is explained within that although basic usage is:

inherit fixheadtails

src_unpack() {
unpack ${A}
cd ${S}
ht_fix_all
}

i have fixed hmake already, i have problems compiling squeak anyways so i will notify the relevant maintainers.
can people please update this bug with any other packages needing fixing, and i shall do so. i will work on those i am aware of now.

You are going to miss calls like 'tail +16c foo'.  From the info pages:

----------------------------------------------------------------
   On older systems, `tail' supports an obsolete option
`-COUNTOPTIONS', which is recognized only if it is specified first.
COUNT is a decimal number optionally followed by a size letter (`b',
`k', `m') as in `-c', or `l' to mean count by lines, or other option
letters (`cfqv').  Some older `tail' implementations also support an
obsolete option `+COUNT' with the same meaning as `-+COUNT'.  POSIX
1003.1-2001 (*note Standards conformance::) does not allow these
options; use `-c COUNT' or `-n COUNT' instead.
----------------------------------------------------------------

That was for tail.  head is pretty much the same, except it only support
'cqv' and not 'cfqv'.

Here is my scripts (ok not optimal in speed ;), which do handle these
cases - hope it helps:

----------------------------------------------------------------
azarah@nosferatu glibc $ cat /usr/src/scripts/fix_head.sh
#!/bin/bash
 
fix_head_arg="$(readlink -f "$1" 2>/dev/null)"
 
source /sbin/functions.sh
 
x=
count=0
 
[ -z "${fix_head_arg}" ] && exit 1
[ ! -d "${fix_head_arg}" ] && exit 1
 
einfo "Scanning ${fix_head_arg} ..."
for x in $(find ${fix_head_arg}/ -type f -exec \
           grep -l -e 'head[[:space:]]\+-[0-9]' -- {} \;)
do
        [ "${count}" -eq 0 ] && eend 0
 
        x_relative="$(echo "${x}" | sed -e "s|^${fix_head_arg}||")"
        einfo " Fixing $(basename "${fix_head_arg}")${x_relative} ..."
        sed -i -e 's|head[[:space:]]\+-\([0-9]\+[bkm]\?\)\([cqv]\?*\)|head -\2 \1|g' \
               -e 's|head -\([^nc ]*\) \([+0-9]\+\)|head -n\1 \2|g' -- "${x}"
        eend $?
 
        [ "${count}" -eq 0 ] && x=1
done
 
[ "${count}" -eq 0 ] && eend 0
 
azarah@nosferatu glibc $ cat /usr/src/scripts/fix_tail.sh
#!/bin/bash
 
fix_tail_arg="$(readlink -f "$1" 2>/dev/null)"
 
source /sbin/functions.sh
 
x=
count=0
 
[ -z "${fix_tail_arg}" ] && exit 1
[ ! -d "${fix_tail_arg}" ] && exit 1
 
einfo "Scanning ${fix_tail_arg} ..."
for x in $(find ${fix_tail_arg}/ -type f -exec \
           grep -l -e 'tail[[:space:]]\+[-+][0-9]' -- {} \;)
do
        [ "${count}" -eq 0 ] && eend 0
 
        x_relative="$(echo "${x}" | sed -e "s|^${fix_tail_arg}||")"
        einfo " Fixing $(basename "${fix_tail_arg}")${x_relative} ..."
        sed -i -e 's|tail[[:space:]]\+\([-+][0-9]\+[bkm]\?\)\([cfqv]\?*\)|tail -\2 \1|g' \
               -e 's|tail -\([^nc ]*\) -\?\([+0-9]\+\)|tail -n\1 \2|g' -- "${x}"
        eend $?
 
        [ "${count}" -eq 0 ] && x=1
done
 
[ "${count}" -eq 0 ] && eend 0
 
azarah@nosferatu glibc $

Ok, paste was not good, here is the 'sed' parts again:

----------------------------fix_head.sh-----------------------------
sed -i -e 's|head[[:space:]]\+-\([0-9]\+[bkm]\?\)\([cqv]\?*\)|head -\2 \1|g' \
       -e 's|head -\([^nc ]*\) \([+0-9]\+\)|head -n\1 \2|g' -- "${x}"
--------------------------------------------------------------------

----------------------------fix_tail.sh-----------------------------
sed -i -e 's|tail[[:space:]]\+\([-+][0-9]\+[bkm]\?\)\([cfqv]\?*\)|tail -\2 \1|g' \
       -e 's|tail -\([^nc ]*\) -\?\([+0-9]\+\)|tail -n\1 \2|g' -- "${x}"
--------------------------------------------------------------------

add dev-utils/arch-1.0_pre17 to the list of broken stuff 

add also app-text/ispell-3.2.06-r6 :)

Another offending Makefile is in the 2.4.20 kernel (sys-kernel/gentoo-sources).
It doesn't break the kernel itself but prevents it generating the compile.h
file correctly, which breaks other ebuilds (so far the alsa-driver ebuild):

@echo \#define LINUX_COMPILER \"`$(CC) $(CFLAGS) -v 2>&1 | tail -1`\" >>
.ver

(line 351)

*** Bug 31871 has been marked as a duplicate of this bug. ***

this will also be expanded to handle sort/chown and other issues addressed
in future.
Working on a sort issue as we speak but i dont expect a commit until tomorrow.

Changes in the chown user.group syntax cause errors in
/etc/cron.daily/syslog.cron:      
/bin/chown root:adm $LOG   does work correctly.
However, the default syntax is /bin/chown root.adm $LOG.

*** Bug 32582 has been marked as a duplicate of this bug. ***

actually, looks like avenj fixed sysklogd's cron.daily script a while ago, so a
remerge should sort you out.

media-sound/abcde emerges, but cddb/freedb lookups are broken until I set the
_POSIX2_VERSION as indicated earlier.

There is also a problem on the alpha platform, where the (binary) compaq-jdk
uses "head -1" in its java wrapper. Setting _POSIX2_VERSION seemed to help.

alpha-team see comment #32

Is it possible to fix this bug directly during compile of coureutils?
Not only ebuilds are broken, even installers (eg: install-crossover-office-2.0.1.sh).

kde_base/arts-1.1.2

Add to that a substantial amount of KDE. Newer versions with variations of the head fix are marked stable for almost every architecture. That doesn't help me when I'm running an Alpha.

I fixed compaq-jre and compaq-jdk, and since I'm in the midst of working on
kde-3.1.4 for alpha-stable, Decade's comment should be moot soon. :-)

this issue isn't resent in debian
maybe we could patch head and tail to accept the old syntax

s/resent/represent

5.0.91-r3 and -r4 are patched already, actually

and how about the coreutils-5.0 ?
how about patching it too and remove all old ebuilds without patch

No, I'm rather pushing 5.0.91 more than 5.0, as I stated recently on one of the
mailing lists, I'd like to get -r4 to stable in the next 2 or 3 weeks. At that
point, I'll remove 5.0 from the tree entirely

coreutils-5.0.91-r4.ebuild is now marked stable on alpha and ia64

Broken builds:
All qmail-ebuilds

coreutils-5.0.91-r4 is stable on x86

but those syntax fixes should still go in

The -n switch also destroys all the sun-jdks as of now.

In each sun-jdk, SUN uses tail +345 and tail -1

tail -1 works, and the file gives a nice warning, but the script tailes terribly. And against the gentoo script, nothing works. This may be an error that should end up at the dev-java guys.

I've fixed it, with manually hacking the downloaded sun-jdks, and replaced the "tail +345" with "tail --lines=+345". This fixes one problem, but there still i some sync problems against the scripts from the gentoo developers. I guess this should end up in a patch though.

Forgot to mention

tail -1 file (works, but gives a warning that might break scripts, like the sun-jdks)
tail +1 file (doesn't work at all anymore, but works with "tail --lines=+1 file"). This breaks alot more than - that just gives a warning right now.

And yes, I will stop bombing you soon :) My testes was on amd64 with both
5.0.91-r4 and 5.2.0. Guess the same problem is in x86 though..

This is insane.  Other distros have all been smart enough to leave the default
behaviour of supporting the old style standard -123 way of telling head and
tail what to do without a stupid warning message to stderr.  *PLEASE* make it
the default so people will stop laughing at you and scripts will stop breaking.

head -123
tail -123
tail +321

those should all work forever on into the future.  its the way they have always
worked.
anything else is incompatible with the real world.

actually, before you start/continue anymore flames, realize the simple fact,
it's a bug

as David noted in comment #47, tail -# works, tail +# does not work

the compat patches are incomplete and no developers ever really noticed ...
i'll take a look at what i missed later

Bug 66713 handles the `tail +#` syntax fix

I'm going to assume this is now being handled correctly and the bug can be
closed.