Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 676886 - www-client/seamonkey-2.49.x will not build if PORTAGE_TMPDIR has even moderately long path
Summary: www-client/seamonkey-2.49.x will not build if PORTAGE_TMPDIR has even moderat...
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Lars Wendler (Polynomial-C)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-01-30 02:54 UTC by segmentation fault
Modified: 2019-01-31 17:45 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description segmentation fault 2019-01-30 02:54:42 UTC
This is more a bug about portage, than about Seamonkey. It is also not clear if it is a bug or a feature request - it depends on how you view it. Feel free to change its type to 'enhancement request' and forward it to the eclass developers. What is clear, however, is that it is very subtle - it took me half a day to track down. And it is also an example of how a miniscule thing (the length of a variable in some header) can kill a behemoth like a browser...In any case, I will provide solutions below to accommodate all tastes. :-)


Problem
=======

After changing some USE flags, I wanted to re-merge Seamonkey. I did not expect any errors, since I had rebuilt that same version more than once a few weeks ago - and nothing (well, almost nothing...) else had changed since (with the exception of a few new kernels). And yet, compilation ended with the error:

js/src> /bin/sh: ./config.status: /some/very/very/very/very/long/path/portage/www-client/seamonkey-2.49.9.1_p0/work/thunderbird-52.9.1/seamonk/_virtualenv/bin/p: bad interpreter: No such file or directory

(In the above, /some/very/very/very/very/long/path is just my PORTAGE_TMPDIR).

It took me some time to realize that the problem was *this* line and not the point at which config.log ended - which was quite a few steps later (no matter whether I had '-j1' or '-j8' in MAKEOPTS):

DEBUG: configure:19144: checking for posix_fallocate
DEBUG: configure:19176: /usr/x86_64-pc-linux-gnu/gcc-bin/7.3.0/x86_64-pc-linux-gnu-gcc -std=gnu99 -o conftest -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -fno-lifetime-dse -march=native -pipe -mno-avx -fno-strict-aliasing -fno-math-errno -pthread -pipe -march=native -pipe -lpthread -Wl,-O1 -Wl,--as-needed -Wl,-rpath=/usr/lib64/seamonkey,--enable-new-dtags -Wl,-z,noexecstack -Wl,-z,text conftest.c -ldl  1>&5
DEBUG: configure:19227: /usr/x86_64-pc-linux-gnu/gcc-bin/7.3.0/x86_64-pc-linux-gnu-gcc -std=gnu99 -c -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare -Wtype-limits -Wunreachable-code -Wno-error=maybe-uninitialized -Wno-error=deprecated-declarations -Wno-error=array-bounds -fno-lifetime-dse -march=native -pipe -mno-avx -fno-strict-aliasing -fno-math-errno -pthread -pipe  -I/usr/include -march=native -pipe conftest.c 1>&5
ERROR: old-configure failed
*** Fix above errors and then restart with\
               "make -f client.mk build"
make: *** [client.mk:362: configure] Error 1
 ^[[31;01m*^[[0m ERROR: www-client/seamonkey-2.49.9.1_p0::gentoo failed (configure phase):
 ^[[31;01m*^[[0m   emake failed

In fact, "checking for posix_fallocate" has absolutely nothing to do with it! The reason for this error is something totally unexpected:


Reason
======

One of the few things that changed since the last compilation was this: I had set my PORTAGE_TMPDIR to be a moderately deep directory (instead of the really short one I had in place previously):

PORTAGE_TMPDIR='/some/very/very/very/very/long/path'

(Actually the directory I used has a path that is 34 characters long, not 35 as in the example here - in any case, I find this to be only moderately long).

Now everything is built under

/some/very/very/very/very/long/path

The file

/some/very/very/very/very/long/path/portage/www-client/seamonkey-2.49.9.1_p0/work/thunderbird-52.9.1/seamonk/js/src/config.status

has a shebang that is too long (longer than 128 characters):

#!/some/very/very/very/very/long/path/portage/www-client/seamonkey-2.49.9.1_p0/work/thunderbird-52.9.1/seamonk/_virtualenv/bin/python

so that I get the error:

js/src> /bin/sh: ./config.status: /some/very/very/very/very/long/path/portage/www-client/seamonkey-2.49.9.1_p0/work/thunderbird-52.9.1/seamonk/_virtualenv/bin/p: bad interpreter: No such file or directory

The reason the shebang is truncated in the above error message is NOT that it was truncated by some length limitation in the debug message itself ( as I was assuming for quite some time), but because Linux itself allocates only 128 characters for the shebang of any executable! See Solution 1.


Solution 1
==========

The 128 character limit is set in the kernel in:

/usr/src/linux/include/uapi/linux/binfmts.h

and you can also find it in:

/usr/include/linux/binfmts.h

You can see it in action in 

https://github.com/torvalds/linux/blob/master/fs/exec.c

in prepare_binprm:

/*
 * Fill the binprm structure from the inode.
 * Check permissions, then read the first 128 (BINPRM_BUF_SIZE) bytes
 *
 * This may be called multiple times for binary chains (scripts for example).
 */
int prepare_binprm(struct linux_binprm *bprm)
{
	int retval;
	loff_t pos = 0;

	bprm_fill_uid(bprm);

	/* fill in binprm security blob */
	retval = security_bprm_set_creds(bprm);
	if (retval)
		return retval;
	bprm->called_set_creds = 1;

	memset(bprm->buf, 0, BINPRM_BUF_SIZE);
	return kernel_read(bprm->file, bprm->buf, BINPRM_BUF_SIZE, &pos);
}

It really reads 128 bytes, no more! 

Change it (in /usr/src/linux/include/uapi/linux/binfmts.h) to a higher value, one that can accomodate your longest expected shebang, recompile your kernel and reboot.

A discussion on the pros and cons of such a change is in

https://www.reddit.com/r/linux/comments/4u3m1b/linux_change_request/

You will have to do this each time you recompile your kernel.


Workaround 1
============

This is something that the user can do to alleviate the problem.

Use a shorter path for PORTAGE_TMPDIR when building Mozilla projects.

You don't need to do this for all your projects, just for the ones that require very long shebangs:

Create

/etc/portage/env/short-shebangs.conf

containing a redefinition of PORTAGE_TMPDIR to some short path of your choice.

# Change PORTAGE_TMPDIR to a directory with a short path,
# to avoid shebangs in build scripts that are longer than 128 characters.
PORTAGE_TMPDIR='/a/short/path'

Then create

/etc/portage/package.env/seamonkey

containing

www-client/seamonkey    short-shebangs.conf

Now the Mozilla package should build without shebang problems.


Workaround 2
============

This is something that the maintainer of the Gentoo Seamonkey (and Firefox?) package can do:

Incorporate a pre-build check in the ebuild that checks the length of $PORTAGE_TMPDIR and, if it exceeds, say, 30, aborts with a message that asks the user to change PORTAGE_TMPDIR to a shorter path.


Workaround 3
============

This is something Gentoo developers can do:

Modify the eclass to check the length of the shebangs of all scripts found below $PORTAGE_TMPDIR/category/package-name, directly after completion of the extraction phase. Find the maximum path length of the shebangs, MAX_SHB_LEN, i.e. the maximum length of the lines that start with "#!" on top of executables, minus 2 to account for "#!" (O.K:, you have to account for whitespace too...).  You can see how Linus does it above. There is probably some (kernel) function that you can copy or call to get it 100% right, as the shell does. Subtract this MAX_SHB_LEN from $BINPRM_BUF_SIZE, as you find it in either 

/usr/src/linux/include/uapi/linux/binfmts.h

or

/usr/include/linux/binfmts.h

If the difference is negative, warn the user automatically and abort - the package maintainer does not have to do anything.
Comment 1 segmentation fault 2019-01-31 17:45:38 UTC
There is a 4th workaround, which may be interesting for Gentoo developers to include in some eclass:

Workaround 4
============

Write a wrapper script, wrapper.sh:

#!/bin/bash
script="$1" 
shebang=$(head -1 "$script")
interp=( ${shebang#\#!} )        # use an array in case a argument is there too
# now run it
exec "${interp[@]}" "$script"

and run any bash script script.sh inside PORTAGE_TMPDIR as

wrapper.sh script.sh

Found at:

https://stackoverflow.com/questions/10813538/shebang-line-limit-in-bash-and-linux-kernel

No kernel recompilations, warnings or aborts necessary with this - even better!