Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 604760 - sys-apps/sed-4.3 - chmod: cannot access 'doc/sed.1-t': No such file or directory
Summary: sys-apps/sed-4.3 - chmod: cannot access 'doc/sed.1-t': No such file or directory
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL: http://lists.gnu.org/archive/html/bug...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-01-05 15:08 UTC by Martin Mokrejš
Modified: 2017-01-06 17:12 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,78.35 KB, text/plain)
2017-01-05 16:11 UTC, Martin Mokrejš
Details
config.log (config.log,359.97 KB, text/plain)
2017-01-05 16:13 UTC, Martin Mokrejš
Details
config.status (config.status,74.28 KB, text/plain)
2017-01-05 16:14 UTC, Martin Mokrejš
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Mokrejš 2017-01-05 15:08:55 UTC
Hi,
  while building Gentoo::RAP on a RedHat host system I run into an issue with ${WORKDIR}/sed-4.3/build-aux/help2man , somehow expanding crap.



rm -f lib/libsed.a
x86_64-pc-linux-gnu-ar cr lib/libsed.a lib/copy-acl.o lib/set-acl.o lib/acl-errno-valid.o lib/acl-internal.o lib/get-permissions.o lib/set-permissions.o lib/c-ctype.o lib/c-strcasecmp.o lib/c-strncasecmp.o lib/close-stream.o lib/closeout.o lib/dfa.o lib/localeinfo.o lib/dirname-lgpl.o lib/basename-lgpl.o lib/stripslash.o lib/exitfail.o lib/getprogname.o lib/hard-locale.o lib/localcharset.o lib/glthread/lock.o lib/malloca.o lib/progname.o lib/qcopy-acl.o lib/qset-acl.o lib/quotearg.o lib/se-context.o lib/se-selinux.o lib/tempname.o lib/glthread/threadlib.o lib/unistd.o lib/version-etc.o lib/version-etc-fsf.o lib/wctype-h.o lib/xmalloc.o lib/xalloc-die.o lib/mbrlen.o lib/mbrtowc.o lib/obstack.o 
x86_64-pc-linux-gnu-ranlib lib/libsed.a
x86_64-pc-linux-gnu-gcc    -O2 -pipe -O2 -pipe  -L/apps/gentoo/usr/lib64 -o sed/sed sed/sed_sed-compile.o sed/sed_sed-execute.o sed/sed_sed-mbcs.o sed/sed_sed-regexp.o sed/sed_sed-sed.o sed/sed_sed-utils.o sed/libver.a lib/libsed.a    
/apps/gentoo/bin/mkdir -p doc
rm -rf doc/sed.1 doc/sed.1-t
./build-aux/help2man                                            \
    --name 'stream editor for filtering and transforming text'  \
    -p sed --include ./doc/sed.x                        \
    -o doc/sed.1-t sed/sed                                              \
  && chmod a-w doc/sed.1-t                                              \
  && mv doc/sed.1-t doc/sed.1
apiversion=9999
chmod: cannot access 'doc/sed.1-t': No such file or directory
make[2]: *** [Makefile:5776: doc/sed.1] Error 1
make[2]: Leaving directory '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'
make[1]: *** [Makefile:3025: all-recursive] Error 1
make[1]: Leaving directory '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'
make: *** [Makefile:2157: all] Error 2
 * ERROR: sys-apps/sed-4.3::gentoo failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=sys-apps/sed-4.3::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-apps/sed-4.3::gentoo'`.
 * The complete build log is located at '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/build.log'.
 * The ebuild environment file is located at '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/environment'.
 * Working directory: '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'
 * S: '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'

>>> Failed to emerge sys-apps/sed-4.3, Log file:

>>>  '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/build.log'

 * Messages for package sys-apps/sed-4.3:

 * ERROR: sys-apps/sed-4.3::gentoo failed (compile phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=sys-apps/sed-4.3::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sys-apps/sed-4.3::gentoo'`.
 * The complete build log is located at '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/build.log'.
 * The ebuild environment file is located at '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/environment'.
 * Working directory: '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'
 * S: '/apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3'

Hmmmm, I was already afraid of this to happen.  Running
  /apps/gentoo/bin/bash bootstrap-rap.sh "/apps/gentoo" stage3
somewhere failed :(  Details might be found in the build log:
  /apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/temp/build.log
  (no build logs found?!?)
I have no clue, really.  Please find friendly folks in #gentoo-prefix on
irc.gentoo.org, gentoo-alt@lists.gentoo.org mailing list, or file a bug
at bugs.gentoo.org under Gentoo/Alt, Prefix Support.  This is most
inconvenient, and it crushed my ego.  Sorry, I give up.
Should you want to give it a try, there is /apps/gentoo/stage3.log


$ ls -latr /apps/gentoo/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3/doc/
total 372
-rw-r--r--  1 mmokrejs mmokrejs   7537 Sep  6  2014 sed.x
-rw-r--r--  1 mmokrejs mmokrejs  23432 Sep 15  2014 fdl.texi
-rw-r--r--  1 mmokrejs mmokrejs    957 Dec 10 07:40 config.texi
-rw-r--r--  1 mmokrejs mmokrejs   1259 Dec 15 01:52 local.mk
-rw-r--r--  1 mmokrejs mmokrejs 136723 Dec 30 09:49 sed.texi
-rw-r--r--  1 mmokrejs mmokrejs     97 Dec 30 10:27 version.texi
-rw-r--r--  1 mmokrejs mmokrejs     97 Dec 30 10:27 stamp-vti
-rw-r--r--  1 mmokrejs mmokrejs 180989 Dec 30 10:27 sed.info
drwxr-xr-x 10 mmokrejs mmokrejs   4096 Jan  5 14:41 ..
drwxr-xr-x  2 mmokrejs mmokrejs   4096 Jan  5 14:41 .
$
Comment 1 Martin Mokrejš 2017-01-05 15:12:56 UTC
Provided I am stuck at "bash bootstrap-rap.sh" step I cannot provide 'emerge --info'. But you all know that Gentoo::RAP is using the x86 portage tree directly, only overriding some eclasses if I am not mistaken. See https://wiki.gentoo.org/wiki/Prefix/libc .
Comment 2 Lars Wendler (Polynomial-C) gentoo-dev 2017-01-05 15:20:28 UTC
Bug in the build system (see URL)
Comment 3 A. Gordon 2017-01-05 15:52:00 UTC
Hi,

Thanks for the report, we'll track up stream at https://bugs.gnu.org/25367 .

I'm not that familiar with Gentoo or RAP -
Can you expand on how is the package built?
e.g. from git / tarball,
and what is "bootstrap-rap.sh" compared to the vanilla "bootstrap.sh" ?


Not directly related, but a "help2man" issue was also reported for cross-compilation ( https://bugs.gnu.org/25358 ), so we might need to rewrite some build rules there.


thanks,
 - assaf
Comment 4 Martin Mokrejš 2017-01-05 16:09:43 UTC
Thank you for your findings. I see the problematic code is in sed-4.3/Makefile:

doc/sed.1: sed/sed$(EXEEXT) .version $(srcdir)/doc/sed.x
        $(AM_V_GEN)$(MKDIR_P) doc
        $(AM_V_at)rm -rf $@ $@-t
        $(AM_V_at)$(HELP2MAN)                                           \
            --name 'stream editor for filtering and transforming text'  \
            -p sed --include $(srcdir)/doc/sed.x                        \
            -o $@-t $(SEDBIN)                                           \
          && chmod a-w $@-t                                             \
          && mv $@-t $@


Do not know why is the '-t' there but evidently makes it into 'doc/sed.1-t' string.


Benda Xu (CCed here) can explain better what the RAP extension is doing, see https://wiki.gentoo.org/wiki/Project:Android . The "bootstrap.sh" is the original Gentoo::Prefix approach, which doe snot allow you to have a separate glibc. Only the RAP approach gives you a way to completely ignore host's glibc. The table in the initially mentioned URL shows which tricks are needed in which approach. The rightmost column "Prefix/libc (native paths method, experimental)" describes some even better approach, which is not yet implemented. I am just a users, I cannot comment on the internals, sorry. But, it is a sort of cross-compile approach, so that on some operating system you can install a Gentoo Linux distribution. A table of supported host systems is not so large like in the case of Gentoo::Prefix approach, but that is because of the additional glibc (actually loader) trickery I guess?

I will attach complete logfile showing how sed tarball was processed. In brief, it used 4.3 tarball. A patch from you can be accepted through this bugzilla record or via https://github.com/gentoo/gentoo . Or fetched from your upstream bug tracker, or git commit ID#. Just provide a note here.
Comment 5 Martin Mokrejš 2017-01-05 16:11:58 UTC
Created attachment 458850 [details]
build.log
Comment 6 Martin Mokrejš 2017-01-05 16:13:48 UTC
Created attachment 458852 [details]
config.log
Comment 7 Martin Mokrejš 2017-01-05 16:14:06 UTC
Created attachment 458854 [details]
config.status
Comment 8 A. Gordon 2017-01-05 16:19:13 UTC
Hi,

Regarding the "-t":
That is intentional, it's a common technique used also in other gnu packages.

The target to build is "doc/sed.1" (which is "$@" in makefile).

All commands operate on a temp file named "$@-t" (expanded to "doc/sed.1-t"),
and the last "mv" command atomically renames the temp file to the final target name:

      help2man [...] -o "$@-t"
      chmod a-w "$@-t"
      mv "$@-t" "$@"

This ensures a failure in one of the steps does not result in partial/empty/corrupted "doc/sed.1".

The "-t" itself is not the problem (at least from a cursory look).

Since the error is:
  chmod: cannot access 'doc/sed.1-t': No such file or directory

This means that for some reason, "help2man" did not generate "doc/sed.1-t".

I'm not sure why, so this still needs checking.
Thanks for the logs - will take a look at them soon.

If you are building from a tarball, then "doc/sed.1" is already pre-built.
Regenerating it is a build bug in sed, and we'll improve it.

-assaf
Comment 9 Martin Mokrejš 2017-01-05 16:22:08 UTC
> we'll track up stream at https://bugs.gnu.org/25367

To answer your question in there, Gentoo does not apply any patch in this case (which would be shown in the build.log). You can look for package internal at https://github.com/gentoo/gentoo/tree/master/sys-apps/sed

My host system (on which I want to install Gentoo into /apps/gentoo/) is running:

$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.4 (Santiago)
$

It could be some application behaves differently on the RedHat platform. Try to check if the configure works well on a similar redhat/centos installation. Or check the config.log or config.status to reproduce the issue.
Comment 10 Martin Mokrejš 2017-01-05 16:30:43 UTC
(In reply to A. Gordon from comment #8)

> This means that for some reason, "help2man" did not generate "doc/sed.1-t".
> 
> I'm not sure why, so this still needs checking.

make SHELL=/apps/gentoo/tmp/bin/bash -j8 

Could be an issue with dependency ordering? I cannot test this easily, aside form just rerunning "make bootstrap-rap.sh" and setting number of threads to 1 at some point. You might be faster to reproduce it on you computer or just checking Makefiles and the build.log file in sync.
Comment 11 Martin Mokrejš 2017-01-05 18:52:08 UTC
(In reply to Martin Mokrejš from comment #10)

> make SHELL=/apps/gentoo/tmp/bin/bash -j8 

So I re-tested this, and yes, with one thread the build process succeeds. Check order of Makefile rules.
Comment 12 A. Gordon 2017-01-05 19:08:35 UTC
Hi,

I haven't looked at the logs yet, but this seems strange.

The part that fails is essentially this:

doc/sed.1: sed/sed doc/sed.x
      help2man [...] -o "$@-t" \
      && chmod a-w "$@-t" \
      && mv "$@-t" "$@"

And "chmod" fails because "doc/sed.1-t" was not found.

Yet there is no palatalization possible in this specific rule.
It is a simple shell command:

     help2man -o doc/sed.1-t && chmod a-w doc/sed.1-t

which means 'help2man' failed to create the file,
yet returned exit code 0, and 'chmod' was still executed afterwards.

'help2man' is a perl script - can you tell which perl is used (e.g. version / path) ?


thanks,
 - assaf
Comment 13 A. Gordon 2017-01-05 19:12:27 UTC
Hi,

I'm also a bit confused about an extra string you have in the logs:
   "apiversion=9999"

It wasn't generate by anything in sed - so it's something in your build system...

any ideas?
Comment 14 A. Gordon 2017-01-05 19:17:16 UTC
A bit of googling shows that in:
   http://dev.gentoo.org/~heroxbd/bootstrap-rap.sh

You have a line that creates a stub perl script:

  # trick "perl -V:apiversion" check of glibc-2.19.
  echo -e "#!${ROOT}/bin/sh\necho 'apiversion=9999'" > "${ROOT}"/usr/bin/perl
  chmod +x "${ROOT}"/usr/bin/perl

Is this being used ?
If so - you fake a working perl (always terminates with code 0),
but never does what it's supposed to do, and sed's help2man fails.

Let me know if this is the issue.

thanks,
 - assaf
Comment 15 Martin Mokrejš 2017-01-05 19:25:10 UTC
We will need to wait for @heroxbd for an answer. ;-)

I wonder why with one thread I could get through.
Comment 16 Martin Mokrejš 2017-01-05 21:33:03 UTC
I did another install attempt using 8 threads and again hit this sed issue (good). Do not worry about the different path $(EPREFIX) in the below outputs.

First of all, by the time sed is compiled/installed in stage3 no perl is yet installed inside the RAP installation:


$ grep "perl" /scratch/mmokrejs/g/stage?.log | grep merged
$

I should have seen something similar to the below but speaking of sys-apps/sed:

$ grep "glibc" /scratch/mmokrejs/g/stage?.log | grep merged
/scratch/mmokrejs/g/stage3.log:>>> sys-libs/glibc-2.23-r3 merged.
$


Regarding the trickered script, indeed there is:

$ head /scratch/mmokrejs/g/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3/build-aux/help2man 
#!/usr/bin/env perl

...
$

Provided there is in build.log:

./build-aux/help2man                                            \
    --name 'stream editor for filtering and transforming text'  \
    -p sed --include ./doc/sed.x                        \
    -o doc/sed.1-t sed/sed                                              \
  && chmod a-w doc/sed.1-t                                              \
  && mv doc/sed.1-t doc/sed.1
apiversion=9999

I am tempted to say /usr/bin/env perl was used ... do not know what env decided to do.



Wrong attempt:

$ pushd  /scratch/mmokrejs/g/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3/
$ strace -v -f ./build-aux/help2man   
...
execve("/usr/lib64/qt-3.3/bin/perl", ["perl", "./build-aux/help2man"]


Better import the env settings:
$ pushd  /scratch/mmokrejs/g/tmp/var/tmp/portage/sys-apps/sed-4.3/work/sed-4.3/
$ . /scratch/mmokrejs/g/etc/profile
$ strace -v -f ./build-aux/help2man
...
open("/scratch/mmokrejs/g/usr/bin/perl", O_RDONLY) = 3
...
read(3, "#!/scratch/mmokrejs/g/bin/sh\nech"..., 80) = 52
...
write(1, "apiversion=9999\n", 16apiversion=9999
...
exit_group(0)                           = ?
+++ exited with 0 +++
$

Seems you are right.
Comment 17 Benda Xu gentoo-dev 2017-01-06 03:54:17 UTC
(In reply to A. Gordon from comment #14)
> A bit of googling shows that in:
>    http://dev.gentoo.org/~heroxbd/bootstrap-rap.sh
> 
> You have a line that creates a stub perl script:
> 
>   # trick "perl -V:apiversion" check of glibc-2.19.
>   echo -e "#!${ROOT}/bin/sh\necho 'apiversion=9999'" > "${ROOT}"/usr/bin/perl
>   chmod +x "${ROOT}"/usr/bin/perl
> 
> Is this being used ?
> If so - you fake a working perl (always terminates with code 0),
> but never does what it's supposed to do, and sed's help2man fails.
> 
> Let me know if this is the issue.

That must be the issue. A stub perl was created in the bootstrap, and perl ultimately need sed to build.

Hmm, there must be a better way to do this.  A hack will always bite back.
Comment 18 Benda Xu gentoo-dev 2017-01-06 04:07:25 UTC
I can reproduce this bug by replacing perl with the stub one on a bootstrapped gentoo.  But it only affects sed-4.3, somehow sed-4.2.2 compiled.
Comment 19 Benda Xu gentoo-dev 2017-01-06 15:00:56 UTC
Provided we have identified the bug, it is fixed in https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=dcc6b5f3d0fa0074fb1dfc5d646601fc03574353

Please download bootstrap-rap.sh again.
Comment 20 Martin Mokrejš 2017-01-06 17:12:00 UTC
true, now it installs fine. thanks.