Created attachment 441258 [details, diff] gentoo/patchlevel-manifest.diff Triage: Was testing our patch series independently and started seeing sporadic errors (~10% of the time) happening without an obvious reason. After much analysis, I discovered the cause is our process ( which I faithfully replicated ) results in an out-of-order MANIFEST file, and this causes breakage and test failures. The only reason we haven't seen these failures wide-spread is because there's some process in 5.24 that automatically re-sorts it during the miniperl compile stage. However, if a race condition occurs, this re-order might be done before Cwd.pm is compiled, leading to the file *not* being re-sorted, leading to a failure in tests that require a given sort order. This is more obvious when you back-port the "fix" to 5.20.x, as it causes tests to *always* fail due to the lack of re-sort mechanism: https://travis-ci.org/gentoo-perl/perl-patchset/builds/146266251 Note: We cannot simply use system sort here, because its sort order and perls differ. The fault was introduced by this commit, which doesn't have a lot of detail or adequate citation, nor evidence of the problem: https://github.com/gentoo/gentoo/commit/24682fa7ca762c19de50999d2f93f9969c5b2e5a > dev-lang/perl: Add patchlevel-gentoo to perl MANIFEST > > To allow miniperl compilation, patchlevel-gentoo.h must be in the MANIFEST. > Configure uses this file to create the /host symlinks. > > Signed-off-by: Gwendal Grignou <gwendal@chromium.org> > > Comment from Gentoo perl team: Being nice here and adding this commit which > ended up without any further reasons or docs in our mailbox. I can't see how it > could hurt. I can't really see any problem that it's supposed to fix > either. Hey chromium team, more words please next time! And this logic is transposed into the 5.24.0-r1 ebuild: https://github.com/gentoo/gentoo/blame/master/dev-lang/perl/perl-5.24.0-r1.ebuild#L239 This logic should be removed, and upon a way of proving its necessity, it should be patched into place in the correct order using the attached patch. I could easily do this myself, but filing bug for extra eyes and documentation. And of course, a paper trail to cite when somebody complains we removed it.
@Gwendal, adding you as CC on this issue as your name is on the original commit, so hopefully you can shed some light on this.
commit 1ddfb9cc7eb204864bcf113b74ca4c5c9780c84a (HEAD -> master, origin/master, origin/HEAD) Author: Andreas K. Hüttel <dilfridge@gentoo.org> Date: Thu Dec 29 17:53:20 2016 +0100 dev-lang/perl: Restore MANIFEST sort order, bug 589302 Package-Manager: Portage-2.3.3, Repoman-2.3.1 dev-lang/perl/files/perl-patchlevel-manifest.patch | 10 ++++++++++ dev-lang/perl/perl-5.24.1_rc4.ebuild | 6 +++--- 2 files changed, 13 insertions(+), 3 deletions(-)
Created attachment 457854 [details] Compilation Failure when "patchlevel-gentoo.h is not in the manifest. #1. Sorry, I missed the email. The patch intent is to add patchlevel-gentoo.h in MANIFEST: It was required for a following patch to [pseudo]cross-compile miniperl only (see https://chromium-review.googlesource.com/#/c/347748/) - note this patch is moot since perl 5.24 do support proper cross-compilation. If absent from MANIFEST, a file would not linked into the portage /work/perl-<>/host directory and compilation of miniperl would fail: >>> Configuring source in /build/samus/tmp/portage/dev-lang/perl-5.22.1-r1/work/perl-5.22.1 ... First let's make sure your kit is complete. Checking... Locating common programs... Checking compatibility between /bin/echo and builtin echo (if any)... Symbolic links are supported. Checking how to test for symbolic links... You can test for symbolic links with 'test -h'. Targetarch not defined. Targethost not defined. Building host miniperl and generate_uudmap binaries In file included from perl.c:34:0: patchlevel.h:140:31: fatal error: patchlevel-gentoo.h: No such file or directory #include "patchlevel-gentoo.h" ^ compilation terminated. [Full log enclosed.] I was not aware of the ordering requirement, I verified the new patch is working as well.
Thanks Gwendal. Seems now we fixed the bug twice, so I reverted my last commit (and the header file name is also now configurable again). https://github.com/gentoo-perl/perl-patchset/commit/23179a630d04c3d40e7c09bccbf2f38f3ad72b93 commit 1d7266e35487144b7a48370571072c74c09c6347 (HEAD -> master, origin/master, origin/HEAD) Author: Andreas K. Hüttel <dilfridge@gentoo.org> Date: Thu Jan 5 10:40:54 2017 +0100 Revert "dev-lang/perl: Restore MANIFEST sort order, bug 589302" This commit / patch is not needed since we already have a fix for the bug. See https://github.com/gentoo-perl/perl-patchset/commit/23179a630d04c3d40e7c09bccbf2f38f3ad72b93 This reverts commit 1ddfb9cc7eb204864bcf113b74ca4c5c9780c84a. dev-lang/perl/files/perl-patchlevel-manifest.patch | 10 ---------- dev-lang/perl/perl-5.24.1_rc4.ebuild | 6 +++--- 2 files changed, 3 insertions(+), 13 deletions(-)
