Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 575958 - openrc tests are fragile as they sed/awk through readelf output
Summary: openrc tests are fragile as they sed/awk through readelf output
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: OpenRC (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: OpenRC Team
URL: https://lists.debian.org/debian-power...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-29 03:08 UTC by Benda Xu
Modified: 2019-02-26 18:39 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Build log (ppc64) (openrc-0.24.1-r1-ppc64-elaine.log,81.62 KB, text/x-log)
2017-11-27 05:45 UTC, A. Wilcox (awilfox)
Details
Build objects (openrc-0.24.1-build.txz,565.71 KB, application/x-xz)
2019-02-14 11:48 UTC, A. Wilcox (awilfox)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Benda Xu gentoo-dev 2016-02-29 03:08:38 UTC
rc_config_directory and rc_config_set_value introduced in c09eeca49145b0 fail to pass test on hidden functions on ppc64el[1] and mips64el[2].

The reason is unclear.  But some clue is found by Dmitry Yu Okunev
> On "ppc64el" static function compiled with "gcc" is visible by "runtests.sh" > (that uses "readelf"), but it's not on amd64.

1. https://buildd.debian.org/status/fetch.php?pkg=openrc&arch=ppc64el&ver=0.20.4-1&stamp=1456247204

2. https://buildd.debian.org/status/fetch.php?pkg=openrc&arch=mips64el&ver=0.20.4-1&stamp=1456251406

Reproducible: Always
Comment 1 William Hubbs gentoo-dev 2016-05-06 19:13:07 UTC
As shown in this bug, this is causing OpenRC's test suite to
fail for these platforms.

Do you have any suggestions?

Thanks,

William
Comment 2 SpanKY gentoo-dev 2016-05-07 20:16:24 UTC
i can't reproduce w/mips64.  would help if someone can attach a tarball of the failing objects.
Comment 3 A. Wilcox (awilfox) 2017-11-27 05:45:23 UTC
Created attachment 506796 [details]
Build log (ppc64)

We're hitting this on ppc64 (*not* ppc64le) in Adélie, which is causing the aforementioned test failure.

Attached is the build log.  Next attachment will be the build tree.

We use the Alpine build tools instead of Portage, and this is the contents of the /etc/abuild.conf on the ppc64 builder (like /etc/portage/make.conf):


export CFLAGS="-O2 -ggdb -mcpu=970 -mtune=power9 -maltivec -mlong-double-64 -fno-inline-small-functions"
export CXXFLAGS="$CFLAGS"
export CPPFLAGS="$CFLAGS"
export LDFLAGS="-Wl,--as-needed"

export JOBS=2
export MAKEFLAGS=-j$JOBS

USE_COLORS=1
SRCDEST=/var/cache/distfiles
REPODEST=/srv/packages/

export OPTIONS="!strip"
export DEFAULT_DBG="YesPlease"
export DISTRO_NAME="Adélie Linux"
export DISTRO_BUG_URL="https://bts.adelielinux.org/"
Comment 4 A. Wilcox (awilfox) 2017-11-27 05:48:38 UTC
As requested by vapier@, a tarball of the build directory with object files is available at http://foxkit.us/linux/openrc-0.24.1-build.txz – unfortunately, it is over the 1,000 KB limit for Bugzilla attachments.

This should resolve the reason that this bug was marked RESO NEEDINFO, so I am changing the status back to CONFIRMED.
Comment 5 Sergei Trofimovich (RETIRED) gentoo-dev 2019-02-13 23:11:22 UTC
(In reply to A. Wilcox (awilfox) from comment #4)
> As requested by vapier@, a tarball of the build directory with object files
> is available at http://foxkit.us/linux/openrc-0.24.1-build.txz

404 Not Found.
Comment 6 A. Wilcox (awilfox) 2019-02-14 11:48:31 UTC
Created attachment 565258 [details]
Build objects

The file has been re-uploaded to my host.  I also used more extreme xz options to make it fit under the 1000 KB limit so it can be hosted on BZ as well.

Please reopen this bug.  It is still very much a bug.
Comment 7 Sergei Trofimovich (RETIRED) gentoo-dev 2019-02-14 22:50:14 UTC
Please paste 'emerge --info' from the affected system as well.
Comment 8 Sergei Trofimovich (RETIRED) gentoo-dev 2019-02-17 12:26:44 UTC
(In reply to A. Wilcox (awilfox) from comment #6)
> Created attachment 565258 [details]
> Build objects

Thanks, that helps. You have two types of test failures:
1. "exported" libc symbols
2. "exported" rc_* symbols

TL;DR:
  Both are openrc bugs and not toolchain bugs. openrc should fix (or better yet remove) tests that sed through readelf output.

More details:

I'll focus on "exported" rc_* symbols here as they are reported in original bug for openrc-0.24.1:

>  * Checking hidden functions in librc.so ...
>  * Missing hidden defs:
> rc_config_directory
> rc_config_set_value
> rc_parse_service_state

Here is a test definition:
  https://github.com/OpenRC/openrc/blob/0.24.x/src/test/runtests.sh#L55

```sh
ebegin "Checking hidden functions in librc.so"
sed -n '/^librc_hidden_proto/s:.*(\(.*\))$:\1:p' ${librc_srcdir}/librc.h \
	| LC_ALL=C sort -u \
	> librc.funcs.hidden.list
readelf -Wr $(grep -l '#include[[:space:]]"librc\.h"' ${librc_srcdir}/*.c | sed 's:\.c$:.o:') \
	| egrep -v -e 'R_PARISC_(DP|SEG)REL' \
	| awk '$5 ~ /^rc_/ {print $5}' \
	| LC_ALL=C sort -u \
	| egrep -v '^rc_environ_fd$' \
	> librc.funcs.hidden.out
syms=$(diff -u librc.funcs.hidden.list librc.funcs.hidden.out | sed -n '/^+[^+]/s:^+::p')
[ -z "${syms}" ]
eend $? "Missing hidden defs:"$'\n'"${syms}"
```

A few things to note here:
1. the proxy of exported symbols is output of readelf on object files, not a final .so library.
2. the proxy of exported symbol is a presence of relocation for a symbol that has 'rc_*' name.
3. this script does not really check if 'rc_*' symbols are local or not.

To double-check it's not a hidden but local symbol you can looks at symbol type:

This is supposed to be local:
  openrc-0.24.1/src/test $ LANG=C readelf -a ../librc/librc-misc.o | fgrep rc_config_set
  000000000dd0  000a0000000a R_PPC64_REL24     00000000000001b0 rc_config_set_value + 0
  000000000ec8  000a0000000a R_PPC64_REL24     00000000000001b0 rc_config_set_value + 0
      10: 00000000000001b0   648 FUNC    LOCAL  DEFAULT [<localentry>: 8]     1 rc_config_set_value

This is supposed to be hidden:
  openrc-0.24.1/src/test $ LANG=C readelf -a ../librc/librc-misc.o | fgrep rc_config_list
  000000000d94  003b0000000a R_PPC64_REL24     0000000000000a80 __RC_rc_config_list + 0
  000000000ea0  003b0000000a R_PPC64_REL24     0000000000000a80 __RC_rc_config_list + 0
      59: 0000000000000a80   496 FUNC    GLOBAL HIDDEN  [<localentry>: 8]     1 __RC_rc_config_list
      62: 0000000000000a80   496 FUNC    GLOBAL DEFAULT [<localentry>: 8]     1 rc_config_list

There is no discrepancy here. Test script should be fixed.

Now, why it's not failing for everyone then? I don't know.
My guess it's because of gcc-6.4.0 applying different inlining
heuristics than gcc-7.3.0/gcc-8.2.0 and a combination of passed
CFLAGS:
    -O2 -ggdb -mcpu=970 -mtune=power9 -maltivec -mlong-double-64 -fno-inline-small-functions
Or maybe your gcc is ubilt differently WRT pie, stack protection, etc.
Hard to say without looking at 'gcc -v'.

Normally 'rc_config_set_value' can get inlined in compiler
frontend as it's a static local function
    https://github.com/OpenRC/openrc/blob/0.24.x/src/librc/librc-misc.c#L207
Comment 9 Sergei Trofimovich (RETIRED) gentoo-dev 2019-02-17 16:44:17 UTC
Sent test removal for review as https://github.com/OpenRC/openrc/pull/292
Comment 10 William Hubbs gentoo-dev 2019-02-26 18:39:26 UTC
These tests have been removed since they do not work well cross
platform.

William