| Summary: | sys-libs/uclibc: pread/pwrite do not work on arm (breaks dev-vcs/git) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Steev Klimaszewski (RETIRED) <steev> |
| Component: | Current packages | Assignee: | Embedded Gentoo Team <embedded> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | blueness |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
strace of git running on uclibc
uclibc savedconfig ad hoc fix for non-atomic pread/write on armv7a-uclibc. |
||
|
Description
Steev Klimaszewski (RETIRED)
2014-02-05 15:19:45 UTC
Created attachment 369614 [details]
strace of git running on uclibc
This was an attempt to run the following:
git clone git://github.com/hardkernel/linux -b odroid-3.8.y --depth=1
Which is the kernel used on an odroid u3 device.
[I] dev-vcs/git
Available versions: 1.8.1.5 1.8.1.5-r1 ~1.8.2.1 1.8.3.2-r1 ~1.8.4.5 ~1.8.5.3 **9999 {+blksha1 cgi +curl cvs doc emacs gnome-keyring +gpg gtk highlight +iconv mediawiki +nls +pcre +perl ppcsha1 +python subversion test +threads tk +webdav xinetd PYTHON_SINGLE_TARGET="python2_6 python2_7" PYTHON_TARGETS="python2_6 python2_7"}
Installed versions: 1.8.3.2-r1(14:30:32 02/05/14)(blksha1 gpg iconv pcre perl python threads -cgi -curl -cvs -doc -emacs -gnome-keyring -gtk -highlight -nls -ppcsha1 -subversion -test -tk -webdav -xinetd PYTHON_SINGLE_TARGET="python2_7 -python2_6" PYTHON_TARGETS="python2_7 -python2_6")
[I] sys-libs/uclibc
Available versions: 0.9.33.2^s (~)0.9.33.2-r1^s 0.9.33.2-r2^s 0.9.33.2-r3^s 0.9.33.2-r4^s (~)0.9.33.2-r5^s 0.9.33.2-r6^s (~)0.9.33.2-r7^s 0.9.33.2-r8^s (~)0.9.33.2-r9^s **9999^s {debug hardened iconv ipv6 nptl rpc savedconfig ssp uclibc-compat wordexp CROSSCOMPILE_OPTS="headers-only"}
Installed versions: 0.9.33.2-r9^s(12:21:38 02/05/14)(savedconfig -debug -hardened -iconv -ipv6 -nptl -rpc -ssp -uclibc-compat -wordexp CROSSCOMPILE_OPTS="-headers-only")
emerge --info Portage 2.2.7 (hardened/linux/uclibc/arm/armv7a, gcc-4.7.3, uclibc-0.9.33.2-r9, 3.4.74 armv7l) ================================================================= System uname: Linux-3.4.74-armv7l-ARMv7_Processor_rev_3_-v7l-with-gentoo-2.2 KiB Mem: 1789860 total, 763028 free KiB Swap: 2097148 total, 2096864 free Timestamp of tree: Wed, 05 Feb 2014 00:00:01 +0000 ld GNU ld (GNU Binutils) 2.23.2 app-shells/bash: 4.2_p45 dev-lang/python: 2.7.5-r3, 3.2.5-r3, 3.3.3 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.69 sys-devel/automake: 1.13.4 sys-devel/binutils: 2.23.2 sys-devel/gcc: 4.7.3-r1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.9 (virtual/os-headers) sys-libs/uclibc: 0.9.33.2-r9 Repositories: gentoo ACCEPT_KEYWORDS="arm" ACCEPT_LICENSE="* -@EULA" CBUILD="armv7a-hardfloat-linux-uclibceabi" CFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard" CHOST="armv7a-hardfloat-linux-uclibceabi" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="-j9 --load-average=9" FCFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j9 -l9" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://xu/gentoo-portage" USE="arm bindist cli cracklib crypt cxx dri fortran iconv ipv6 modules ncurses nptl openmp pax_kernel pcre pic readline session ssl tcpd uclibc unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="uclibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="dummy fbdev v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON So, here is some fun that I ran into - I updated uclibc on another machine, that still has git 1.7.12.4 installed.
[I] sys-libs/uclibc
Available versions: 0.9.33.2^s ~0.9.33.2-r1^s 0.9.33.2-r2^s 0.9.33.2-r3^s 0.9.33.2-r4^s ~0.9.33.2-r5^s 0.9.33.2-r6^s ~0.9.33.2-r7^s 0.9.33.2-r8^s ~0.9.33.2-r9^s **9999^s {{crosscompile_opts_headers-only debug hardened iconv ipv6 nptl rpc savedconfig ssp uclibc-compat wordexp}}
Installed versions: 0.9.33.2-r8^s(14:36:13 02/05/14)(savedconfig -crosscompile_opts_headers-only -debug -hardened -iconv -ipv6 -nptl -rpc -ssp -uclibc-compat -wordexp)
(git version)
Installed versions: 1.7.12.4(04:02:25 01/24/13)(blksha1 curl gpg pcre perl python threads webdav -cgi -cvs -doc -emacs -gtk -highlight -iconv -nls -ppcsha1 -subversion -test -tk -xinetd)
kunlun ~ # git clone git://github.com/offensive-security/kali-arm-build-scripts
Cloning into 'kali-arm-build-scripts'...
remote: Reusing existing pack: 191, done.
remote: Total 191 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (191/191), 151.07 KiB, done.
fatal: premature end of pack file, 47 bytes missing
fatal: index-pack failed
So now, on newer uclibc, even git 1.7 fails.
will attach the config that is used by these machines for uclibc, perhaps we're missing something on arm...?
Created attachment 369626 [details]
uclibc savedconfig
uclibc config used (originally comes from 0.9.33.2 (vanilla, not -rX))
(In reply to Steev Klimaszewski from comment #4) > Created attachment 369626 [details] > uclibc savedconfig > > uclibc config used (originally comes from 0.9.33.2 (vanilla, not -rX)) You need the following from uclibc's master branch: http://git.uclibc.org/uClibc/commit/?id=564a95241f8468145ff85998ef9173c46aed7636 Unfortunately, the amount of work done on pread/pwrite since 0.9.33.2 came out make it difficult to cleanly apply that patch to the head of the 0.9.33.x branch. I'm going to have to hack a bit to port it over to the branch. BTW, you don't need all that commit for arm, only the __UCLIBC_SYSCALL_ALIGN_64BIT__ stuff (hunk 1). However, I'm not 100% sure why we need 64-bit alignment. (In reply to Anthony Basile from comment #5) > BTW, you don't need all that commit for arm, only the > __UCLIBC_SYSCALL_ALIGN_64BIT__ stuff (hunk 1). However, I'm not 100% sure > why we need 64-bit alignment. I narrowed it down to the difference between _syscall6 and _syscall5. In uclibc-0.9.33.2, pread/write uses _syscall5, but in uclibc-9999, __UCLIBC_SYSCALL_ALIGN_64BIT__ is defined in libc/sysdeps/linux/arm/bits/uClibc_arch_features.h and this triggers the use of _syscall6. Why _syscall5 is not atomic but _syscall6 is, I'm not sure yet. I'll give a quick-fix patch in my next post, but its just for analysis purpose or as an ad hoc fix for just armv7a, but it would break other arches. Created attachment 370656 [details, diff]
ad hoc fix for non-atomic pread/write on armv7a-uclibc.
Okay here's why I'm confused: both _syscall5 and 6 expand the same way on arm_eabi:
1. in libc/sysdeps/linux/common/bits/syscalls-common.h _syscall{5,6} are defined as follows
#define _syscall5(args...) SYSCALL_FUNC(5, args)
#define _syscall6(args...) SYSCALL_FUNC(6, args)
2. SYSCALL_FUNC expands as follows
#define SYSCALL_FUNC(nargs, type, name, args...) \
type name(C_DECL_ARGS_##nargs(args)) { \
return (type)INLINE_SYSCALL(name, nargs, C_ARGS_##nargs(args)); \
}
3. and INLINE_SYSCALL is defined as
define INLINE_SYSCALL(name, nr, args...) INLINE_SYSCALL_NCS(__NR_##name, nr, args)
# define INLINE_SYSCALL_NCS(name, nr, args...) \
(__extension__ \
({ \
INTERNAL_SYSCALL_DECL(__err); \
(__extension__ \
({ \
long __res = INTERNAL_SYSCALL_NCS(name, __err, nr, args); \
if (unlikely(INTERNAL_SYSCALL_ERROR_P(__res, __err))) { \
__set_errno(INTERNAL_SYSCALL_ERRNO(__res, __err)); \
__res = -1L; \
} \
__res; \
}) \
); \
}) \
)
4. INTERNAL_SYSCALL_NCS is defined in include/bits/syscalls.h where for __ARM_EABI__ it expands to
#define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \
(__extension__ \
({unsigned int __internal_sys_result; \
{ \
register int __a1 __asm__ ("r0"), _nr __asm__ ("r7"); \
LOAD_ARGS_##nr (args) \
_nr = (name); \
__asm__ __volatile__ ("swi 0x0 @ syscall " #name \
: "=r" (__a1) \
: "r" (_nr) ASM_ARGS_##nr \
: "memory"); \
__internal_sys_result = __a1; \
} \
(int) __internal_sys_result; }) \
)
5. So both _syscall5 and 6 expand the same way. Can the non-atomicity of _syscall5 be a kernel issue?
i had to backport some things upstream first: http://git.uclibc.org/uClibc/commit/?h=0.9.33&id=e619d79c98316388638354d79a12ee6d80911bba http://git.uclibc.org/uClibc/commit/?h=0.9.33&id=8165f1616a95ecef7b407b11368bfd138889d82c http://git.uclibc.org/uClibc/commit/?h=0.9.33&id=525d31e51fca97051978aabd5a6295ffb8f82a17 then pull them into Gentoo: http://sources.gentoo.org/gentoo/src/patchsets/uclibc/0.9.33.2/13_all_64bit_syscall_align.patch?rev=1.1 http://sources.gentoo.org/gentoo/src/patchsets/uclibc/0.9.33.2/14_all_pread_pwrite_threads.patch?rev=1.1 http://sources.gentoo.org/gentoo/src/patchsets/uclibc/0.9.33.2/15_all_pread_pwrite_renamed.patch?rev=1.1 http://sources.gentoo.org/gentoo/src/patchsets/uclibc/0.9.33.2/16_all_pread_pwrite_renamed_typo.patch?rev=1.1 http://sources.gentoo.org/gentoo/src/patchsets/uclibc/0.9.33.2/17_all_pread_pwrite_64bit_align.patch?rev=1.1 (In reply to Anthony Basile from comment #5) i'm afraid all of your leads seem to be off track. check out the section "Architecture-specific requirements" of the syscall(2) man page as it explains things. should be all set now in the tree; thanks for the report! Commit message: Add more pread/pwrite fixes from upstream http://sources.gentoo.org/sys-libs/uclibc/uclibc-0.9.33.2-r10.ebuild?rev=1.1 (In reply to SpanKY from comment #10) > (In reply to Anthony Basile from comment #5) > > i'm afraid all of your leads seem to be off track. check out the section > "Architecture-specific requirements" of the syscall(2) man page as it > explains things. Its because the 64-bit values need to be aligned to an EVEN register pair. So the macro _syscall6(ssize_t, __syscall_pwrite, int, fd, const void *, buf, size_t, count, int, dummy, off_t, offset_hi, off_t, offset_lo) means somethng like syscall(SYS_pread64, fd, buf, count, dummy, offset_hi, offset_lo); which are loaded as follows for arm_eabi: r0=fd r1=buf r2=count r3=dummy r4=offset_hi r5=offset_lo and the alignment is on r4/r5 --- and EVEN pair. While _syscall5(ssize_t, __syscall_pread, int, fd, void *, buf, size_t, count, off_t, offset_hi, off_t, offset_lo) gives syscall(SYS_pread64, fd, buff, count, offset_hi, offset_lo) and now we get r3=offset_hi, r4=offset_lo --- and ODD pair. Do I pass? Anyhow, I don't know arm that well, but this is like mips. (In reply to Anthony Basile from comment #12) that is correct. you get a cookie ;). |