libpcreposix.so from dev-libs/libpcre implements POSIX regex functions (regcomp, regexec, regerror, regfree), just like glibc. Unfortunately, these two implementations aren't binary-compatible due to, at least, different size of regex_t caller-provided pattern buffer storage area. One of victims of this incompatibility is net-mail/cyrus-imapd[sieve], which crashes on any regex usage (since it compiles against libpcre's implementation for more features, but ends with being linked to glibc's one). Simple way to reproduce the crash it to run: echo 'require "regex"; if header :regex "To" ".*foo@bar?" { stop; }' >/tmp/stest; /usr/lib/cyrus/sievec /tmp/stest /tmp/stest.out Result: *** Error in `/usr/lib/cyrus/sievec': free(): invalid next size (fast): 0x000055ecab45eba0 *** Looks like exim also might be a victim: https://bugs.exim.org/show_bug.cgi?id=1830 Debian uses a patch to libpcre to rename these functions in libpcreposix.so and redirect callers to these new names for code that includes libpcre's pcreposix.h header: https://sources.debian.net/patches/pcre3/2:8.39-2/pcreposix.patch/ I can confirm that with this patch applied to libpcre and net-mail/cyrus-imapd rebuilt it no longer aborts when calling regex functions.
https://bugs.exim.org/show_bug.cgi?id=1830 This is actually the bug for pcre it's self (pcre is using the same bugzilla system as Exim). From comments in that ticket Debian has been applying their patch for a long time. The original Debian bug report 22525 is no longer accessible. *** Error in `/usr/lib/cyrus/sievec': corrupted double-linked list: 0x0804d8e8 *** ======= Backtrace: ========= /lib/libc.so.6(+0x687a6)[0xb7e1f7a6] /lib/libc.so.6(+0x6e65d)[0xb7e2565d] /lib/libc.so.6(+0x70a66)[0xb7e27a66] /lib/libc.so.6(__libc_malloc+0x5f)[0xb7e28c37] /lib/libc.so.6(+0xcb33a)[0xb7e8233a] /lib/libc.so.6(regcomp+0xb4)[0xb7e83297] /usr/lib/libcyrus_sieve.so.0(+0xd354)[0xb7fac354] /usr/lib/libcyrus_sieve.so.0(+0xd8e4)[0xb7fac8e4] /usr/lib/libcyrus_sieve.so.0(+0x11155)[0xb7fb0155] /usr/lib/libcyrus_sieve.so.0(sieve_script_parse+0xc8)[0xb7fab381] /usr/lib/cyrus/sievec[0x8049081] /lib/libc.so.6(__libc_start_main+0xf4)[0xb7dcf49b] /usr/lib/cyrus/sievec[0x80492a6] ======= Memory map: ======== #### trimmed #### Program received signal SIGABRT, Aborted. 0xb7fdbbcd in __kernel_vsyscall () (gdb) bt #0 0xb7fdbbcd in __kernel_vsyscall () #1 0xb7de2f38 in raise () from /lib/libc.so.6 #2 0xb7de4471 in abort () from /lib/libc.so.6 #3 0xb7e1f7ab in __libc_message () from /lib/libc.so.6 #4 0xb7e2565d in malloc_printerr () from /lib/libc.so.6 #5 0xb7e27a66 in _int_malloc () from /lib/libc.so.6 #6 0xb7e28c37 in malloc () from /lib/libc.so.6 #7 0xb7e8233a in re_compile_internal () from /lib/libc.so.6 #8 0xb7e83297 in regcomp () from /lib/libc.so.6 #9 0xb7fac354 in verify_regex (parse_script=parse_script@entry=0x804dcd8, s=0x8053b40 ".*some sieve regex I don't want to share", cflags=97, cflags@entry=33) at sieve/sieve.y:1663 #10 0xb7fac8e4 in verify_regexs (parse_script=parse_script@entry=0x804dcd8, comp=<optimized out>, sa=<optimized out>, sa=<optimized out>) at sieve/sieve.y:1683 #11 0xb7fb0155 in sieveparse (parse_script=0x804dcd8) at sieve/sieve.y:568 #12 0xb7fab381 in sieve_script_parse (interp=0x804c7d8, script=script@entry=0x804c008, script_context=script_context@entry=0xbfffe9ec, ret=ret@entry=0xbfffe9e8) at sieve/script.c:224 #13 0x08049081 in is_script_parsable (ret=0xbfffe9e0, errstr=<synthetic pointer>, stream=0x804c008) at sieve/sievec.c:217 #14 main (argc=3, argv=0xbfffeab4) at sieve/sievec.c:105
Created attachment 464672 [details, diff] pcre3 pcreposix patch
Comment on attachment 464672 [details, diff] pcre3 pcreposix patch I am having exactly the same problem: cyrus-imapd segfaults when it gets to a sieve script which contains regular expressions. Even compiling a sieve script using sievec results in a segfault. I was able to fix all problems with cyrus and sieve by applying the attached patch which I got from this site: https://sources.debian.net/patches/pcre3/2:8.39-2/pcreposix.patch/ Ruediger
this is an issue to be resolved upstream. i don't think we should be renaming symbols on them and breaking their ABI. any app that uses pcreposix.h better be linking against -lpcreposix. using the header but not linking against that lib is an error.
I am not sure if I understand you right when you say that "this is an issue to be resolved upstream". Do you mean the problem is with the cyrus-imapd? If so, could you please throw some light upon this problem? I made some comments in the cyrus issue tracker https://github.com/cyrusimap/cyrus-imapd/issues/1848 but now I am not sure if I was right. Thank you in advance Ruediger
(In reply to Ruediger Hahn from comment #5) glancing at the source, it looks like a bug in cyrus-imapd. i'll comment on the upstream bug.
(In reply to SpanKY from comment #6) nm, i was looking at unittest files. the source includes pcreposix.h explicitly, and then also links against -lpcreposix. that means those regex symbols will be interposed at runtime over the glibc ones. double check which programs are actually linked against libpcreposix. something like: qlist net-mail/cyrus-imapd | xargs scanelf -qn
> (In reply to SpanKY from comment #6) > > nm, i was looking at unittest files. the source includes pcreposix.h > explicitly, and then also links against -lpcreposix. that means those regex > symbols will be interposed at runtime over the glibc ones. > > double check which programs are actually linked against libpcreposix. > something like: > qlist net-mail/cyrus-imapd | xargs scanelf -qn I'm effected by this (net-mail/cyrus-imapd-2.5.10, use: berkdb clamav http pam perl server sieve sqlite ssl) and can provide the following information if it's helpful $ ldd /usr/lib64/cyrus/sievec linux-vdso.so.1 (0x00007ffff6176000) libcyrus_sieve.so.0 => /usr/lib64/libcyrus_sieve.so.0 (0x00007fd6f9114000) libcyrus_min.so.0 => /usr/lib64/libcyrus_min.so.0 (0x00007fd6f8eda000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fd6f8cbe000) libc.so.6 => /lib64/libc.so.6 (0x00007fd6f8925000) libcyrus_com_err.so.0 => /usr/lib64/libcyrus_com_err.so.0 (0x00007fd6f8722000) libcyrus.so.0 => /usr/lib64/libcyrus.so.0 (0x00007fd6f83e9000) libcrypto.so.1.0.0 => /usr/lib64/libcrypto.so.1.0.0 (0x00007fd6f7fa6000) libpcreposix.so.0 => /usr/lib64/libpcreposix.so.0 (0x00007fd6f7da3000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00007fd6f7a3e000) libical.so.1 => /usr/lib64/libical.so.1 (0x00007fd6f77d8000) libicalss.so.1 => /usr/lib64/libicalss.so.1 (0x00007fd6f75bf000) libicalvcal.so.1 => /usr/lib64/libicalvcal.so.1 (0x00007fd6f73a9000) libjansson.so.4 => /usr/lib64/libjansson.so.4 (0x00007fd6f719c000) libz.so.1 => /lib64/libz.so.1 (0x00007fd6f6f85000) /lib64/ld-linux-x86-64.so.2 (0x00007fd6f9335000) libsasl2.so.3 => /usr/lib64/libsasl2.so.3 (0x00007fd6f6d67000) libssl.so.1.0.0 => /usr/lib64/libssl.so.1.0.0 (0x00007fd6f6af8000) libdb-5.3.so => /usr/lib64/libdb-5.3.so (0x00007fd6f6739000) libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00007fd6f6425000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fd6f6221000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fd6f5faf000) libm.so.6 => /lib64/libm.so.6 (0x00007fd6f5cb3000) $ scanelf -qn /usr/lib64/cyrus/sievec libcyrus_sieve.so.0,libcyrus_min.so.0,libpthread.so.0,libc.so.6 /usr/lib64/cyrus/sievec $ qlist net-mail/cyrus-imapd | xargs scanelf -qn | grep libpcreposix libcyrus_com_err.so.0,libcyrus_min.so.0,libcyrus.so.0,libpthread.so.0,libcrypto.so.1.0.0,libpcreposix.so.0,libc.so.6 /usr/lib64/libcyrus_sieve.so.0.0.0 libxml2.so.2,libical.so.1,libicalss.so.1,libicalvcal.so.1,libpthread.so.0,libjansson.so.4,libpcreposix.so.0,libz.so.1,libc.so.6 /usr/lib64/libcyrus_min.so.0.0.0 libcyrus_com_err.so.0,libcyrus_min.so.0,libcyrus.so.0,libpthread.so.0,libcrypto.so.1.0.0,libpcreposix.so.0,libc.so.6 /usr/lib64/libcyrus_sieve.so libcyrus_com_err.so.0,libcyrus_min.so.0,libcyrus.so.0,libpthread.so.0,libcrypto.so.1.0.0,libpcreposix.so.0,libc.so.6 /usr/lib64/libcyrus_sieve.so.0 libxml2.so.2,libical.so.1,libicalss.so.1,libicalvcal.so.1,libpthread.so.0,libjansson.so.4,libpcreposix.so.0,libz.so.1,libc.so.6 /usr/lib64/libcyrus_min.so libxml2.so.2,libical.so.1,libicalss.so.1,libicalvcal.so.1,libpthread.so.0,libjansson.so.4,libpcreposix.so.0,libz.so.1,libc.so.6 /usr/lib64/libcyrus_min.so.0
Created attachment 468556 [details] net-mail/cyrus-imapd build.log The build.log from a build of cyrus-imapd
I have the same problem with cyrus-imapd-3.0.1, but it breaks much more than just sieve. All processes crashes with the same backtrace as in comment #1: lmtpd, imapd, reconstruct… just make cyrus-imapd unusable. Using the pcreposix patch above from debian fixes all the issues (after rebuilding all packages relying on libpcre of course)
hi all! TL;DR: First, this a me too! Second, though having no deep insight in what is actually going wrong: filtering --as-needed alters linking in a way that both sievec of 2.5 and much of 3.0.1 doesnt't crash with gentoo's unpatched libpcre. Long Version: On my System there are more binaries than cyrus' linked with libpcreposix. These usually don't crash using Gentoos unpatched libpcre. Having learned that the order of libraries when linking determines which (versions of) symbols are found, I believe I've observed that binaries that don't crash link libpreposix.so before libc.so: $ldd /usr/sbin/pcretest [...] libpcreposix.so.0 => /usr/lib64/libpcreposix.so.0 (0x00007feb1ae70000) [...] libc.so.6 => /lib64/libc.so.6 (0x00007feb1a655000) Whereas _all_ binaries form cyrus-imapd 2.5/3.0 that link libpcreposix seem to do it the other way round: $ldd sievec [...] libc.so.6 => /lib64/libc.so.6 [...] libpcreposix.so.0 => /usr/lib64/libpcreposix.so.0 This my vague idea of an issue in the process of linking cyrus' binaries. I tried playing around with cyrus' build system without any previous knowledge of autotools: it didn't like me:( However, filtering --as-nedded in the ebuild as decribed here: https://wiki.gentoo.org/wiki/Project:Quality_Assurance/As-needed prevents the crashes on my site. Furthermore the order of linking changes: without --as-needed: $ldd sievec [...] libpcreposix.so.0 => /usr/lib64/libpcreposix.so.0 [...] libc.so.6 => /lib64/libc.so.6 No further insights, sorry.... Markus
I can confirm, that filtering as-needed flag makes at least sievec working (only tested with cyrus-imapd-3.0.2).
filtering --as-needed works also for cyrus-imapd-2.5.11. Please revbump. ty!
Wouldn't it make sense to add this flag to the cyrus-imapd ebuild then? Yesterday I wanted to upgrade to 3.0.4 and everything just crashed. Because I didn't have the time to check for the reason I reverted to 2.5.10 and a patched library. This evening I will try again with the suggested filter.
Assigning to net-mail/cyrus-imapd maintainer.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7c8e1ea5dacdafacb4cf4d9f811a9a3146970f5c commit 7c8e1ea5dacdafacb4cf4d9f811a9a3146970f5c Author: Eray Aslan <eras@gentoo.org> AuthorDate: 2018-01-22 14:20:45 +0000 Commit: Eray Aslan <eras@gentoo.org> CommitDate: 2018-01-22 14:20:45 +0000 net-mail/cyrus-imapd: bump to 3.0.5 Filter --as-needed for working sieve implementation. Should really be resolved at toolchain level Bug: https://bugs.gentoo.org/604466 Package-Manager: Portage-2.3.20, Repoman-2.3.6 net-mail/cyrus-imapd/Manifest | 1 + net-mail/cyrus-imapd/cyrus-imapd-3.0.5.ebuild | 221 ++++++++++++++++++++++++++ 2 files changed, 222 insertions(+)}
I fail to see how filtering anything during cyrus-imapd build is the correct solution here. Toolchain guys really need to sort this out. But please check net-mail/cyrus-imapd-3.0.5 for an as-needed filtered build.
cyrus-imapd-3.0.5 works here with sieve-filtering enabled and being used. No manual patches were required
I can confirm that now 3.0.5 no longer crashes without a patch to libpcre. Closing the bug now.
