Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 452470 - mail-filter/opendkim needs optimization
Summary: mail-filter/opendkim needs optimization
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Net-Mail Packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-16 10:51 UTC by Christian Roessner
Modified: 2013-01-18 13:48 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
opendkim-2.7.4.ebuild (opendkim-2.7.4.ebuild,5.94 KB, text/plain)
2013-01-18 00:08 UTC, Christian Roessner
Details
strl patch grabbed from 2.7.2 build (opendkim-2.7.4-strl.patch,1.13 KB, patch)
2013-01-18 00:09 UTC, Christian Roessner
Details | Diff
git commit 23548465adccd682ba9ecba58025f852d2353bad (opendkim-2.7.4-disablecryptoinit.patch,3.39 KB, patch)
2013-01-18 00:10 UTC, Christian Roessner
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Roessner 2013-01-16 10:51:10 UTC
I have created my own overlay for mail-filter/opendkim.

I copied 2.7.2-ebuild to 2.7.4 and modified the strl-patch.

I have three issues:

1.

First of all, the dependency for strl is missing (dev-libs/libstrl)

Documentation requires flag --with-milter.

IUSE="+berkdb gnutls ldap lua +milter opendbx poll sasl +ssl static-libs unbound"

    econf \
        $(use_with milter) \
…


If using ldap, it might be interesting to add --enable-ldap_caching

    if use ldap; then
        myconf+=" $(use_with sasl)"
        myconf+=" --enable-ldap_caching"
    fi  


2.

If using sasl, you can use SASL/EXTERNAL for binding to the LDAP server. The init-Script does not support this. I had to manually add the following lines to get things working:

/etc/init.d/opendkim:


start() {
        # RNS: workaround startup issues, where ldap.conf is not read
        export LDAPTLS_CERT=/etc/ssl/certs/mx0.roessner-net.de.pem
        export LDAPTLS_KEY=/etc/ssl/private/mx0.roessner-net.de.key.pem
        export LDAPTLS_CACER=/etc/ssl/certs/ca-certificates.crt
        export LDAPTLS_REQCERT=demand

        check_cfg || return 1
    
        # Remove stalled Unix socket if no other process is using it
        local UNIX_SOCKET=$(sed -ne 's/^[[:space:]]*Socket[[:space:]]\+\(unix\|local\)://p' "${CONFFILE}")
…

Of course this is dirty hack and options should go somewhere to /etc/conf.d/opendkim


3.

If you have multiple instances of opendkim, you get segmentation faults when stopping a service. The problem is that instead of killing just the instance you want, the init script kills all opendkim processes. And this leads to a set fault.

Unfortunately if using Gentoo hardened, grsec might recognize this and interpret this as a brute force attack, blocking this software for 15 minutes. So of hardened systems the init script currently is a problem.


Summarize:

2.7.4 is working here perfectly


Portage 2.1.11.31 (hardened/linux/amd64, gcc-4.6.3, glibc-2.15-r3, 3.7.0-hardened x86_64)
=================================================================
System uname: Linux-3.7.0-hardened-x86_64-Intel_Core_2_Duo_P9xxx_-Penryn_Class_Core_2-with-gentoo-2.1
Timestamp of tree: Tue, 15 Jan 2013 22:30:01 +0000
ld GNU ld (GNU Binutils) 2.22
ccache version 3.1.8 [enabled]
app-shells/bash:          4.2_p37
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.7.3-r2, 3.2.3
dev-util/ccache:          3.1.8
dev-util/pkgconfig:       0.27.1
sys-apps/baselayout:      2.1-r1
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.5
sys-devel/autoconf:       2.68
sys-devel/automake:       1.11.6
sys-devel/binutils:       2.22-r1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.6 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo croessner
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs ccache compressdebug config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/ rsync://de-mirror.org/gentoo/"
LANG="C"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl adns aio amd64 bacula-clientonly bacula-console bash-completion berkdb bindist btrfs bzip2 caps cli cracklib crypt curl cxx device-mapper dri gdbm gpm hardened iconv ipv6 justify logrotate loop-aes lzo mmap mmx modules mudflap multilib ncurses nls nptl ntp openmp pam pax_kernel pcre pppd readline session sse sse2 ssl tcpd threads unicode urandom vim-syntax xattr zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" PHP_TARGETS="php5-3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Eray Aslan gentoo-dev 2013-01-16 11:25:34 UTC
(In reply to comment #0)
> First of all, the dependency for strl is missing (dev-libs/libstrl)

Not really.  Opendkim has an internal libstrl implementation which is cleaner and is probably faster than libstrl.  Opendkim uses the internal implementation when libsrl is not present.  Moreover, make check fails when opendkim is linked against libstrl (but passes with internal implementation).  This is the main reason I did not bump opendkim yet.  I need to find some time and figure out why make check fails when linked against libstrl.

> Documentation requires flag --with-milter.

I am not sure I understand.  I will check when I am at my dev box.

> If using ldap, it might be interesting to add --enable-ldap_caching

No.  It does not do what you think it does.  You will have to restart opendkim when ldap data changes with --enable-ldap_caching.

> Of course this is dirty hack and options should go somewhere to
> /etc/conf.d/opendkim

I will have a look.

> If you have multiple instances of opendkim, you get segmentation faults when
> stopping a service.

That looks like a problem with start-stop-daemon not with opendkim.
Comment 2 Christian Roessner 2013-01-16 14:03:39 UTC
I removed --with-milter, --enable-ldap_caching. Also removed strl package and rebuilt.

Here is the init script segfault problem:

[83139.618188] opendkim[4687]: segfault at 1f0 ip 000072d6962be394 sp 00007401911a9c60 error 4 in libpthread-2.15.so[72d6962b4000+18000]
[83139.618216] grsec: Segmentation fault occurred at 00000000000001f0 in /usr/sbin/opendkim[opendkim:4687] uid/euid:105/105 gid/egid:998/998, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0
[83139.618286] grsec: bruteforce prevention initiated against uid 105, banning for 15 minutes
Comment 3 Christian Roessner 2013-01-17 14:39:19 UTC
(gdb) continue
Continuing.
[Thread 0x7051a9b60700 (LWP 15401) exited]

Program received signal SIGSEGV, Segmentation fault.
0x00007051ace02394 in pthread_mutex_lock () from /lib64/libpthread.so.0
(gdb) bt
#0  0x00007051ace02394 in pthread_mutex_lock () from /lib64/libpthread.so.0
#1  0x00000e96a32a99e0 in ?? ()
#2  0x00007051ad157bc7 in CRYPTO_add_lock () from /usr/lib64/libcrypto.so.1.0.0
#3  0x00007051ad40c5c0 in SSL_CTX_free () from /usr/lib64/libssl.so.1.0.0
#4  0x00007051ae09fee6 in ldap_int_tls_destroy () from /usr/lib64/libldap-2.4.so.2
#5  0x00007051af1baa7c in ?? () from /lib64/ld-linux-x86-64.so.2
#6  0x00007051aca855d1 in ?? () from /lib64/libc.so.6
#7  0x00007051aca85625 in exit () from /lib64/libc.so.6
#8  0x00007051aca6f494 in __libc_start_main () from /lib64/libc.so.6
#9  0x00000e96a32940d1 in ?? ()
#10 0x00007b2529531cf8 in ?? ()
#11 0x000000000000001c in ?? ()
#12 0x0000000000000003 in ?? ()
#13 0x00007b2529531f7d in ?? ()
#14 0x00007b2529531f90 in ?? ()
#15 0x00007b2529531f93 in ?? ()
#16 0x0000000000000000 in ?? ()
(gdb) stop
(gdb) 

UP TO HERE, grsec is _not_ blocking opendkim.

(gdb) quit
A debugging session is active.

	Inferior 1 [process 15399] will be detached.

Quit anyway? (y or n) y
Detaching from program: /usr/sbin/opendkim, process 15399

NOW opendkim gets blocked (brute force detection)

Do you want to contact Murray? I have subscribed opendkim-users and I know Muray personally, so I could figure things out and ask to get things fixed? But I do not want to do this, unless you want to.
Comment 4 Eray Aslan gentoo-dev 2013-01-17 15:04:42 UTC
(In reply to comment #3)
> Do you want to contact Murray? I have subscribed opendkim-users and I know
> Muray personally, so I could figure things out and ask to get things fixed?

Please do contact upstream.  It would be a great help.

You will need to get a backtrace with debugging symbols though.  The following might help:
http://www.gentoo.org/proj/en/qa/backtraces.xml
Comment 5 Christian Roessner 2013-01-17 23:11:54 UTC
Upstream is informed. It is a known bug:

https://sourceforge.net/tracker/?func=detail&atid=1147701&aid=3531477&group_id=269812

Murray is soon starting 2.8 betas. I asked to get a back port patch for 2.7.4

His answer:

This is a known problem for installations that use openldap with opendkim. openldap goes through some steps to ensure that libcrypto is set up with mutexes as openssl requires, but opendkim does the same set of steps; during shutdown, both of them call the opposite routine to free those resources but that results in a double-free and/or heap corruption, and you get this crash.

I contend that openldap shouldn't be doing this; libraries shouldn't be initializing each other, as that's the job of the application.  But understanding that openldap is probably not as agile as we are and may disagree, opendkim 2.8.0 includes the option to skip the libcrypto setup steps in order to avoid this problem.

I will be starting 2.8.0 betas soon, so you can give this option a try in the near future.
Comment 6 Christian Roessner 2013-01-18 00:08:38 UTC
Created attachment 335976 [details]
opendkim-2.7.4.ebuild

This build is not perfect, as I used patch and not patch for one patch. I don't know how to get this done correctly.
Comment 7 Christian Roessner 2013-01-18 00:09:29 UTC
Created attachment 335978 [details, diff]
strl patch grabbed from 2.7.2 build
Comment 8 Christian Roessner 2013-01-18 00:10:43 UTC
Created attachment 335980 [details, diff]
git commit 23548465adccd682ba9ecba58025f852d2353bad

I have removed the RELEASE_NOTES part, as "patch" did not succeed.
Comment 9 Christian Roessner 2013-01-18 00:11:50 UTC
By adding "DisableCryptoInit yes" to your configuration, start/stop will no more seg fault, if using openldap.
Comment 10 Eray Aslan gentoo-dev 2013-01-18 13:48:28 UTC
Thanks for your report.

+*opendkim-2.7.4 (18 Jan 2013)
+
+  18 Jan 2013; Eray Aslan <eras@gentoo.org>
+  +files/opendkim-2.7.4-DisableCryptoInit.patch,
+  +files/opendkim-2.7.4-bsd.patch, +files/opendkim.init.r3,
+  +opendkim-2.7.4.ebuild:
+  Version bump - bug #452324. Start before mta - bug #451114. Use libbsd instead
+  of internal library - bug #441790.  Add option to disable crypto
+  initialization thanks to Christian Rößner - bug #452470.
+