| Summary: | qmail-1.03-r16 | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Jens Ott <jo> |
| Component: | New packages | Assignee: | Qmail Team (OBSOLETE) <qmail-bugs+disabled> |
| Status: | VERIFIED TEST-REQUEST | ||
| Severity: | enhancement | CC: | caluml, christophe, eoin-gentoobugs, fuzz, gentoo-bugs, jo, per.sil, rajiv, slyoldfox |
| Priority: | High | ||
| Version: | 1.4_rc4 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 56124, 83415, 87695, 98961, 99497, 99598, 101532, 103701, 103704 | ||
| Bug Blocks: | 29485 | ||
| Attachments: |
New ebuild and related files (patches)
qmail-1.03-r16-spp.tar.gz qmail_vpopmail_scripts.2004-12-10.tar.gz qmail-dk-0.1.tar.gz qmail-1.03-r16-spp.2004-12-12.tar.gz qmail-dk-0.1-r1.tar.gz qmail-dk-0.2.tar.gz |
||
|
Description
Jens Ott
2004-02-05 08:03:39 UTC
Created attachment 25014 [details]
New ebuild and related files (patches)
This tar.bz2 contains all modified files to the qmail-Ebuild
I've updated the qmail-spf patch. Just three minor improvements. You can use that instead before releasing r16: http://www.saout.de/misc/spf/qmail-1.03-r15-spf-pre2.patch Sorry again. I just noticed that my rediff was broken (I shared err_smf with the mfcheck patch which I did not include in the rediff). I updated the -pre2 patch to actually compile. One person downloaded it using wget, don't know who, but the referrer shows he came from the bugzilla page. My apologies. the ChkUsr-patch will NOT be integrated. I'm in the process of designing a better method for ChkUsr that will use an external program (eg not integrated into qmail-smtpd) to check if users exist and behavior to take. the primary reasoning for this is so that it can be used with qmail-mysql and qmail-ldap as well without much extra work. sorry, i haven't got to the chkusr stuff I wanted to do yet. What about http://qmail-spp.sourceforge.net/ Do you think it is worth including it? qmail-smtp calls external programs to check the envelope. Thus you may check the sender and each recipient or any combination. This gives the flexibility to use a module for qmail-ldap and another one for qmail-mysql. wow, that's MUCH better than chkusr. it's the more generic type of thing that I was looking for :-) i'll take a look at it when I get a moment, as that looks to be the ideal route. chkusr would just be a plugin for it :-). Created attachment 44144 [details] qmail-1.03-r16-spp.tar.gz [new ebuild package] added the following patches to qmail-1.03-r15 - Domain Keys http://qmail.bec.at/qmail-1.03-dk-0.53.patch this patch adds a queue-replacement "qmail-dk" that creates and verifys mail header signatures. It is not activated by default. You have to replace qmail-queue manually. Using QMAILQUEUE environment variable does not help if you use qmail-scanner! Can be removed safely without breaking the other patches. - SPP (SMTP plugin framework) http://qmail-spp.sourceforge.net/ for checking recipient or sender addresses via external tools. The SPP patch can be applied independently from domain keys!! - logging any relay attempts http://www.palomine.net/qmail/logrelay.patch patch had to be changed to be usable. Original only applied to vanilla sources - mfcheck patch (in deactivated modus - if you want it, you must uncomment it) http://www.qmail.org/qmail-1.03-mfcheck.3.patch The patch had to be adapted to be able to apply it to the current r15 release. Original patch only works with vanilla sources I do not want to interfere with r15 so I created a new version r16. Dear Robin, please review the added patches. I hope some of them are worth including in portage tree. Regards Perolo - SPP (SMTP plugin framework) sounds very interesting imho. Can't wait to have it in portage! Finally something that will allow *anything* to be checked, with this patch I can even write a plugin to auto-msg me on IRC/IM when I am online and when rpct == me ;) Great plugin! that looks good, but the method for having qmail-scanner (or anything other qmail-queue plugin) along with qmail-dk looks to be quite complex - any ideas on simplifying it? this is esp. important for cases where they already use more than one qmail-queue plugin. mfcheck question - why not include the patch and just have a 0 in /var/qmail/control/mfcheck ? *** Bug 55492 has been marked as a duplicate of this bug. *** Using qmail-dk with qmail-scanner can be done like this: 1) rename qmail-queue to qmail-queue.orig. replace qmail-queue with qmail-dk. qmail-dk either checks DKQUEUE which queue to use or uses qmail-queue.orig as default. Since qmail-scanner uses qmail-queue to deliver mails, this works with qmail-scanner. => You activated qmail-dk 2) qmail-dk still does nothing unless DKSIGN (to create a signature) or DKVERIFY (to verify incoming email) is set. DKSIGN is prefered over DKVERIFY, so if DKSIGN is set the message is signed in case no signature exists, no matter if DKVERIFY is set. So you must ensure, that only your authorized clients get DKSIGN set for qmail-smtpd. Unfortunately I found no way to achieve this using relay-ctrl. Hence I patched relay-ctrl (see Bug Bug 71605). DKSIGN gets through relay-ctrl-check This solves the problem with POP/IMAP before SMTP but not with SMTP-AUTH. I will play around tomorrow to see how to do the trick. Perolo: any update on the stuff you were looking at? *** Bug 40307 has been marked as a duplicate of this bug. *** Created attachment 45663 [details] qmail_vpopmail_scripts.2004-12-10.tar.gz I am conducting in-deep tests on qmail to run with + SpamAssassin (via qmail-scanner) + procmail (for filtering) + qmail-dk (domainkeys) + qmail-spp + vpopmail + SPF (sender policy framework via SpamAssassin) I will try to send a patch to the maintainer of the Gentoo-HowTo on this subject. Meanwhile I post these two new scripts: **** check_rctp_existence.sh .... checks the existence of a mailbox during SMTP session. Each "RCPT" command is checked. To use this you need: 1) the qmail-spp patch 2) set the SUID-bit of /var/vpopmail/bin/vuserinfo chmod u+s /var/vpopmail/bin/vuserinfo this is necessary because qmail-smtp runs as user qmaild but vuserinfo needs to run as root - I do not why and see no reason for it but can't change it right now. 3) add the following lines to /var/qmail/control/smtpplugins --------------- [rcpt] :/var/vpopmail/bin/check_rctp_existence.sh --------------- This is my first example on how to implement checks with qmail-spp. It is quite simple :) but working perfect with vpopmail. I have already posted this script to the maintainer of qmail-spp. **** qmail-queue.dk.pl (for qmail with domainKeys patch) .... integrates domainKeys with qmail. You need the qmail-spp patch too, because this patch makes qmail-smtpd set the environment variable "SMTPAUTHUSER" if the user has authenticated. You may use "RELAYCLIENT" instead but I do want to avoid side effects. If you have some client-IPs that are allowed to relay because they are on your internet net but infected by a spammer-trojan, these emails would get signed too. So I think SMTPAUTHUSER is the far better solution to rely on. To use it you have to do the following steps: 1) <save qmail-queue.dk.pl to /var/qmail/bin> 2) edit qmail-queue.dk.pl to set the proper path to the key with DKSIGN 3) cd /var/qmail/bin; mv qmail-queue qmail-queue.orig 4) cp qmail-queue.dk.pl qmail-queue; chown qmailq:qmail qmail-queue ... this is important to use qmail-scanner with qmail-dk - the only alternative is to patch qmail-scanner but I did not want to patch everything :) qmail-queue.dk.pl reads all AUTH_(.*) environment variables and sets all variables with name $1 to these values. I thought of setting AUTH_DKSIGN=.... in conf-smtpd but it is not passed to qmail-queue.dk.pl. I will adapt qmail-queue.dk.pl to read a configuration file in /var/qmail/control to be able to set the proper key based on the sender. Then a mail-roaster is able to use different keys for each sender or one for a group of senders - much flexible. So see this version as a pre-alpha testing script. If you have an alternative, please post it. Another word on domain-keys. I found it tricky to set the proper DNS entries, because I had overseen, that you MUST USE selectors. So you need two DNS settings: 1) _domainkey.your.domain ... to set the policy of your domain probably: "o=~; r=postmaster@your.domain; t=y" 2) a selector to retrieve the key from your domain. The name of the private key file is used as a selector by qmail-dk. Therefore your private key file name MUST be identical with the selector you choose. eg: testing._domainkey.your.domain is the base-name of the private key file and the name of the DNS text entry, reading somewhat: "k=rsa; p=<YOUR base64 encoded public key>.." (see http://domainkeys.sourceforge.net and "man qmail-dk" on how to create these entries.) See this example of mine (with djbdns): ----------djbdns data file --------------- # general policy '_domainkey.geber.at:o=~; r=postmaster@geber.at; t=y:600 # selector for the proper key 'buero._domainkey.geber.at:k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC8he41Jqd8LDpHpoGlU7S8Z/3d23TUnPa99a9pHoPhKTi/uQZy1klhW36sj8aYrkjhCTWx4/ukq3QjeuT2PSB21136+KBpQQ9KYrYeCxgmTKDQFoRkQngIcdVg1FxG8f2xNPsKyIZKosaBON+bM3YwnsSnc4b3p8gOxx5me3cdYwIDAQAB:600 ------------------------------------------ my key file is: /var/qmail/control/buero._domainkey.geber.at This works for me but testing is not finished yet. I start the test phase by Monday. Then 5 users will use my mail-server on a daily-basis. This will tell if the setup is robust enough. PS: I found it quite tricky to use procmail with qmail. All hints I have found on the Net do not work. So I created my own scripts to work with procmail. Since then procmail seems quite valuable if you want to write the email to different folders based on email headers. (especially a separate SPAM-folder) I will publish these scripts separately with the HowTo. procmail isn't hard to use with qmail, just put this in your .qmail file: "|IFS=' ' && exec /var/qmail/bin/preline /usr/bin/procmail || exit 75" My .procmailrc is here: http://www/~robbat2/sample.procmailrc From your latest comment, there are a few things I'd like to comment on. 1. I'm trying to move away from files in /var/qmail/bin EVER containing configuration settings that are modified by the user (since that directory isn't under CONFIG_PROTECT). This means the settings inside qmail-scanner.pl and qmail-queue.dk.pl need to move somewhere else (perferablly under /var/qmail/control). 2. for settings with the potential for a lot of sub-files, they should go into a new directory. Eg: /var/qmail/control/buero._domainkey.geber.at should move to /var/qmail/control/dk/buero._domainkey.geber.at 3. Every setting should need only a configuration knob - moving qmail-queue files around for DK is not suitable, there is too much potential for things getting broken. A better means must be found. Robin: I am working on it. I totally agree with you that no configuration should be set in a file in qmail/bin. At the moment it is just for testing. I will move the keys and settings to /var/qmail/control/dk
If necessary I will create a patch to qmail-scanner to pass an environemt variable to qmail-scanner-queue.pl, which queue-program to call. Is "QSCANNERQUEUE" OK? Then we do not have to move/rename any binary. The scenario would be:
1) QMAILQUEUE defines /var/qmail/bin/qmail-scanner-queue.pl
qmail-scanner is called
2) QSCANNERQUEUE defines /var/qmail/bin/qmail-queue.dk.pl
qmail-scanner calls qmail-queue.dk.pl instead of qmail-queue
3) qmail-queue.dk.pl calls /var/qmail/bin/qmail-dk
4) DKQUEUE defines /var/qmail/bin/qmail-queue
qmail-dk calls qmail-queue
There will be a configuration file "/var/qmail/control/dk/usermap.conf", each line of the form:
user-regex: keyfile
*) user regex will be matched agains the value of SMTPAUTHUSER or POPAUTHUSER or wherever the username will be available.
*) the keyfile will be relative to /var/qmail/control/dk/, or absolute if starting with "/".
*) the authenticated user does not match any entry, then "/var/qmail/control/dk/default._domainkey.%" is used. qmail-dk replaces "%" with the domain name of the sender.
*) in the next step, qmail-queue.dk.pl will be rewritten in C using similar techniques like qmail (the username map of qmail).
The only thing left is: relay-ctrl needs to be checked to work properly with IMAP and POP3. The username is not added to the IP-file on my mail-server. The name of the authenticated user must be written to the environment file too and must be available on next SMTP session. qmail-queue.dk.pl will look for it to enable domainKeys with IMAP/POP-before-SMTP.
If anyone can provide some work on it, it would be great. I will certainly need some time :)
Step one finished, see Bug 73994 qmail-scanner-queue.pl patched Created attachment 45793 [details] qmail-dk-0.1.tar.gz OK. I have worked two days and a night :) This package contains everything any qmail package (qmail, qmail-ldap, qmail-mysql, ...) need to support domainKeys. - qmail-dk the original implementation - qmail-queue-dk an application to prepare the environment before calling qmail-dk - a patch to qmail-scanner-queue.pl to read the next app to call from $ENV{'QSCANNERQUEUE'}. usage: qmail-queue-dk is an alternative queue app. It can be called directly by qmail if you set QMAILQUEUE="/var/qmail/bin/qmail-queue-dk". If the user has authenticated then DKSIGN is set to the proper key. /var/qmail/bin/qmail-dk is called to create or verify the message signature and pass on the message to qmail-queue. The user has authenticated if SMTPAUTHUSER has been set. Hence you need the qmail-spp patch or a similar patch to qmail-smtpd. The original SMTP-AUTH patch does not set any environment variable. control-flow of qmail-queue-dk: ------------------------------- The user name is read from SMTPAUTHUSER or POPAUTHUSER 1) check if the user has authenticated. If not step to 2) else read all env vars with leading "AUTH_", and copy their values to vars with "AUTH_" stripped from the name. 2) load global file for environment settings "/var/qmail/dk/anonymous.env" for anonymous users "/var/qmail/dk/authenticated.env" for authenticated users 3) check if the user has authenticated. If not step to 5) else match the username against regular expressions in /var/qmail/control/dk/userkey.map file. Read the settings from the first matching line. Ignore all other lines. a) in case a key file is set, use it b) if a file name for additional environemt values is set, read it 4) ensure DKSIGN is properly set 5) ensure DKQUEUE is properly set 6) call qmail-dk userkey.map ------------------------------- the user name is matched against userkey.map file. empty lines or lines with a leading '#' are ignored. Each line start with a regular expression to match the user name. The regular expression is immediately followed by a colon ':', some whitespace the user setting. The user setting can be *) <the name of the key file> *) env=<the name of a file to read environment settings from> *) key=<the name of a key file> See this example (with vpopmail): ------------ .*@bugs.gentoo.org: env=gentoo_bugs_users.env .*@.*.gentoo.org: key=default._domainkey.gentoo.org .*@gentoo.org: default._domainkey.gentoo.org ------------ the user name depends on your qmail-installation. The above example assumes you have a virtual mail manager like vpopmail installed. the key file and the environment file can either be absolute or relative. If relative (no leading "/"), then "/var/qmail/control/dk/" is prepended. Before qmail-dk is called, DKSIGN is checked to see if it is absolute or relative. If it is relative, "/var/qmail/control/dk/" is prepended. if DKQUEUE is relative, then "/var/qmail" is prepended. TODO ----- If the user authenticates via POP3, POPAUTHUSER must be set. relay-ctrl does NOT do this for you. I will fix this later. relay-ctrl seems to set no env var with the user name, at least not on my system with courier-IMAP. IMPORTANT NOTES --------------- BEWARE that qmail-dk refuses to pass the message on to qmail-queue if DKSIGN is set but the key file does not exists. Watch the log-files if you adapt your settings. qmail-queue-dk write a note which key file has been set. During my tests I found a bug with qmail-smtpd that prevents qmail-queue* programs to log messages to STDERR as soon as a user authenticates. qmail-queue-dk writes some messages to detect missing key files. So please ensure proper patch to qmail-smtpd is applied. See Bug 74124 this package relies on qmail-smtpd to set the environment variable SMTPAUTHUSER. qmail-spp patch adds this feature. PLEASE consider this package as pre-alpha. It needs thoroughly testing as all other patches that this packes relies on. I have conducted my tests with qmail-scanner installed. Please report any bugs. The included ebuild is working but no thrill. I think proper place for the package would be "mail-filter". Created attachment 45810 [details] qmail-1.03-r16-spp.2004-12-12.tar.gz since support for domainkeys has its own package now, I removed these patches and created a new package. The patch from bug 74124 is included. Created attachment 47308 [details] qmail-dk-0.1-r1.tar.gz Thanks for the new qmail packackage. I have been waiting for this :) I managed to get domainkeys working, although I had to patch the library. The new qmail-dk package uses the new features to add a 'h=' header line to the signature header. See Bug 71501 and: https://sourceforge.net/tracker/index.php?func=detail&aid=1093952&group_id=107680&atid=648373 Since some mail-filters seems to reposition some header lines, the new environment variable DKIGNORE has been introduced. If you use qmail-scanner then add the following line to conf-smtpd export DKIGNORE="X-Spam-Level:X-Spam-Status:X-Spam-Checker-Version:X-Spam-Checker" This makes qmail-dk ignore the qmail-scanner header lines when calculating the signature. (Checking is performed case insensitive) There still may be bugs in my set of patches. I regard them as pre-alpha. They work for me at the moment but may not work for you. If you find any bugs, please let me know. Happy new year!! Hello! I have added a new ebuild qmail-1.03-r16 with the spp-patch to portage. It is currently hardmasked. I hope to be able to work on qmail in the next days. Greets, Michael Created attachment 47618 [details]
qmail-dk-0.2.tar.gz
Hi,
I have created a new, clean ebuild. All unnecessary patches have been removed
and a man page for qmail-queue-dk has been created. I think this is it.
This ebuild should work with qmail, qmail-mysql and qmail-ldap. Dependencies
have been set accordingly. Since qmail-dk depends on the SPP patch-set,
versions for qmail-ldap and qmail-mysql to depend on have been set to future
versions. I hope they will include qmail-spp then.
A word to domain-keys compatibility:
There are two big issues with this domainkeys implementation:
1) mail filter may add header lines at arbitrary position that could break
the signature.
2) only information about the signed headers ('h=' list) would help. This
implementation adds this info. But I think the default implementation of
parsing this list is errorneous when multiple headers of the same names are
signed (eg: 'Received' - see my patches to libdomainkeys). To be compatible
with other implementations in the wild, either use qmail-queue-dk as the first
queing program (QMAILQUEUE points to qmail-queue-dk and qmail-dk calls
qmail-scanner) to be sure only ONE 'Received' line is available or exclude the
'Received' line from being signed. To exlude it, add "Received" to DKIGNORE.
I will keep you updated in a new bug report on this subject.
Perolo
PS: Thank you, Michael, for the new qmail ebuild.
I'll take a look into DomainKeys soon, the only thing I know currently is, that it's from Yahoo. Thanks for your work already! Greets, Michael I had to take out the spp, mfcheck and logrelay patches because I've replaced the TLS-patch. That has broken a lot of other patches too and needed alot of rediffes. I'm negotiating with Perolo who does the rediffes for his patches. any estimation on when the spp-patch is being added back to r16? I would like to test this a bit, I really need some easy way to chk_usr and reject mails that are currently just being accepted on my MX2 *and* taking tons of cpu to be scanned .. Hello Marc Since the original author didn't mail me or attach something to this bug, I'm going to rediff his patch. However, that will take some days, because I'm currently busy with the 2005.0 release for ppc. Greets, Michael Hi Michael, That would be awesome. No hurries, can't really have it faster anyhow unless i'd do it myself, only, i don't have that much experience ;) Meanwhile I'll just let my test machine idle a bit .. and script some more on my custom chkusr scriptie. Tnx a bunch, again! Here is another patch that may be applicable, but possibly only to qmail-ldap. #62526 Rediffed spp, mfcheck and logrelay patches are in CVS. Resync, remerge qmail-1.03-r16 and have fun! I just tried out qmail-1.03-r16 and it seemed to break something with tls - clients using smtp with tls and authentication (vpopmail auth) timed out and failed sending (thunderbird client). The config works fine on r15 (using noauthcram). Is there some configuration that might be needed for tls on r16? Otherwise, any hints on how to debug the problem? The 'check_rctp_existence.sh' script is not working.
Look at the following lines from this script:
if [ ! -z "`/bin/ls ${HOMEPATH}/.qmail-${BOX}`" ]; then
[ "${SPP}" == "1" ] && echo "${MSG_OK}"
exit 0
fi
The script runs with qmail rights and $HOMEPATH is something like /home/vpopmail/domains/foobar.tld and that gives you a permission denied! The qmail user can't get into the vpopmail directory. So bad luck.
So this is not working. B-(
I converted the ChkUsr-Patch V1.X into a seperate binary. Now I got a C program that will dynamically check for a) catchall domains b) .qmail-xxx files c) check for alias c) check for existence of the user d) mailing lists like the original ChkUsr-Patch V1.X did. The checker exits with code 0 (bad user) or 1 (good user) and must run as vpopmail user. That are the positives things. The negatives are: - no real testing yet (rebuilding our mail server is in progress). Testing on the command line worked well. - the binary uses the bstring library from http://bstring.sourceforge.net/. Sorry, but I don't want to transfer any qmail original source to this program (due to Bernsteins strange license) and I don't want to get some buffer overflows within the C "string" handling so I made this decision. - the code is ugly but working - binary is ~80 kb (Linux) The program is definitive not in the condition for a release but any testing person is welcome. If the code works in the real world I will made post the source code to this bug If someone is interested. If someone is interested directly in the source please drop a mail. Comments are welcome too. Thanks for listening! B-) Include the qms-analog patch also, it's somewhat done in bugs 76256 and 76257, also their site is, http://www.qms-analog.teel.ws/ This is a must have to me... qmail-1.03-r16 is now in ~ARCH. Please test it as much as you can and report here wether it works or not. About qmail-analog: That needs to be in its own ebuild. It's not related to qmail directly. Due to some modifications of the ebuild (added or removed patches), all who have -r16 already installed should reinstall it again. Target for stabelizing it is mid of August to end of August. No more patches will be added, only bug fixes will be done on -r16. Hello, Has anyone tried or considered implementing the SPAMCONTROL patch: http://www.fehcom.de/qmail/spamcontrol.html ? Fred (In reply to comment #37) > Has anyone tried or considered implementing the SPAMCONTROL patch? Please open a bug and let it block bug 95892. -r16 will not get new patches. qmail-1.03-r16 is due to stable on 2005-10-20 Stable on hppa, mips, ppc, sparc and x86. Closing |
