From the DSPAM mailing list: > Greetings, > > I know it's been a long time since the last DSPAM release, and I've > decided - after much testing over the past year - that it was good > and appropriate to release the changes in CVS. Because the > configuration has changed to some degree, and new features have been > added, I thought it best to call this release 3.8.0 instead of 3.6.9, > as I had been originally planning. Don't be fooled, though - it's > very similar to 3.6, but has been optimized significantly and some > important bugfixes have been made. It also sports some new features > such as Bill Yerazunis' Orthogonal Sparse Bigram <OSB> tokenizer. > > There are some very positive things happening with the project, and > I'll be making an announcement soon that I think will answer a lot of > questions and help the project move forward again out of the small > slump it's been in. Until all ducks are in a row, though, I'm going > to release 3.8.0 as an unofficial release. You can download it here: > > http://www.zdziarski.com/projects/dspam/sources/ > MD5 (dspam-3.8.0.tar.gz) = 056b8c8b3ad9415a52c01b22ff1e64cf > > Or grab it from cvs.zdziarski.com (I haven't yet committed every last > detail into the production CVS). > > Jonathan Reproducible: Always
Created attachment 115094 [details] mail-filter/dspam/dspam-3.8.0.ebuild
Created attachment 115095 [details] mail-filter/dspam/files/3.8.0-dspam.cron
Created attachment 115096 [details] mail-filter/dspam/files/dspam.rc
Created attachment 115098 [details] mail-filter/dspam/files/logrotate.dspam
Created attachment 115099 [details] mysql_purge-4.1-optimized.sql If some one wants to have faster purging with MySQL then use this SQL file and execute this inside the DSPAM database: alter table dspam_token_data add index(spam_hits); alter table dspam_token_data add index(innocent_hits); alter table dspam_token_data add index(last_hit); Copy the SQL file to your /etc/mail/dspam/ directory. The included cron script will pick up the proper SQL file when purging. More info about the script: http://securitydot.net/txt/id/32/type/articles/
Created attachment 115101 [details] distfiles/dspam-3.8.0-patches-20070331.tar.gz I know, I know... binary files should be avoided in Bugzilla. The binary has the same patches as the 3.6.8 version in Portage except that patch 11 is not needed anymore and a new patch (number 16) was added.
I prefer to wait for this to become official (plus I'm very busy atm), but if someone else wanna perform this bump, it's okay by me. Careful though, there are other open dspam bugs in my input tray.
(In reply to comment #7) > I prefer to wait for this to become official (plus I'm very busy atm), but if > someone else wanna perform this bump, it's okay by me. > > Careful though, there are other open dspam bugs in my input tray. > Okay. I understand your point. What open DSPAM bugs do you have in your input tray? Maybe I could help?
I was talking about bug 172781. Nothing to be done there except applying the patch and maybe modifying the cron script. You forgot about www-apps/dspam-web. This should also be bumped, although I don't know if there is something changed in it.
(In reply to comment #9) > I was talking about bug 172781. Nothing to be done there except applying the > patch and maybe modifying the cron script. > > You forgot about www-apps/dspam-web. This should also be bumped, although I > don't know if there is something changed in it. > So you want the cron script to use /bin/sh or another shell instead of bash? Is that right?
Created attachment 115556 [details] mail-filter/dspam/files/3.8.0-dspam.cron - uses /bin/sh instead of /bin/bash - handles correctly missing *.data files - processes SQLite and SQLite3 databases
Created attachment 115578 [details] mail-filter/dspam/files/3.8.0-dspam.cron - added support for remote MySQL server
Created attachment 115579 [details] mail-filter/dspam/files/3.8.0-dspam.cron - added remote MySQL support when running "OPTIMIZE TABLE" commands
Created attachment 115582 [details] mail-filter/dspam/files/3.8.0-dspam.cron - fixed typo in SQLite3 purge part
Created attachment 115584 [details] www-apps/dspam-web/dspam-web-3.8.0.ebuild The requested dspam-web 3.8.0 ebuild
Created attachment 115637 [details] mail-filter/dspam/dspam-3.8.0.ebuild - fixed naming issue with cron script - added cron script for purging old user log entries
Created attachment 115676 [details] mail-filter/dspam/files/3.8.0-dspam.cron Added dspam_logrotate instead of deploying two cron scripts.
Created attachment 115678 [details] mail-filter/dspam/dspam-3.8.0.ebuild Removed separate cron job for dspam_logrotate (dspam_logrotate is now included in 3.8.0-dspam.cron).
What else is missing now? Is there anything other needed to be fixed or included?
Created attachment 116391 [details] distfiles/dspam-3.8.0-patches-20070416.tar.gz
Created attachment 116392 [details] mail-filter/dspam/dspam-3.8.0.ebuild
Created attachment 116394 [details] www-apps/dspam-web/dspam-web-3.8.0.ebuild
BTW: DSPAM 3.8.0 is now official -> http://dspam.nuclearelephant.com/download.shtml Version 3.8.0 ------------- [20061210.1435] jonz: fixed message corruption problems with direct delivery when using direct delivery (e.g. DeliveryHost), certain servers require a linefeed after carriage return otherwise the message will become malformatted. [20060818.0700] jonz: added msg tagging support added ability to add tagline to messages based on their classification; see tagSpam and tagNonspam preferences in README [20060607.1200] jonz: removed depricated oracle driver removed outdated oracle driver; no maintainer, lack of interest [20060606.0000] jonz: added ldap client to build added ldap client headers to makefile, would not build on some systems [20060601.0500] jonz: fix for dynamic storage drivers api fixed _ds_pref_del call to storage library [20060601.0300] jonz: webui history fix for 12:00 noon bugfix to display 12 noon as 12p, not 12a [20060530.0145] jonz: added connect check for pgsql added a connection check for pgsql, to reconnect on failure in daemon mode [20060530.0130] jonz: added logging of viruses added logging of viruses (and the source) to agent [20060527.1700] jonz: added HashPctIncrease option in dspam.conf HashPctIncrease: Increase the next extent size by n% from the size of the last extent. The default behavior, when HashPctIncrease is not used, is to always use HashExtentSize with no increase. This is useful in accommodating systems where the default HashExtentSize can be too small for certain high-volume users. [20060527.1530] jonz: cache runtime user information added caching of runtime user information, so this information is not polled every message when running in daemon mode. also elimiates the need for getpwuid_r when running in daemon mode (unless using mysql or pgsql), which some operating systems do not have. [20060527.1530] jonz: moved TIME_ME into DEBUG when debug is active, TIME_ME automatically runs, reporting processing time to debug [20060526.1900] jonz: fix for library TIME_ME measurements fixed bug where negative processing times were reported using TIME_ME [20060526.1600] jonz: turned off locking when not using syslog or logging no need to lock on LOG() when not logging [20060526.0230] jonz: rewrite for hash_drv offset caching rewrote offset caching in hash_drv; fixed some bugs which may have caused a crash on extent addition [20060525.1100] jonz: fix for segfault on undefined DeliveryHost or ClientHost fix for segfault in daemon mode when DeliveryHost or ClientHost is not specified [20060524.0300] jonz: added --client support for dspam_train use --client after username [20060523.0300] jonz: more code optimizations various optimizations to: - tokenizer core - hash_drv driver (store offset for writes) - libdspam (preference lookups) - optimizations for osb/sbph [20060522.0300] jonz: added ProcessorURLContext ProcessorURLContext creates Url* context-specific tokens for URLs; this is the default in previous (and current) versions [20060522.0300] jonz: optimized osb/sbph tokenizer replaced several strlcat's with simple len counting to eliminate thousands of unnecessary calls to strlen() and speed up osb/sbph tokenization process [20060519.0300] jonz: fix for segfault in vsyslog() fix segfault caused by bad use of va_args when vsyslog is called [20060519.0130] jonz: fix for segfault in dlopen() failure fixed bug causing segfault when dlopen() to storage driver library fails. dspam still won't work any better if dlopen is failing but huzzah. [20060519.0100] jonz: fix for performance template / local domain added fix to display correct local domain in performance template, and only display local domain if the username doesn't include an @ sign [20060517.0700] jonz: fix for preference delete fixed infinite loop on all non-preference-extension calls to delete a preference [20060516.0200] jonz: changed SupressWebStats SupressWebStats is now WebStats in dspam.conf, and setting is inverted. [20060516.0200] jonz: fix for agent flags discovered that agent flags required a 64-bit variable to hold all flags, but only 32-bit variable was being used; this may have caused unpredictable behavior when using SBPH, "unlearning" a message, or processing summaries. [20060516.0200] jonz: added OSB tokenizer osb (orthogonal sparse bigram) is similar to sbph, however only bigrams are used to form sparse tokens; this uses far fewer resources than sbph with very similar results [20060516.0200] jonz: interface change: added tokenizer variable added tokenizer variable to DSPAM_CTX and added following tokenizer flags: DSZ_WORD Use WORD (uniGram) tokenizer DSZ_CHAIN Use CHAIN (biGram) tokenizer DSZ_SBPH Use SBPH (Sparse BP Hashing) tokenizer DSZ_OSB Use OSB (Orthogonal Sparse biGram) WARNING: This is an API change and constitutes a new major version. Third party applications may fail to compile/run against this. [20060414.1145] jonz: fix for segfault on log write err when using --with-logfile, if file cannot be opened, dspam segfaulted [20060513.1100] jonz: fixed compiler warnings on sqlite drivers signed-ness warnings, nothing significant [20060514.0900] jonz: discontinued support for berkeley db deprecated bdb drivers finally removed from distribution [20060512.2105] jonz: copyright modifications reassignment to Jonathan Zdziarski instead of using my corporate face [20060512.2100] jonz: removed some legacy piecess - removed dspam_corpus (replaced by newer dspam_train) - removed dspam_genaliases (replaced by parse-to-headers, virtual users, etc) [20060512.0100] jonz: segfault fix for UIDInSignature fixed a critical bug that can cause segfaults when correcting messages using UIDInSignature options. database handle is refreshed, but new pointer is never used. [20060510.0800] jonz: fix to recognize trainPristine "off" in preferences preference turned "off" should override config turned "on"
Created attachment 116401 [details] mail-filter/dspam/dspam-3.8.0.ebuild - Changed download URL - Added msgtag.spam and msgtag.nonspam
Created attachment 116403 [details] www-apps/dspam-web/dspam-web-3.8.0.ebuild Changed download URL
Created attachment 116404 [details, diff] 3.8.0-dspam.cron.patch Steve, you have done a great work with this. I saw atleast one thing that would not work on dash: $((foo + bar)) should be $(($foo + $bar)) The rest is mostly nitpicks, removing useless use of grep/cat etc. The patch needs testing.
(In reply to comment #26) > Created an attachment (id=116404) [edit] > 3.8.0-dspam.cron.patch > > Steve, you have done a great work with this. > Thanks > I saw atleast one thing that would not work on dash: > > $((foo + bar)) should be $(($foo + $bar)) > Perfect. This will work in bash and sh as well. > The rest is mostly nitpicks, removing useless use of grep/cat etc. > No, no, no.... Don't lower your great work. It is good. Using just one tool for the job instead of cat, pipe to grep, grep, pipe to sed, etc... is much better. Thanks! > The patch needs testing. > Patched cron job works without problems on my DSPAM 3.8.0 and MySQL 5.0.38.
Created attachment 116462 [details] mail-filter/dspam/files/3.8.0-dspam.cron New cron job. Patched with the changes from Natanael Copa.
Stop posting cron scripts! I already work on the new script ;)
(In reply to comment #29) > Stop posting cron scripts! > I already work on the new script ;) > Post it! POST IT! I want to test it :)
Fixed in cvs, thanks guys! I've rewritten the cron script starting from your version. I hope it works well for everyone. The analyzer graph still has problems. Maybe one of you could take care of this in a future bug?
(In reply to comment #31) > Fixed in cvs, thanks guys! > > I've rewritten the cron script starting from your version. I hope it works well > for everyone. > > The analyzer graph still has problems. Maybe one of you could take care of this > in a future bug? > What is the problem with the analyzer graph? On my setup I have no problem with the analyzer graph.
Created attachment 116513 [details] dspam-web weird graphs A picture is better than 1000 words.
