Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 597512 - mail-filter/spamassassin: improvements in update-spamassassin-rules.cron
Summary: mail-filter/spamassassin: improvements in update-spamassassin-rules.cron
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Philippe Chaintreuil
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-10-19 08:17 UTC by Marcin Mirosław
Modified: 2022-12-28 18:54 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcin Mirosław 2016-10-19 08:17:24 UTC
I'd like to add support for updating rules from channels other than official. It's easy to add option "--channelfile" to sa-update. Problem appears when we want to add channel without GPG support, such channel needs add option "--nogpg". But "--nogpg" is global flag, it alse affects channels whuch supports gpg signing.
I've got two concepts:
- use two files, my-extra-channels.gpg.txt and my-extra-channels.nogpg.txt and check them in two runs
- one file with additional channels with field "nogpg" for channels which doesn't support gpg. Like:

xyz.sa.channel.com
abc.sa.channel.com   nogpg
[etc]

but it needs additional parsing and it doesn't looks clean and easy solution for me.
What do you think about this, maybe you have an idea for other, better way to achieve it?
Comment 1 Marcin Mirosław 2016-10-19 08:20:34 UTC
I can't assign this bug to me, is it ok or not?
Comment 2 Philippe Chaintreuil 2016-10-19 12:07:23 UTC
As a footnote, I have several --gpgkey options in my sa-update cron job.  So there's that wrench in the works as well.

Second option might be more long the lines:

updates.spamassassin.org
xyz.sa.channel.com		856AA88A
xyz.sa.channel.com		nogpg
[etc]

Something like "grep -v nogpg my-extra-channels.txt| awk '{print $2}' | sort  | uniq | sed '/^\s*$/d'" would get you the gpg keys and "grep -v nogpg my-extra-channels.txt| awk '{print $1}' | sort  | uniq" would get you the gpg channel names.  Same again, but without the "-v" on grep would get you the nogpg channel names.

There's a bug that if a channel had "nogpg" in it, it would always end up in the nogpg pile.  If you wanted to go this way though and that was a worry, there's probably a 20 line python script to parse this into three files anyway.

There's also a point that could be made that this seems like something upstream could benefit from as well.
Comment 3 Michael Orlitzky gentoo-dev 2016-10-19 12:29:24 UTC
Yeah, sure. I've never used an unofficial channel, so take this with a grain of salt, but here's my first impression. In the cron job, we could have two bash arrays,

  # Additional unofficial channels.
  GPG_CHANNELS=()
  NOGPG_CHANNELS=()

In the existing call to sa-update, we can add in the unofficial channels:

  # Try to update the rules.
  gpg_channel_cmd=""
  for gpgchan in "${GPG_CHANNELS[@]}"; do
      gpg_channel_cmd+=" --channel ${gpgchan}"
  done
  sa-update $gpg_channel_cmd
  gpg_result=$?

Then, if there are no-gpg channels, we do those too:

  nogpg_channel_cmd = ""
  for nogpgchan in "${NOGPG_CHANNELS[@]}"; do
      nogpg_channel_cmd+=" --channel ${nogpgchan}"
  done

  nogpg_result=0
  if [ -n "$nogpg_channel_cmd" ]; then
      nogpg_channel_cmd="--nogpg ${nogpg_channel_cmd}"
      sa-update $nogpg_channel_cmd
      nogpg_result=$?
  fi

Finally, we have to compute the overall exit code from the two commands:

  overall_result = 0
  if (($gpg_result == 0)); then
      if (($nogpg_result != 0)); then
          overall_result = $nogpg_result
      fi
  else
      overall_result = $gpg_result
  fi

All of that is totally untested, but in theory it should work exactly the same for people with no additional channels. If you pass --channel to sa-update, does it still use the official channel too? If not, we'll need to add that to GPG_CHANNELS by default. And if it does, we should find a way to make it not do so on the second call that uses --nogpg.

My comment just collided with Philippe's... I think it should be possible to make GPG_CHANNELS an associative array to handle differing keys as well.
Comment 4 Philippe Chaintreuil 2016-10-19 12:49:01 UTC
> If you pass --channel to sa-update, does it still use the official channel too?

I do not believe it does.  I have it listed explicitly in my channel file.  But the man page hints that it might always be used implicitly. 

==============================================================================      --channel
           sa-update can update multiple channels at the same time.  By
           default, it will only access "updates.spamassassin.org", but more
           channels can be specified via this option.  If there are multiple
           additional channels, use the option multiple times, once per
           channel.  i.e.:
==============================================================================

Footnote: sa-update doesn't actually match gpg keys to channels, so your array doesn't *have* to be associative.  Figured I'd mention it in case it makes your life easier.  (I kept them together in my previous suggestion because it seems like it keeps maintenance of the file easier for the user.)
Comment 5 Michael Orlitzky gentoo-dev 2016-10-24 01:54:06 UTC
(In reply to Philippe Chaintreuil from comment #4)
> 
> Footnote: sa-update doesn't actually match gpg keys to channels, so your
> array doesn't *have* to be associative.  Figured I'd mention it in case it
> makes your life easier.  (I kept them together in my previous suggestion
> because it seems like it keeps maintenance of the file easier for the user.)

Ah, I see the problem. I guess you could specify all of the keys at once, but if you had two trusted sources of signatures, that would let the second source forge the first one's signatures and vice-versa.

You should ask about this on the SA devel list... my attempt is not very simple, and I really like Philippe's example from comment #2. If they'd be willing to accept that format for --channelfile, the end result would be a lot nicer than if we try to kludge it with bash. Even if they shoot it down, one of the SA devs may have a similar proposal.
Comment 6 Philippe Chaintreuil 2022-12-28 18:54:53 UTC
I'm unaware of any 3rd party spamassassin rule sets at this point.

SARE shutdown in 2009. [1][2]

OpenProtect.org shutdown in March 2016. [3][4]

... so I'm going to mark this as won't fix since there doesn't seem to be a need for it.  (Nor an easy way for me to test anything I'd try.)

If someone in the future finds a need for additional channels, please re-open.


[1] https://cwiki.apache.org/confluence/display/spamassassin/RuleUpdates#RuleUpdates-SAREChannels
[2] https://cwiki.apache.org/confluence/display/SPAMASSASSIN/SareChannels
[3] Feb 2016 (before): https://web.archive.org/web/20160211000357/https://www.openprotect.com/
[4] March 2026 (after): https://web.archive.org/web/20160306202006/https://www.openprotect.com/