Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 171404 - Gentoo-announce not synchronised or synchronised with delays to published GLSas and Portage metadata
Summary: Gentoo-announce not synchronised or synchronised with delays to published GLS...
Status: RESOLVED REMIND
Alias: None
Product: Gentoo Infrastructure
Classification: Unclassified
Component: Mailing Lists (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Gentoo Infrastructure
URL: http://security.gentoo.org/
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-03-19 06:40 UTC by Sabahattin Gucukoglu
Modified: 2007-03-21 07:47 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sabahattin Gucukoglu 2007-03-19 06:40:14 UTC
Gentoo-announce frequently gets GLSA mail with noticeable delays after both portage and the GLSA index are fixed up.  I also occasionally see bugs missing from the announce list entirely, which has brought me into the habbit of using glsa-check and checking security.gentoo.org every time there's a new scurge of fixes.  Nothing ever changes about my extremely lenient mail configuration between times, which leads me to suspect that perhaps mail isn't going where it oughta.  Since related bugs always show that the GLSA is issued in a very efficient fashion, and that it is very rare to find the announcement being submitted much longer after the last platform has check in, I think this is more of a mail issue than a staff issue.

Reproducible: Sometimes

Steps to Reproduce:
Always check all GLSA sources every time an announcement arrives.
Actual Results:  
Often, they are all synchronised.  Sometimes, they are synchronised after a delay which can be found by checking publication and errata modification dates, but more often there is simply less mail than there are GLSas published.  In the worst case, I give up after several days have passed without the mail and not up until registering with Bugzilla have I thought to give notice of something which obviously wasn't serious enough for most users.

Expected Results:  
With allowance for time for mail distribution and a check by GLSA author that all maintainer checks are in, which should never take more than a few hours, I should see mail corresponding to the web page and portage tree, if not immediately then very soonafter, and eventually come to rely on the mail exclusively.

I am running a nonrestrictive sendmail that bends right over for all the unholy MTAs/spambots/worms out there.  There are no SMTP-level blocks whatever (if more admins gave a @*$! about their mailers' configurations, I would certainly do differently and not be plagued by so many false positives).  My filtering method is exclusively bayesian (spamprobe) and nope, the announces aint in my spam folders, either.
Comment 1 Raphael Marichez (Falco) (RETIRED) gentoo-dev 2007-03-19 08:05:09 UTC
Thanks for your report. I confirm there are some email loss... and it's indeed confusing.

As a summary, there are two situations:

- emails hitting certain recipients of -announce but not all of them. We don't re-send GLSAs in that case, or some recipients will have the email twice. We may change this policy if we have users feedback. It happens ~ 20% of the time.

- emails *apparently* never hitting -announce, or emails really not hitting g-announce and we don't even receive the moderation request (g-announce is indeed moderated). In that case we resend the GLSA later. Recently, it was the case of GLSA-200703-13 and GLSA-200703-15.
Comment 2 Sabahattin Gucukoglu 2007-03-19 18:26:41 UTC
(In reply to comment #1)
> Thanks for your report. I confirm there are some email loss... and it's indeed
> confusing.
> As a summary, there are two situations:

These essentially explain what I'm seeing, thanks!

> - emails hitting certain recipients of -announce but not all of them. We don't
> re-send GLSAs in that case, or some recipients will have the email twice. We
> may change this policy if we have users feedback. It happens ~ 20% of the time.

Does anyone know exactly how bad this lossage is?  I mean, what metrics are in the MTA or MLM running the list which might explain such a thing?  Can someone find the lists of addresses which bounced and caused the distribution to somehow fail?  This sounds fishily like a case of if-only-I-had-the-darn-logs syndrome.  I know it's a sendmail with mlmmj (and I'm wondering how they managed to add verp then).  But it seems to me that the problem itself is findable and fixable and that the postmaster shouldn't be happy until it is.  It's rather odd.

> - emails *apparently* never hitting -announce, or emails really not hitting
> g-announce and we don't even receive the moderation request (g-announce is
> indeed moderated). In that case we resend the GLSA later. Recently, it was the
> case of GLSA-200703-13 and GLSA-200703-15.

Right, I chanced upon 200703-15 which prompted me to start this bug.  Thanks for your attention.
Comment 3 Andrea Barisani (RETIRED) gentoo-dev 2007-03-19 18:31:14 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > Thanks for your report. I confirm there are some email loss... and it's indeed
> > confusing.
> > As a summary, there are two situations:
> 
> These essentially explain what I'm seeing, thanks!
> 
> > - emails hitting certain recipients of -announce but not all of them. We don't
> > re-send GLSAs in that case, or some recipients will have the email twice. We
> > may change this policy if we have users feedback. It happens ~ 20% of the time.
> 
> Does anyone know exactly how bad this lossage is?  I mean, what metrics are in
> the MTA or MLM running the list which might explain such a thing?  Can someone
> find the lists of addresses which bounced and caused the distribution to
> somehow fail?  This sounds fishily like a case of if-only-I-had-the-darn-logs
> syndrome.  I know it's a sendmail with mlmmj (and I'm wondering how they
> managed to add verp then).  But it seems to me that the problem itself is
> findable and fixable and that the postmaster shouldn't be happy until it is. 
> It's rather odd.

VERP info: http://dev.gentoo.org/~lcars/misc/sendmail-hacks.txt
(it's also in README.sendmail inside mlmmj tarball)

It is not a "findable and fixable" problem, or at least it's not that trivial. The postmaster (me) spend *many* hours trying to find the issue without luck, and I assure that it's not trivial and no, I'm not happy. We will consider using a different software if we cannot track down the bug soon, I'm currently no longer involved in mlmmj management so maybe the new admin will have better luck.
Comment 4 Sabahattin Gucukoglu 2007-03-20 12:13:12 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > Thanks for your report. I confirm there are some email loss... and it's indeed
> > > confusing.
> > > As a summary, there are two situations:
> > 
> > These essentially explain what I'm seeing, thanks!
> > 
> > > - emails hitting certain recipients of -announce but not all of them. We don't
> > > re-send GLSAs in that case, or some recipients will have the email twice. We
> > > may change this policy if we have users feedback. It happens ~ 20% of the time.
> > 
> > Does anyone know exactly how bad this lossage is?  I mean, what metrics are in
> > the MTA or MLM running the list which might explain such a thing?  Can someone
> > find the lists of addresses which bounced and caused the distribution to
> > somehow fail?  This sounds fishily like a case of if-only-I-had-the-darn-logs
> > syndrome.  I know it's a sendmail with mlmmj (and I'm wondering how they
> > managed to add verp then).  But it seems to me that the problem itself is
> > findable and fixable and that the postmaster shouldn't be happy until it is. 
> > It's rather odd.
> VERP info: http://dev.gentoo.org/~lcars/misc/sendmail-hacks.txt
> (it's also in README.sendmail inside mlmmj tarball)

Wa?  You are sick! :-)  (but it's damned impressive, all the same)

> It is not a "findable and fixable" problem, or at least it's not that trivial.
> The postmaster (me) spend *many* hours trying to find the issue without luck,
> and I assure that it's not trivial and no, I'm not happy. We will consider
> using a different software if we cannot track down the bug soon, I'm currently
> no longer involved in mlmmj management so maybe the new admin will have better
> luck.

My choice of MLM is ecartis (not yet in tree) which understands DSN (so, if you're still with sendmail or move to postfix, this may be a nice choice).

Right, well, otherwise there is nothing more to do until some change happens to lists.gentoo.org (which is probably of more interest to the other users of the listserver) so I think probably we resolve this bug as Remind.  Thanks (and good luck to current postmaster, seems like it's in good hands).
Comment 5 Raphael Marichez (Falco) (RETIRED) gentoo-dev 2007-03-20 22:00:50 UTC
i disagree with closing that bug since it's really a worrying issue... err, we already have lots of difficult open bugs waiting for a fix.
Comment 6 Sabahattin Gucukoglu 2007-03-21 07:47:14 UTC
(In reply to comment #5)
> i disagree with closing that bug since it's really a worrying issue... err, we
> already have lots of difficult open bugs waiting for a fix.

It is a worrying issue, but it's for infrastructure and potentially affects more than gentoo-announce.  I've marked as remind because I don't think much more can be done right now, and security team isn't the problem - listmanager and mailer configuration is.  So we should be coaching for users to even be aware that a mail delivery problem exists (and if any new bug gets opened on that, please CC me it).  If this bug attracts enough attention then it may make sense to reopen it, because it's only been until now a bug relating to mail loss (as far as I've managed to tell) has been filed, and although I agree it's a problem no-one else has been bothered enough to mention it if they ever thought much of it (I nearly didn't).

So the more interesting question related specifically to the GLSas is ... is any likely short-term recommendation forthcoming, do you think?  Does it make sense to broadcast this possible problem to gentoo-announce (or elsewhere) and have everyone check just how many security bugs they've been missing and just how bad the situation is?  Because that's precisely what's pertinent to this bug and that's really the thing I'm most bothered about - people having not got all their fixes because they've not even been aware of them.  I routinely don't have access to a web browser.  And in avoiding the problems of mail loss, how about alternative channels for GLSAs - ATOM feed, perhaps?