Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 611142

Summary: mail-mta/netqmail-1.06-r5: 451_See_http://pobox.com/~djb/docs/smtplf.html
Product: Gentoo Linux Reporter: Alex Efros <powerman-asdf>
Component: Current packagesAssignee: Rolf Eike Beer <eike>
Status: RESOLVED FIXED    
Severity: normal CC: arnt, eike, gentoobugs, net-mail+disabled, nuitari, proxy-maint
Priority: Normal Keywords: PullRequest
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://github.com/gentoo/gentoo/pull/13491
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Qmail patch, modified to turn LF into CRLF in the part of the header that's before the first Received field

Description Alex Efros 2017-02-27 21:15:54 UTC
Looks like qmail-smtputf8.patch added in -r5 is broken. When -r5 tries to send email to either -r4 or -r5 it fails with:
  451 See http://pobox.com/~djb/docs/smtplf.html

-r4 sends same email without issues to both -r4 and -r5.

(I used mutt to send this email, but same issue happens with emails sent by portage after emerging packages, so it's unlikely an issue with mutt.)
Comment 1 Arnt Gulbrandsen 2017-03-06 13:29:43 UTC
Do you think you could make a shell script that fails with r5 and works with r4? (Or any other script, really. I haven't the faintest notion what the difference can be, so something that sends mail with zero user input ought to help.)
Comment 2 Alex Efros 2017-03-06 14:43:14 UTC
I'm not sure I get what you want. I was able to reproduce this issue right now using this command to send email:

  echo -e "To: root@MY.SERVER.HERE\nSubject: test\n\nTest." | sendmail -t
Comment 3 Arnt Gulbrandsen 2017-03-06 15:02:17 UTC
That's fine, I'll try that command tonight.

I asked because I send mail without problems, the patch doesn't modify any linefeeds (or actually any whitespace) and so I'm a bit puzzled.
Comment 4 Arnt Gulbrandsen 2017-03-06 15:24:50 UTC
Actually, I suspect that command may not make my qmail break, but I can guess what happens. Fix coming.
Comment 5 Alex Efros 2017-03-06 18:10:10 UTC
I suppose traffic dump should make clear what's doing wrong, but I've tried to emerge it with USE=-ssl and emerge failed. This looks like another bug, so if you'll fix patching without ssl then I'll be able to continue debugging this issue.
Comment 6 Arnt Gulbrandsen 2017-03-06 20:44:20 UTC
I've reproduced it, and found it in the source. I have to think a little more about the nicest way to fix it, though.

Regarding SSL: The SSL patch implements two things, namely RFCs 1869 (SMTP extensibility framework) and 2595 (TLS using 1869) (both RFCs have been updated or superseded, but that's irrelevant). My patch implements RFC 6531, which depends on 1869, since 6531 too is an SMTP extension. I have a version of the patch that implements 1869 instead of depending on the SSL patch's implementation, which I haven't published yet. I don't really want to either... Servers without SSL have passed their use-by date on the internet IMO. Can I somehow declare that this patch really needs the SSL patch?
Comment 7 Alex Efros 2017-03-06 21:35:54 UTC
(In reply to Arnt Gulbrandsen from comment #6)
> Can I somehow declare that this patch really needs the SSL patch?

qmail-smtputf8.patch right now is applied unconditionally, so if it require ssl patch, then you should either remove ssl USE flag and apply ssl patch unconditionally too, or add new USE flag and apply qmail-smtputf8.patch only when it set - then you'll be able to declare relationship between these two USE flags in $REQUIRED_USE (https://devmanual.gentoo.org/ebuild-writing/eapi/index.html#eapi=4).
Comment 8 Arnt Gulbrandsen 2017-03-22 13:05:36 UTC
Created attachment 467898 [details, diff]
Qmail patch, modified to turn LF into CRLF in the part of the header that's before the first Received field
Comment 9 Arnt Gulbrandsen 2017-03-22 13:06:20 UTC
Sigh.

Alex: Your test command doesn't break with my qmail, and for whatever reason I cannot find the will to futz around with my qmail configuration until it becomes susceptible.

But I think I know the problem, it's a real bug in my code, and I have a one-line change. Can you test it?

Sorry.
Comment 10 nuitari 2018-01-20 06:21:39 UTC
Hi Arnt,

I've started having some weird problems with LF in messages that used to be fine. The one line change in qmail-remote fixed it. Thank you.
Comment 11 Alexander Hof 2019-01-15 14:11:48 UTC
(In reply to Arnt Gulbrandsen from comment #9)
> Sigh.
> 
> Alex: Your test command doesn't break with my qmail, and for whatever reason
> I cannot find the will to futz around with my qmail configuration until it
> becomes susceptible.
> 
> But I think I know the problem, it's a real bug in my code, and I have a
> one-line change. Can you test it?
> 
> Sorry.

I can also confirm the fix works.
Comment 12 Rolf Eike Beer archtester 2019-08-07 10:13:13 UTC
Arnt, have you fixed the patch on your website, so new builds will just do it correctly, or does the ebuild need any changes?
Comment 13 Sergey 2019-10-11 12:57:22 UTC
I confirm this bug, it still exists in netqmail-1.06-r7

Probably it appears only when receiver is qmail too (as I checked on my other qmail-based server):

@400000005da073d32fb883bc status: local 0/10 remote 1/20
@400000005da073d4095e857c delivery 4: deferral: 185.132.240.3_failed_after_I_sent_the_message./Remote_host_said:_451_See_http://pobox.com/~djb/docs/smtplf.html./

when I compile netqmail-1.06-r7 without qmail-smtputf8.patch all works fine:

@400000005da07ae83a1a428c status: local 0/10 remote 6/20
@400000005da07ae9216e99f4 delivery 2: success: 185.132.240.3_accepted_message./Remote_host_said:_250_ok_1570798303_qp_11930/
Comment 14 Rolf Eike Beer archtester 2019-10-11 16:18:33 UTC
Arnt, I really don't want to ship a modified patch in Gentoo. Could you please put a new version, preferably with a date stamp, on your webspace that I could link as upstream?

While at it, I would change the checkutf8message() to return int, which would be what it wrote into the global utf8message variable. That could be local in smtp() then. While at it you could mark your new functions as static, and initialize state and pos in checkutf8message() right at their declaration. Bonus points if you create the diff with "-p" then, which makes review a bit easier.
Comment 15 Rolf Eike Beer archtester 2019-10-12 08:39:29 UTC
Just for the record: this patch has the same problem that qmail-remote has when it is used alone and the input already contains CRLF line endings (e.g. because of a PHP script injects the mail like this): it will recode that to CRCRLF, which is also bogus.
Comment 16 Alex Efros 2019-10-29 12:06:03 UTC
I've just updated to netqmail-1.06-r9 because of openssl update, and now I have same issue - two servers with same netqmail-1.06-r9 unable to send email from one to another!

I've removed smtputf8 patch from ebuild, and this solved the issue. Is there any other options how to fix it?
Comment 17 Larry the Git Cow gentoo-dev 2019-11-01 14:11:25 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c0c074a84d82aba52e68925cbb158e8d4b57f0bf

commit c0c074a84d82aba52e68925cbb158e8d4b57f0bf
Author:     Rolf Eike Beer <eike@sf-mail.de>
AuthorDate: 2019-10-29 22:09:12 +0000
Commit:     Joonas Niilola <juippis@gentoo.org>
CommitDate: 2019-11-01 13:59:14 +0000

    mail-mta/netqmail: fix SMTPUTF8 mode
    
    This fix was provided by the original patch author, but he sadly did not provide
    a new patch.
    
    Closes: https://bugs.gentoo.org/611142
    Signed-off-by: Rolf Eike Beer <eike@sf-mail.de>
    Signed-off-by: Joonas Niilola <juippis@gentoo.org>

 .../netqmail/files/qmail-smtputf8-crlf-fix.patch   |  12 ++
 mail-mta/netqmail/netqmail-1.06-r10.ebuild         | 196 +++++++++++++++++++++
 2 files changed, 208 insertions(+)