Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 127388

Summary: pptpd syslog flood
Product: Gentoo Linux Reporter: Thomas Stein <himbeere>
Component: Current packagesAssignee: Gentoo Dialup Developers <net-dialup>
Status: RESOLVED FIXED    
Severity: normal CC: pva
Priority: High    
Version: 2006.0   
Hardware: x86   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: reduce flood patch

Description Thomas Stein 2006-03-23 23:49:13 UTC
Hello.

Since upgrading to pptpd 1.3.0 my syslog is flooded by pptpd with messages like:

Mar 24 08:45:34 mail pptpd[25901]: GRE: accepting packet #4986
Mar 24 08:45:34 mail pptpd[25901]: GRE: accepting packet #4987
Mar 24 08:45:34 mail pptpd[25901]: GRE: accepting packet #4988
Mar 24 08:45:35 mail pptpd[25901]: GRE: accepting packet #4989
Mar 24 08:45:35 mail pptpd[25901]: GRE: accepting packet #4990
Mar 24 08:45:35 mail pptpd[25901]: GRE: accepting packet #4991
Mar 24 08:45:35 mail pptpd[25901]: GRE: accepting packet #4992
Mar 24 08:45:35 mail pptpd[25901]: GRE: accepting packet #4993
Mar 24 08:45:36 mail pptpd[25901]: GRE: accepting packet #4994
Mar 24 08:45:36 mail pptpd[25901]: GRE: accepting packet #4995
Mar 24 08:45:45 mail pptpd[28072]: GRE: accepting packet #1268
Mar 24 08:45:45 mail pptpd[28072]: GRE: accepting packet #1269
Mar 24 08:45:46 mail pptpd[28072]: GRE: accepting packet #1270
Mar 24 08:45:47 mail pptpd[28072]: GRE: accepting packet #1271
Mar 24 08:45:50 mail pptpd[28072]: GRE: accepting packet #1272

I have disabled debug in pptpd.conf and options.pptpd. 

emerge info:

mail ~ # emerge info
Portage 2.0.54 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.5-r0, 2.6.16 i686)
=================================================================
System uname: 2.6.16 i686 AMD Athlon(TM) XP 2600+
Gentoo Base System version 1.6.14
dev-lang/python:     2.2.3-r1, 2.3.5
sys-apps/sandbox:    1.2.11
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1
sys-devel/libtool:   1.5.20
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=athlon-xp -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=athlon-xp -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks sandbox sfperms strict"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo http://darkstar.ist.utl.pt/gentoo/ http://gentoo.inode.at/"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="x86 apm arts avi berkdb bitmap-fonts bzip2 crypt cups curl dri eds emboss encode expat foomaticdb fortran gdbm gif gmp gstreamer gtk2 idn imlib ipv6 java jpeg kde libg++ libwww mad mhash mikmod motif mp3 mpeg ncurses nls ogg oggvorbis opengl oss pam pcre pdflib perl php png python quicktime readline sdl slang spell ssl tcpd tiff truetype truetype-fonts type1-fonts udev vorbis xml2 xmms xv zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS

best regards
t.
Comment 1 Thomas Stein 2006-03-24 00:00:50 UTC
Hello.

Found this in pptpd 1.3.1 Changlog:

Thu Dec 29 11:04:13 2005  James Cameron  <quozl@us.netrek.org>

        * pptpgre.c (pptp_gre_init): prevent initial unwanted GRE ACK.
        From: Jonathan Barker <jabarker@itstrategic.ca>

Maybe its already fixed. I tried to rename the 1.3.0 ebuild to 1.3.1 but there is something wrong with the gentoo patches then.

regards
t.
Comment 2 Lars Weiler (RETIRED) gentoo-dev 2006-03-25 07:56:11 UTC
Not an infra bug
Comment 3 Alin Năstac (RETIRED) gentoo-dev 2006-03-26 02:00:40 UTC
fixed in pptpd-1.3.1.

FYI: Even this version has this flood activated. I had to comment that syslog call.
Comment 4 Thomas Stein 2006-03-26 02:57:16 UTC
Thank you Alin.

best regards
t.
Comment 5 Thomas Stein 2006-03-26 23:20:45 UTC
Created attachment 83212 [details, diff]
reduce flood patch

Hello.

FYI: I have an answer from a PopTop Developer:
---
You have debug logging enabled in syslog configuration.  Versions
1.3.0 and 1.3.1 of pptpd report each packet to syslog with a level of
LOG_DEBUG.
---

best regards
thomas


Please try the attached patch and let me know if it fixes it.

Or turn off debug logging in syslog.
Comment 6 Alin Năstac (RETIRED) gentoo-dev 2006-03-26 23:43:36 UTC
I know the flood is visible only on hosts that logs (to console or to a file) messages with LOG_DEBUG level, but it is insane to generate a syslog call for every packet you receive. 
this shouldn't exist at all, even if user activated the debug option of the pptpd.
Comment 7 Peter Volkov (RETIRED) gentoo-dev 2006-09-04 23:11:11 UTC
Alin. I've contacted upstream developers and here is his answer:

================begin of mail====================================
> segfault using -C option:
> http://bugs.gentoo.org/show_bug.cgi?id=132898
> and proposed by Alin Nastac (mrness at gentoo.org) patch
> (pptpd-1.3.2-connections) in attachment.

Taken.  Also added an informational message to show when the number of
connections has been constrained by the number of IP addresses given.
Included in 1.3.3.

> And another bug is pptpd syslog flood with GRE packets:
> http://bugs.gentoo.org/show_bug.cgi?id=127388

Rejected.  Turn off debug mode in pptpd.conf if you don't want to see
these syslog messages.  I see opinions that it should not do this, but I
don't see any good argument.  We have needed these messages to diagnose
problems.

> [Gentoo] Maintainer of pptpd told that it "is insane to generate a
> syslog call for every packet you receive." What do you think about
> that?

Gladly insane if it helps us support it here.
================end of mail======================================

So 1. In next version we'll not need pptpd-1.3.2-connections-arg.patch.
2. I think it's good idea to remove the patch that comments logging of all GRE packets in versions starting from 1.3.2. If you do not like syslog calls may be it's good idea to direct all output into some files in debug mode. But with current solution nobody manages to help our users in poptop mailing list.
Comment 8 Peter Volkov (RETIRED) gentoo-dev 2006-09-04 23:14:08 UTC
And while we are on topic. 1.3.3 was released today :)
Changes to pptpd since 1.3.2 are:
- add missing connections option in sample pptpd.conf [Cameron]
- add message to indicate when constrained by IP range [Cameron]
- fix segfault on -C option, Closes Gentoo Bug #132898 [Nastac]
- support mixed architecture build on x86_64 [Gorlov]
- fix configure --with and --without option processing again [Cameron]
- include libutil in logwtmp build [Gorlov]
- fix spec file [Howarth]
Comment 9 Alin Năstac (RETIRED) gentoo-dev 2006-09-05 00:06:06 UTC
I still find the "GRE: accepting packet" syslog call pretty useless, even if it is enabled only on debug.
The only useful information (if there's any) from the developer pov is the sequence number. However, users might want to solve their problems and therefore enable debug option, which will have the following side effects:
 - decreased performance
 - busier CPU and possibly HDD
 - cluttered logs

I wouldn't mind if debug had several levels and this particular syslog call would be called at the ultimate level, but this isn't the case.
Comment 10 Peter Volkov (RETIRED) gentoo-dev 2006-09-05 00:35:55 UTC
(In reply to comment #9)
> I still find the "GRE: accepting packet" syslog call pretty useless, even if it  is enabled only on debug.

May be. But if you look throught poptop mailing archives, you will find, that any users that askes for help is required to show their debug logs. Currently gentoo users are not supported there...

> The only useful information (if there's any) from the developer pov is the
> sequence number. However, users might want to solve their problems and
> therefore enable debug option, which will have the following side effects:
>  - decreased performance
>  - busier CPU and possibly HDD
>  - cluttered logs

You do not need debug information when server runs in production. You only need it if something does not work. So personally I do not see any problems here. But well I've tried to ask upstream devs again why they need this information. May be this clarifies something ...
Comment 11 Alin Năstac (RETIRED) gentoo-dev 2006-09-06 12:31:44 UTC
(In reply to comment #10)
> May be. But if you look throught poptop mailing archives, you will find, that
> any users that askes for help is required to show their debug logs. Currently
> gentoo users are not supported there...

And you hope they will support gentoo if we drop the patch?
The only support gentoo users will receive from upstreams all over the net will be quality challenged ebuilds provided by volunteers with good heart but not enough knowledge to make a better one.
The best support they will ever receive is through g.o sites.

> You do not need debug information when server runs in production. You only need
> it if something does not work. So personally I do not see any problems here.

I see one special problem. In this condition, just enabling the debug will affect your test case so badly that in fact you will test a totally different thing (bandwidth will be cut to a fraction of the non-debug case, packages get dropped, etc).

Does the pppd allow you to generate a log line for every PPP frame? Did you found an Ethernet driver with such an option? No, there aren't, because information provided by such logs wouldn't worth a penny.

I am waiting for your reply before bumping the version of pptpd.
Comment 12 Peter Volkov (RETIRED) gentoo-dev 2006-09-07 00:43:38 UTC
(In reply to comment #11)
> And you hope they will support gentoo if we drop the patch?

I'm subsribe more then 1 year on that mailing list. I've also asked a questions there and I am sure that they will support gentoo users also (Of course if gentoo users will not ask infinity+1 time about poptop and NAT ;) ).

> The only support gentoo users will receive from upstreams all over the net will
> be quality challenged ebuilds provided by volunteers with good heart but not
> enough knowledge to make a better one.

I'm sorry. I did not undestand above text. :(

> The best support they will ever receive is through g.o sites.

This is debatable statement.
 
> I see one special problem. In this condition, just enabling the debug will
> affect your test case so badly that in fact you will test a totally different
> thing (bandwidth will be cut to a fraction of the non-debug case, packages get
> dropped, etc).

I agree with you here but take a look what James answered:

On Tue, Sep 05, 2006 at 11:35:06AM +0400, Peter Volkov wrote:
> James, but could you, please, explain, how GRE sequence number (I mean
> "GRE: accepting packet #%d") can help you to diagnose problems?

Sure.  The new packet reordering code may have defects.  Whenever we get
a problem report that claims to relate to packet reordering, the GRE
sequence number issued by this debug line can be related to other debug
lines and tcpdump output.  The time between receiving a problem report
and solving it may be reduced.

+++++++++++++++++++++++++end of James's message+++++++++++++++++++++++++

pptpd is not the kernel module and race conditions are rare here. It is the kernel or complex multithreaded applications wich normaly do not have problems in debuging mode while does not work without. It's common case there, but I do not think it's common here.

But well. I'll follow your answer James again. I think the best solution is to have gredebug option to enable/disable this output in pptpd. Currenly gentoo may fix this problem with verbosedebug USE flag to enable/disable this patch.
Comment 13 Alin Năstac (RETIRED) gentoo-dev 2006-09-07 06:20:14 UTC
Peter, what do you think of this as local USE flag?

net-dialup/pptpd:gre-extreme-debug - Log all GRE accepted packages when in debug mode (required if you want upstream support)
Comment 14 Peter Volkov (RETIRED) gentoo-dev 2006-09-07 06:42:55 UTC
(In reply to comment #13)
> Peter, what do you think of this as local USE flag?
> 
> net-dialup/pptpd:gre-extreme-debug - Log all GRE accepted packages when in
> debug mode (required if you want upstream support)

I think this is best idea, while upstream does not introduce something better, like runtime option. Thank you!
Comment 15 Alin Năstac (RETIRED) gentoo-dev 2006-09-07 23:37:46 UTC
pptpd-1.3.3 commited to cvs.

Peter, please mark the new version as stable after a probation time of one month.
Of course, only if there aren't open bugs about it ;)
Comment 16 Peter Volkov (RETIRED) gentoo-dev 2006-09-08 00:53:53 UTC
Sorry, I'm not in arch team :) I'm a part of netmon herd only. 

But I'll try not to forget and fill in bug ;)