Bugzilla Bug 75668

When loading a previously useful firewall script (Used on lots of x86 boxen),
after loading 857 lines, I receive memory allocation errors.  It doesn't matter
if I load the 857 lines in the INPUT or OUTPUT chain.  I can get 428 in one,
and that leaves 429 for the other.  Or 857 in one chain, either INPUT or
OUTPUT.

Reproducible: Always
Steps to Reproduce:
1. Set up a ruleset with 900 rules.
2. iptables-restore < rule-set
3. Or, script it with a shell script

Actual Results:  
iptables-restore --> failed
Scripting --> iptables - Memory Allocation error  <- for each rule after 857.

Expected Results:  
Inserted the rules with no errors.

uname -a
Linux sparky 2.4.27-sparc #2 Thu Oct 7 10:49:35 MDT 2004 sparc64 sun4u TI
UltraSparc IIi (Sabre) GNU/Linux
512Mbyte RAM.  1Gig Swap, ~swap usage = 20M Actual Free Mem ~ 6-10M at most all
times.
iptables v1.2.11
Kernel has most items compiled in except for iptable_mangle & ipt_state.

billybobsa: is this still applicable?

Yes it is...

Thanks.

So I tried testing this on a Blade 1000 running a vanilla 2.6.12-rc3 kernel and
was able to load 20,000 rules in before I stopped (probably could have taken a
lot more, memory wasn't taxed that much).  After talking it over with a fellow
SPARC developer (Gustavo), we're thinking it's something to do with the 2.4
kernel.

As 2.6 kernels aren't always overly stable on SPARC64, it's probably not
recommended to try using them for now until the lockup issues subside.  However
if you are daring, 2.6.12-rc3 isn't a bad one to try, especially on an IDE
based SPARC64 system.

I am running this in production machines at an ISP...  At the moment, I have no
interest in the 2.6 series kernels.  I'd just like to see it fixed in the 2.4.x
 None of the x86 kernels exibit this issue...

These are SCSI machines to boot <G>

Would it be possible for you to post your ruleset here?

Created an attachment (id=61546) [edit]
IPTables Rule set requested by weeve@gentoo.org

Sure.  Here ya go...

Also, I have tried this up to a 2.4.30 kernel with the same result (broken). 
However, I have brought that T1 105 Netra up on a 2.6.11-r8 kernel, and the
problem is not present.  Works like one would expect it to <G>

If this isn't what you need, drop me a note...

what prints out the error ?  iptables ?  kernel ?

btw, what version of iptables showed this error ?  does 1.3.2 work properly ?

This seems to be present in all versions in the portage tree currently.  I
asked
Dave Miller, SPARC64 kernel maintainer and networking kernel hacker, about this
recently and he said;

"He could be hitting the kmalloc() limit via the netfilter
32-bit userland compat code in:

       arch/sparc64/kernel/sys_sparc32.c:do_netfilter_replace()"

so does that mean we can dump this bug onto the sparc team ? ;)

They are kernel errors.  Also, a bit more on the issue that I just uncovered.

Running the same kernel on a Sparc E220, with NFS running, when the NetApp got
busy with other functions, I would also get Memory Allocation errors if the 220
got put on hold for a extended period of time.  With a 2.6.8 kernel, the
allocation errors went away...  This HAS to be a kernel issue, not an iptables
issue...

See my comments just posted earlier this evening.  I do believe, now that I had
the issue with the E220 and the same 2.4 kernel and NFS lengthly waits, that is
IS a Sparc 2.4 kernel memory issue on the 2.4.x, not an iptables issue.

I am not having any allocation error issues with the 2.6.x series kernel on
Sparc... (yet!) [Knock on wood]

I wish I could help further, but I have removed all the 2.4.x kernels from my
Sparc boxes due to this Mem Alocation error, so have really nothing to test
with.  Although, I do have one Netra here at home that if it was absolutly
necessary, I could fire it up on a different HD and conduct directed testing.

I am not a C guy, so cannot do much more tha do what I am asked...

This appears to be caused by the 32 bit interfaces into the kernel, as hinted
by
Dave Miller on sparc-linux mailing list.  I've compiled a static 64 bit
iptables
and iptables-restore that you can try:
http://dev.gentoo.org/~squash/iptables.gz
http://dev.gentoo.org/~squash/iptables-restore.gz
md5sums:
2ddb82a400c06fed40f9b20b4b30d7a4  iptables-restore.gz
3ace295c062385dc794453b2400c9ba0  iptables.gz

Please try these.

Created an attachment (id=66506) [edit]
Results of tests...

Okay,

Tried your programs with success (sort of).

I booted my Netra here at home with a 2.4.30 kernel for this test.  I have
attaced a zip file that contains some results.	I tried it two different ways.

1) Moved your iptables and iptables-restore in to /sbin after saving the
originals in /root/save.  I have a /etc/firewall/current-iptables which holds
the entire firewall as a duplicate as it is loaded with my firewall program.  I
used iptables-restore < /etc/firewall/current-iptables and it loaded without a
hitch.	Some 6000+ lines.  There is a iptables.lst in the zip that contains the
output of:
iptables -L -n -v.  Looks good and seems to function correctly.

2) Let the system reboot with the 2.4.30 kernel and let my firewall program
load the firewall as I usually do under the 2.6 kernel.  Encluded in the zip is
a bootup-screen txt file that is a copy/paste of the boot from the start.  I
had a lot of seg faults as you can see on some of the initial commands, however
most of the firewall loaded except those commands that seg faulted.

Synopsis:  You static binaries appear to be the answer.  Why they seg faulted
on those rules is beyound me at this point in time.

Loading with iptables-restore functions flawlessly.

Also included in the zip is a tarball of my /etc/firewall/* and
/etc/init.d/firewall <- the program that runs in the default run level for you
to look at.  The /etc/firewall/firewall.conf.iptables is sourced by
/etc/init.d/firewall during boot.

Hope this helps.

We have verified the cause of the issue, and it is with the 32 bit interfaces
into the 64 bit kernel.  However, there is little to no effort to repair it
(unless one of use does it ourselves), so realisticly here are your choices:
1. Broaden your rules so that you don't actually need to have 900 lines of them
2. Use the 64 bit binaries until 2.6 is stable
3. Rewrite the 32 bit interface

Not necessarily in that order! Unless someone has something to contribute, I'll
close this bug as UPSTREAM in a couple days.

Well, as I mentioned previously, I have migrated to the 2.6 on my production
box.

[22:21][root@sparky:~]# uname -a
Linux sparky 2.6.11-gentoo-r8 #1 Sat May 14 16:10:48 MDT 2005 sparc64 sun4u TI
UltraSparc IIi (Sabre) GNU/Linux

[22:21][root@sparky:~]# uptime
 22:21:46 up 71 days,  8:53,  1 user,  load average: 0.00, 0.00, 0.00
[22:21][root@sparky:~]#

So far, so good (Knock on wood).  Seems they hit a good one on
2.6.11-gentoo-r8.
 At least as far as I am concerned.  Sparky's running just about everything
usable from the outside.  HTTP, Sendmail, MD & Clam, MySql, Proftp, Webmail
with
Imap (Horde/Imp) etc., etc.  Served up well over 60Gig of data last month.  At
about 25Gig as if this month so far.  Totals:

          RX packets:129815397 errors:0 dropped:0 overruns:0 frame:0
          TX packets:209871646 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:28617070835 (27291.3 Mb)  TX bytes:288836810034 (275456.2
Mb)

Since bootup on this kernel 71 days ago.  Not too shabby.

So, I can let this go as the 2.6 seems to be up to snuff so far.

Thanks for all your assistance...

Created an attachment (id=66557) [edit]
Fix for kmalloc ioctl32 netfilter b0rkedness on sparc64

Please try this attached patch, from preliminary tests it should fix this
issue.

I will try this, however, it will be (probably) this weekend unless I get some
free time to re-do a kernel (2.4.31) as I am running 2.4.30.

Thanks...

Created an attachment (id=66570) [edit]
Proper and nice patch

Silly kmalloc versus vfree fix0red.

Created an attachment (id=66615) [edit]
Iptables listing - Sparc64 Kernel 2.4.31

Congratulations, it worked.

Attached is my list of rules and all 6082 of them are present...

Thanks a bunch.  Think it will make the .32 kernel? <G>

This was sent upstream to the sparc kernel maintainer (davem) and accepted, so
i
guess so.
If 2.4.32 takes long (it's currently in the -pre3 iteration) Joker may be able
to spin a newer sparc-sources with the fix.

In portage as sparc-sources-2.4.31-r1 (currently ~sparc) which also includes
the
bzero_noasi gcc 3.4.x build fix.
The patchset may take a couple of hours to hit the mirrors.

sparc-sources-2.4.31-r2 stable, this bug is gone.