Bug 585638

Summary:	sys-kernel/hardened-sources-4.5.7 - PAX: size overflow detected in function ext4_mb_new_group_pa fs/ext4/mballoc.c:3696
Product:	Gentoo Linux	Reporter:	Hank Leininger <hlein>
Component:	Hardened	Assignee:	The Gentoo Linux Hardened Team <hardened>
Status:	RESOLVED OBSOLETE
Severity:	normal	CC:	alexanderyt, pageexec, re.emese, spender
Priority:	Normal
Version:	unspecified
Hardware:	All
OS:	Linux
Whiteboard:
Package list:		Runtime testing required:	---

Description Hank Leininger 2016-06-11 22:10:28 UTC

4.5.7-hardened with CONFIG_PAX_SIZE_OVERFLOW errors early in init scripts on one of my servers (but not another w/different hardware configuration).

Hardened profile, using gcc-4.9.3 hardened.

I booted with pax_size_overflow_report_only captured the following:

[snip]
[   59.733296] grsec: mount of / to /tmp/tmp.lKJDwN0akA by /bin/mount[mount:1979] uid/euid:0/0 gid/egid:0/0, parent /lib64/rc/sh/openrc-run.sh[openrc-run.sh:1956] uid/euid:0/0 gid/egid:0/0
[   59.765009] grsec: unmount of /dev/loop4 by /bin/umount[umount:1981] uid/euid:0/0 gid/egid:0/0, parent /lib64/rc/sh/openrc-run.sh[openrc-run.sh:1956] uid/euid:0/0 gid/egid:0/0
[   59.947597] PAX: size overflow detected in function ext4_mb_new_group_pa fs/ext4/mballoc.c:3695 cicus.1008_81 max, count: 117, decl: pa_lstart; num: 0; context: ext4_prealloc_space;
[   59.963743] CPU: 5 PID: 2002 Comm: dmesg Tainted: G           O    4.5.7-hardened-1 #3
[   59.963745] Hardware name: Supermicro X8DTU/X8DTU, BIOS 2.1b       12/30/2011
[   59.963747]  0000000000000286 0000000000000286 ffffc900016db8a0 ffffffff8123d5c7
[   59.963751]  ffffffff8185b440 ffffffff816f9049 0000000000000e6f ffffc900016db8d0
[   59.963755]  ffffffff81126554 ffff8800bf030000 ffff8800bf033000 ffff8806226f9818
[   59.963758] Call Trace:
[   59.963766]  [<ffffffff8123d5c7>] dump_stack+0x4e/0x71
[   59.963771]  [<ffffffff81126554>] report_size_overflow+0x3f/0x7a
[   59.963774]  [<ffffffff811baf7d>] ext4_mb_new_group_pa+0xbd/0x1fc
[   59.963778]  [<ffffffff811bf1cb>] ext4_mb_new_blocks+0x1b3/0x3ad
[   59.963780]  [<ffffffff811bf1cb>] ? ext4_mb_new_blocks+0x1b3/0x3ad
[   59.963784]  [<ffffffff811b4da0>] ext4_ext_map_blocks+0xd6b/0x1149
[   59.963786]  [<ffffffff811b4da0>] ? ext4_ext_map_blocks+0xd6b/0x1149
[   59.963790]  [<ffffffff81190010>] ? ext4_find_unwritten_pgoff.isra.12+0x8a/0x2d7
[   59.963794]  [<ffffffff81196546>] ext4_map_blocks+0x33b/0x4d1
[   59.963796]  [<ffffffff81196546>] ? ext4_map_blocks+0x33b/0x4d1
[   59.963799]  [<ffffffff81199252>] ext4_writepages+0x45f/0x9a0
[   59.963801]  [<ffffffff81199252>] ? ext4_writepages+0x45f/0x9a0
[   59.963806]  [<ffffffff810db23c>] do_writepages+0x3f/0x50
[   59.963809]  [<ffffffff810db23c>] ? do_writepages+0x3f/0x50
[   59.963812]  [<ffffffff810d26ec>] __filemap_fdatawrite_range+0x5e/0x76
[   59.963815]  [<ffffffff810d2724>] filemap_flush+0x20/0x28
[   59.963817]  [<ffffffff811969f6>] ext4_alloc_da_blocks+0x1c/0x25
[   59.963820]  [<ffffffff8118fac1>] ext4_release_file+0x26/0xa7
[   59.963822]  [<ffffffff81122398>] __fput+0xf9/0x1a7
[   59.963825]  [<ffffffff81122481>] ____fput+0x12/0x1a
[   59.963828]  [<ffffffff810604c5>] task_work_run+0x72/0x8e
[   59.963832]  [<ffffffff8100134e>] prepare_exit_to_usermode+0x81/0x99
[   59.963835]  [<ffffffff810013d8>] syscall_return_slowpath+0x72/0x7d
[   59.963840]  [<ffffffff81488442>] int_ret_from_sys_call+0x25/0x9f
[   59.963846] PAX: size overflow detected in function ext4_mb_new_group_pa fs/ext4/mballoc.c:3696 cicus.1009_82 max, count: 121, decl: pa_lstart; num: 0; context: ext4_prealloc_space;
[   59.979998] CPU: 5 PID: 2002 Comm: dmesg Tainted: G           O    4.5.7-hardened-1 #3
[   59.979999] Hardware name: Supermicro X8DTU/X8DTU, BIOS 2.1b       12/30/2011
[   59.980001]  0000000000000286 0000000000000286 ffffc900016db8a0 ffffffff8123d5c7
[   59.980004]  ffffffff8185b440 ffffffff816f9049 0000000000000e70 ffffc900016db8d0
[   59.980007]  ffffffff81126554 ffff8800bf030000 ffff8800bf033000 ffff8806226f9818
[   59.980011] Call Trace:
[   59.980014]  [<ffffffff8123d5c7>] dump_stack+0x4e/0x71
[   59.980016]  [<ffffffff81126554>] report_size_overflow+0x3f/0x7a
[   59.980019]  [<ffffffff811bafb2>] ext4_mb_new_group_pa+0xf2/0x1fc
[   59.980022]  [<ffffffff811bf1cb>] ext4_mb_new_blocks+0x1b3/0x3ad
[   59.980024]  [<ffffffff811bf1cb>] ? ext4_mb_new_blocks+0x1b3/0x3ad
[   59.980027]  [<ffffffff811b4da0>] ext4_ext_map_blocks+0xd6b/0x1149
[   59.980030]  [<ffffffff811b4da0>] ? ext4_ext_map_blocks+0xd6b/0x1149
[   59.980032]  [<ffffffff81190010>] ? ext4_find_unwritten_pgoff.isra.12+0x8a/0x2d7
[   59.980035]  [<ffffffff81196546>] ext4_map_blocks+0x33b/0x4d1
[   59.980038]  [<ffffffff81196546>] ? ext4_map_blocks+0x33b/0x4d1
[   59.980040]  [<ffffffff81199252>] ext4_writepages+0x45f/0x9a0
[   59.980043]  [<ffffffff81199252>] ? ext4_writepages+0x45f/0x9a0
[   59.980046]  [<ffffffff810db23c>] do_writepages+0x3f/0x50
[   59.980049]  [<ffffffff810db23c>] ? do_writepages+0x3f/0x50
[   59.980051]  [<ffffffff810d26ec>] __filemap_fdatawrite_range+0x5e/0x76
[   59.980054]  [<ffffffff810d2724>] filemap_flush+0x20/0x28
[   59.980056]  [<ffffffff811969f6>] ext4_alloc_da_blocks+0x1c/0x25
[   59.980059]  [<ffffffff8118fac1>] ext4_release_file+0x26/0xa7
[   59.980061]  [<ffffffff81122398>] __fput+0xf9/0x1a7
[   59.980063]  [<ffffffff81122481>] ____fput+0x12/0x1a
[   59.980066]  [<ffffffff810604c5>] task_work_run+0x72/0x8e
[   59.980068]  [<ffffffff8100134e>] prepare_exit_to_usermode+0x81/0x99
[   59.980071]  [<ffffffff810013d8>] syscall_return_slowpath+0x72/0x7d
[   59.980074]  [<ffffffff81488442>] int_ret_from_sys_call+0x25/0x9f
[   61.666124] Netfilter messages via NETLINK v0.30.
[   61.692748] ip_set: protocol 6
[snip]

(The system is loop-aes'ed, and this happens after entering the passphrase, which is why this "early" crash came at 59 secs kernel-time.)

The box that triggered these errors is a Supermicro X8DTU dual Xeon 5645 w/24 GB RAM and 18 TB filesystem.

The other box that has no trouble with 4.5.7-hardened is a Supermicro X9DRH-7TF, 128 GB RAM, 16 TB filesystem.

Comment 1 Hank Leininger 2016-06-12 04:00:15 UTC

FWIW, another box with similar hardware does not trigger overflows in ext4_mb_new_group_pa, using the exact same .config as the problem box.  Varying the .config on the problem box (in various minor ways I would not have expected to make a difference) does not make a difference.

Same: motherboard, RAM size, hardened gcc 4.9.3, Adaptec RAID controller, 18 TB device, loop-aes.

Different: BIOS (stable box has 2.0c, unstable has 2.1b), CPUs (stable: E5607, unstable: E5645), RAM chips (stable: 6x 1066 Kingstons, unstable: 6x 1333 Samsungs).

...And, the contents of the filesystems themselves.  The unstable system's 18 TB array is only 46% full, _but_ it has a quite unusually high inode consumption: over 171 million inodes in use.

I would blame the hardware since I can't reproduce elsewhere yet, but prior to this I've not had any stability issues or unusual errors out of the system that is hitting this.

Comment 2 Anthony Basile gentoo-dev

2016-10-01 14:16:40 UTC

can you test hardened-sources-4.7.6

Comment 3 PaX Team 2016-10-02 12:16:35 UTC

can you still reproduce this? the offending ext4_grp_offs_to_block calculation is a somewhat complex arithmetic expression, we'd need to print out the runtime values of the terms to know if there's a real problem in there. e.g., ext4_free_extent.fe_start is an int that is left shifted and it's sign extended only after that, it can easily truncate (and IIRC the signed shift is undefined if that happens).