Bug 162864 – x11-base/xorg-server possible memory leak on some arches

• Status: RESOLVED UPSTREAM
• Product: Gentoo Linux

[Description Constantin Baranov 2007-01-20 00:53:02 UTC]

By default X.org on some architectures prevents itself to use mmap() for memory allocation. This may cause memory fragmentation after some application has created and than destroyed large pixmap in X's heap.

Source files: hw/xfree86/loader/loader.c and hw/xfree86/loader/elfloader.c.

Possible solution: define symbols UseMMAP and/or DoMMAPedMerge in CFLAGS.

I have compiled xorg-server with CFLAGS="(usual flags) -DUseMMAP -DDoMMAPedMerge". It works stable (nvidia-drivers) without hard memory fragmentation.

Reproducible: Always

Steps to Reproduce:
1. Run any application creates large pixmaps. For example, open and scroll large djvu-book in KDVI.
2. Run some other X11 applications.
3. Close application launched in first step.

Actual Results:  
X server cannot free memory where large pixmap was placed. Resident size of X process much more than actually required.

Expected Results:  
When mmap() is enabled large pixmaps is allocated using mmap() and can be really freed in any order regardless of any other objects exist in X's address space.

Portage 2.1.1-r2 (default-linux/amd64/2006.1, gcc-4.1.1, glibc-2.4-r4, 2.6.19.2 x86_64)
=================================================================
System uname: 2.6.19.2 x86_64 Intel(R) Pentium(R) D CPU 2.80GHz
Gentoo Base System version 1.12.6
Last Sync: Fri, 19 Jan 2007 01:47:01 +0000
app-admin/eselect-compiler: [Not Present]
dev-java/java-config: [Not Present]
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  [Not Present]
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.14
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-pipe -O3 -march=nocona -ffunction-sections"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c"
CXXFLAGS="-pipe -O3 -march=nocona -ffunction-sections"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer parallel-fetch sandbox sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /usr/portage/local/layman/sunrise"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 X alsa bitmap-fonts bzip2 cli cracklib cups dlloader dvd elibc_glibc encode gif glut iconv input_devices_keyboard input_devices_mouse jpeg jpeg2k kde kdeenablefinal kernel_linux leim mad mikmod ncurses nptl nptlonly opengl pam pdf png readline ssl svg threads tiff truetype truetype-fonts type1-fonts udev unicode userland_GNU video_cards_nvidia video_cards_vesa xml xorg xvid zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS

[Comment 1 Jakub Moc (RETIRED) 2007-01-20 00:59:23 UTC]

Please, report such stuff upstream.

[Comment 2 Joshua Baergen (RETIRED) 2007-01-27 20:32:05 UTC]

I agree with Jakub.  Please find or file a bug at https://bugs.freedesktop.org and post the link here.  Thanks!

[Comment 3 Constantin Baranov 2007-01-28 20:45:29 UTC]

It doesn't seem like a Xorg's bug. Look at hw/xfree86/loader/loader.c:395! I think we should provide additional USE flag (mmap?) which would append "-D UseMMAP -D DoMMAPedMerge" to CFLAGS.

[Comment 4 Joshua Baergen (RETIRED) 2007-01-28 22:31:21 UTC]

I don't think you referenced the right line, or else you have a different copy.  Please paste here whatever you're talking about.

And I don't feel particularly inclined to add special defines in Gentoo's install of the server unless there's a really good reason.  It really should be added in the upstream makefiles instead.

[Comment 5 Constantin Baranov 2007-02-07 19:33:58 UTC]

=x11-base/xorg-server-1.2.0 doesn't use mallopt() to prevent mmap() usage and works fine.