Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 39515 - sort -g borks on large files in coreutils-5.0.91-r{3,4}
Summary: sort -g borks on large files in coreutils-5.0.91-r{3,4}
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Seemant Kulleen (RETIRED)
URL:
Whiteboard:
Keywords:
: 64222 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-01-26 16:32 UTC by qube99
Modified: 2004-12-22 00:43 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
sort.c.patch (sort.c.patch,341 bytes, patch)
2004-08-21 20:56 UTC, Sven Wegener
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description qube99 2004-01-26 16:32:49 UTC
I've tried coreutils-5.0.91-r{2,3,4}, so this problem seems to have been introduced in -r3.  Running sort -g on a file larger than ~25MB causes sort to use up all memory then either a) crash the system or b) cause the OOM killer to kill it.

Reproducible: Always
Steps to Reproduce:
1. Run sort -g on a large data set, 25MB or larger.  Problem is reproduceable in valid sorts ("<number> ..." formatted lines) and non-valid (I tested on a large tar.gz)

Actual Results:  
Sort quickly grows consuming all physical memory, draining cache and buffers, 
then draining swap file.  System becomes mostly unresponsive, the oom killer 
kills sort (and possible another program (xmms in my case)), and life goes on. 

Expected Results:  
Grown to a sane size (seems to be ~40MB-100MB) and sorted the results. 

Gentoo Base System version 1.4.3.12 
Portage 2.0.50_pre16 (default-x86-1.4, gcc-3.3.2, 
glibc-2.2.5-r2,2.3.3_pre20031222-r0, 2.6.0-gentoo-r2) 
================================================================= 
System uname: 2.6.0-gentoo-r2 i686 AMD Athlon(tm) Processor 
ccache version 2.2 [enabled] 
Autoconf: 
sys-devel/autoconf-2.58,sys-devel/autoconf-2.52d-r1,sys-devel/autoconf-2.59 
Automake: sys-devel/automake-1.7.8 
ACCEPT_KEYWORDS="x86" 
AUTOCLEAN="no" 
CFLAGS="-march=athlon-xp -O2 -pipe" 
CHOST="i686-pc-linux-gnu" 
COMPILER="gcc3" 
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.1/share/config /usr/kde/3.2/share/config /usr/kde/3/share/config /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" 
CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" 
CXXFLAGS="-march=athlon-xp -O2 -pipe" 
DISTDIR="/usr/portage/distfiles" 
FEATURES="autoaddcvs ccache fixpackages sandbox userpriv" 
GENTOO_MIRRORS="http://gentoo.seren.com/gentoo ftp://mirrors.tds.net/gentoo 
http://mirrors.tds.net/gentoo ftp://csociety-ftp.ecn.purdue.edu/pub/gentoo/ 
http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo/" 
MAKEOPTS="-j2" 
PKGDIR="/usr/portage/packages" 
PORTAGE_TMPDIR="/mnt/large/portage/tmp" 
PORTDIR="/usr/portage" 
PORTDIR_OVERLAY="/usr/local/portage" 
SYNC="rsync://rsync.gentoo.org/gentoo-portage" 
USE="X Xaw3d aalib acl apm arts avi berkdb bonobo cdr crypt cups dga directfb 
doc dvd dvdr encode esd evo foomaticdb gdbm ggi gif gnome gpm gtk gtk2 gtkhtml 
guile imlib jpeg kde ldap libg++ libwww mad mikmod motif mozilla mpeg mysql 
nas ncurses nls oggvorbis opengl oss pam pdflib perl png python qt quicktime 
readline ruby samba scanner sdl slang spell sse ssl svga tcltk tcpd tetex 
truetype usb x86 xinerama xml2 xmms xv zlib"
Comment 1 Seemant Kulleen (RETIRED) gentoo-dev 2004-03-05 19:35:34 UTC
can you reproduce this in coreutils-5.2.0? (about to check into portage)
Comment 2 qube99 2004-03-24 19:48:18 UTC
Yes, it occurs in 5.2.0 too.
Comment 3 Seemant Kulleen (RETIRED) gentoo-dev 2004-07-02 13:49:26 UTC
I'm not able to either reproduce or find anyone who can troubleshoot.  Not sure what to do about this one, to be honest.  I'm assuming 5.2.1 still displays this behaviour?

qube, can you edit the coreutils-5.2.1 ebuild and remove all the epatch lines (comment out the entire src_unpack function, come to think of it), and see if vanilla coreutils suffers?
Comment 4 Seemant Kulleen (RETIRED) gentoo-dev 2004-07-02 13:50:29 UTC
I'm not able to either reproduce or find anyone who can troubleshoot.  Not sure what to do about this one, to be honest.  I'm assuming 5.2.1 still displays this behaviour?

qube, can you edit the coreutils-5.2.1 ebuild and remove all the epatch lines (comment out the entire src_unpack function, come to think of it), and see if vanilla coreutils suffers?
Comment 5 qube99 2004-08-20 23:01:42 UTC
Right, coreutils-5.2.1's sort borks as well with all the patches.  Without the patches, coreutils-5.2.1's sort seems to behave sanely.
Comment 6 Seemant Kulleen (RETIRED) gentoo-dev 2004-08-20 23:19:25 UTC
Sven, wanna have a look see?
Comment 7 Sven Wegener gentoo-dev 2004-08-21 20:56:52 UTC
Created attachment 37910 [details, diff]
sort.c.patch

There ya go Seemant.
Comment 8 qube99 2004-08-22 15:15:06 UTC
The patch seems to fix it for me for both valid and invalid files.
Comment 9 Seemant Kulleen (RETIRED) gentoo-dev 2004-08-25 18:52:02 UTC
thanks Sven.     fixed in -r2
Comment 10 Jens Teubner 2004-12-22 00:43:46 UTC
*** Bug 64222 has been marked as a duplicate of this bug. ***