Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 379303 - sort crashes on big files
Summary: sort crashes on big files
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Gentoo Linux bug wranglers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-15 17:54 UTC by mike
Modified: 2011-08-15 20:22 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description mike 2011-08-15 17:54:11 UTC
Calling sort -u -S 1G file.txt > output.txt from sys-apps/coreutils-8.7 crashes sort. Increasing memory size doesn't help. The size of file.txt is approximately 1G. The program was called with other text-files with similar size and it crashes only in some cases (same file every time). I cant upload it here because of the size, but it's just a plain textfile.

Reproducible: Always

Steps to Reproduce:
1. call sort -u -S 1G file.txt > output.txt with a min 1G plaintext-file
Actual Results:  
Sort crashes with "Killed" or "Segmenation fault", depending on the file. Also some sortXXXX-files in /tmp ARE NOT deleted if sort crashes! Your disk will become clobbered with trash in a very short time when sorting big files.

Expected Results:  
NO crash!
Comment 1 mike 2011-08-15 17:55:47 UTC
Portage 2.1.10.3 (default/linux/x86/10.0, gcc-4.4.5, glibc-2.12.2-r0, 2.6.38+ i686)
=================================================================
System uname: Linux-2.6.38+-i686-Intel-R-_Core-TM-2_Duo_CPU_L9400_@_1.86GHz-with-gentoo-2.0.3
Timestamp of tree: Mon, 15 Aug 2011 11:45:01 +0000
ccache version 2.4 [enabled]
app-shells/bash:          4.1_p9
dev-java/java-config:     2.1.11-r3
dev-lang/python:          2.6.6-r2, 2.7.1-r1, 3.1.3-r1
dev-util/ccache:          2.4-r9
dev-util/cmake:           2.8.4-r1
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.8.3-r1
sys-apps/sandbox:         2.4
sys-devel/autoconf:       2.13, 2.68
sys-devel/automake:       1.9.6-r3, 1.11.1
sys-devel/binutils:       2.20.1-r1
sys-devel/gcc:            4.4.5
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82
sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers)
sys-libs/glibc:           2.12.2
Repositories: gentoo
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.0/conf /usr/share/openvpn/easy-rsa /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=native -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs ccache distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync3.de.gentoo.org/gentoo-portage"
USE="X acl acpi alsa bash-completion berkdb bzip2 cli cracklib crypt cups cxx dri dri2 dvi fortran gdbm gpm iconv ipv6 jpeg libv4l2 mmx modules mudflap ncurses nls nptl nptlonly opengl openmp pam pcre perl png pppd python qt3support qt4 readline sdl session sse sse2 sse3 ssl ssse3 sysfs tcpd threads unicode v4l v4l2 vaapi x86 xinerama xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="braindump flow karbon kexi kpresenter krita tables words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 2 Alex Legler (RETIRED) archtester gentoo-dev Security 2011-08-15 18:00:42 UTC
We'll need a stack trace to debug this further.
See http://www.gentoo.org/proj/en/qa/backtraces.xml
Comment 3 mike 2011-08-15 19:02:40 UTC
I didn't compile with debugsymbols so these ones are missing in the following backtrace. This bug only seems to happen if sort is called with "-u" option.


Starting program: /bin/sort -u -S 1G bigfile.txt
[Thread debugging using libthread_db enabled]
[New Thread 0x77af2b70 (LWP 7473)]
[Thread 0x77af2b70 (LWP 7473) exited]
[New Thread 0x77af2b70 (LWP 7828)]
[Thread 0x77af2b70 (LWP 7828) exited]
[New Thread 0x77af2b70 (LWP 8248)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x77af2b70 (LWP 8248)]
0x0804c3a3 in ?? ()
#0  0x0804c3a3 in ?? ()
No symbol table info available.
#1  0x0804c65d in ?? ()
No symbol table info available.
#2  0x0804d4c4 in ?? ()
No symbol table info available.
#3  0x0804da61 in ?? ()
No symbol table info available.
#4  0xb7fb4d23 in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#5  0xb7f22bfe in clone () from /lib/libc.so.6
No symbol table info available.
Comment 4 mike 2011-08-15 19:57:48 UTC
I've made an upstream-report..
Comment 5 mike 2011-08-15 20:22:04 UTC
Seems to be solved in Version 8.8, which is not stable on gentoo until now.