Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 574354 - sys-apps/coreutils-8.25 cp slows down to 0.7 MB/s on file transfers
Summary: sys-apps/coreutils-8.25 cp slows down to 0.7 MB/s on file transfers
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-10 18:13 UTC by Marco Napetti
Modified: 2016-10-11 12:56 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marco Napetti 2016-02-10 18:13:37 UTC
I'm trasferring all the contents of an USB attached 1TB drive to an internal sata2 drive.
Both drives are ext4 formatted and mounted without using fuse or anything else, the external drive is 99% full.

If I launch it with a single "cp -fruav /mnt/tmp/* /media/data/" it starts with a normal speed, but it slows down to more or less 0.7 MB/s, I don't have a real speed monitor, but I can see it by looking at the growing value of "du -sh /media/data/" or simply by doing "ls -lh" on the directory where it is actually copying the files.
It kept 2 days to transfer more or less 100 GB of data.

I searched on the internet, I found similar problems, but they all end up about missing kernel modules, and I checked my kernel modules, and everything is loaded.

To solve the problem I had to split it to atomic copies, using find and an AD-HOC script:
find /mnt/tmp/ -type f -exec /tmp/script.sh "{}" \;

where /tmp/script.sh was:
#!/bin/bash
cp -fruav "$1" "${1/\/mnt\/tmp/\/media\/data}"
Comment 1 Marco Napetti 2016-02-10 18:15:21 UTC
I forgot to mention that, with the "atomic" solution, it took few hours to finish.
Comment 2 Marco Napetti 2016-02-10 18:18:52 UTC
emerge --info sys-apps/coreutils
Portage 2.2.27 (python 3.4.3-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-5.3.0, glibc-2.22-r1, 4.4.1-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.4.1-gentoo-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_6000+-with-gentoo-2.2
KiB Mem:     2055408 total,    175848 free
KiB Swap:    8393924 total,   8393924 free
Timestamp of repository gentoo: Tue, 09 Feb 2016 17:00:01 +0000
sh bash 4.3_p42-r2
ld ld di GNU (Gentoo 2.25.1 p1.1) 2.25.1
app-shells/bash:          4.3_p42-r2::gentoo
dev-java/java-config:     2.2.0::gentoo
dev-lang/perl:            5.22.1::gentoo
dev-lang/python:          2.7.11-r2::gentoo, 3.4.3-r7::gentoo
dev-util/cmake:           3.4.3::gentoo
dev-util/pkgconfig:       0.29::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.20.4::gentoo
sys-apps/sandbox:         2.10-r1::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r1::gentoo
sys-devel/automake:       1.11.6-r2::gentoo, 1.14.1-r1::gentoo, 1.15-r1::gentoo
sys-devel/binutils:       2.25.1-r1::gentoo
sys-devel/gcc:            5.3.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r1::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.22-r1::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

bliss-overlay
    location: /var/lib/layman/bliss-overlay
    sync-type: laymansync
    sync-uri: https://github.com/fearedbliss/bliss-overlay.git
    masters: gentoo
    priority: 50

kde
    location: /var/lib/layman/kde
    sync-type: laymansync
    sync-uri: git://anongit.gentoo.org/proj/kde.git
    masters: gentoo
    priority: 50

steam-overlay
    location: /var/lib/layman/steam-overlay
    sync-type: laymansync
    sync-uri: git://github.com/anyc/steam-overlay.git
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA skype-4.0.0.7-copyright AdobeFlash-11.x google-chrome Oracle-BCLA-JavaSE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -pipe -O2"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.0/ext-active/ /etc/php/cgi-php7.0/ext-active/ /etc/php/cli-php7.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -pipe -O2"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps=y --quiet-build=y"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="it_IT.UTF-8"
LC_ALL="it_IT.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/home/portage"
USE="X a52 aac acl acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cups cxx dbus declarative dri dts dvd dvdr emboss encode exif fam firefox flac fortran gdbm gif glamor gpm gtk iconv ipv6 jpeg kipi lcms ldap libnotify mad mmx mmxext mng modules mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds qml qt3support qt5 readline sdl seccomp semantic-desktop session spell sse sse2 ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vorbis widgets wxwidgets x264 xattr xcb xcomposite xinerama xml xscreensaver xv xvid zlib" ABI_X86="64 32" ALSA_CARDS="emu10k1" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_core authn_dbd authn_dbm authn_default authn_file authz_dbm authz_core authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif slotmem_shm so speling status socache_shmcb unique_id unixd userdir usertrack vhost_alias" APACHE2_MPMS="worker" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext sse sse2 sse3" DVB_CARDS="tda10046" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="it" LIRC_DEVICES="livedrive_midi" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby23" SANE_BACKENDS="epson2" USERLAND="GNU" VIDEO_CARDS="radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

sys-apps/coreutils-8.25::gentoo was built with the following:
USE="acl nls (policykit) xattr -caps -gmp -hostname -kill -multicall (-selinux) -static -vanilla" ABI_X86="64"
Comment 3 SpanKY gentoo-dev 2016-02-12 17:43:22 UTC
i doubt this is cp related.  try using `rsync -av --progress /mnt/tmp/ /media/data/` and see if it performs any better.
Comment 4 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2016-10-11 12:56:17 UTC
Please do as vapier asked for and report your results here.