I have a server running gentoo-sources-5.10.x and a client running libreelec inside a kvm which runs on gentoo running gentoo-sources-5.4.92 the server exports a folder via nfs as follows: /mnt/share 10.0.0.0/24(ro,nohide,insecure,no_subtree_check) up until 5.10.10, I was able to view fhd content via nfs from the server on libreelec. since 5.10.11, I'm unable to, I see artifacts random sounds and it usually stops there. Reproducible: Always
Created attachment 685803 [details] config.gz
$ emerge --info gentoo-sources nfs-utils Portage 3.0.13 (python 3.7.9-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-10.2.0, glibc-2.32-r5, 5.10.11-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.10.11-gentoo-x86_64-AMD_Ryzen_7_2700X_Eight-Core_Processor-with-gentoo-2.7 KiB Mem: 32834168 total, 29202588 free KiB Swap: 16437244 total, 16437244 free Head commit of repository gentoo: aa7b59dfb5f221348eb495f358a7b953839e3bf3 sh bash 5.0_p18 ld GNU ld (Gentoo 2.35.1 p2) 2.35.1 app-shells/bash: 5.0_p18::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.30.3::gentoo dev-lang/python: 2.7.18-r6::gentoo, 3.7.9-r2::gentoo, 3.8.7-r1::gentoo, 3.9.1-r1::gentoo dev-util/cmake: 3.18.5::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.7::gentoo sys-apps/openrc: 0.42.1-r1::gentoo sys-apps/sandbox: 2.20::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r5::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.16.2-r1::gentoo sys-devel/binutils: 2.35.1-r1::gentoo sys-devel/gcc: 10.2.0-r5::gentoo sys-devel/gcc-config: 2.3.2-r1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.3::gentoo sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers) sys-libs/glibc: 2.32-r5::gentoo Repositories: gentoo location: /mnt/portage/tree/official sync-type: git sync-uri: https://anongit.gentoo.org/git/repo/gentoo.git priority: -1000 Utopia_Platinia location: /mnt/portage/tree/overlay masters: gentoo priority: 0 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="@FREE @FREE AdobeFlash-11.x MPEG-4 Skype-TOS linux-fw-redistributable no-source-code hplip-plugin OPERA-2014 unRAR fs2_open Alasir all-rights-reserved ms-teams-pre OPERA-2018" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native -mtune=native" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=native -mtune=native" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--keep-going --quiet-build=n" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=native -mtune=native" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs clean-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userpriv usersandbox usersync" FFLAGS="-O2 -pipe -march=native -mtune=native" GENTOO_MIRRORS="ftp://ftp.gtlib.gatech.edu/pub/gentoo ftp://ftp.ussg.iu.edu/pub/linux/gentoo" LANG="en_IL.US-ASCII" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en es" MAKEOPTS="-j17 -l8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--timeout=180" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi activities alsa amd64 berkdb bidi branding bzip2 cairo cdda cdr cli crypt cryptsetup cups d3d9 dbus declarative dri drm dts dvd dvdr elogind emboss encode exif ffmpeg flac fortran gdbm geckodriver gif gpm gtk gui iconv icu ipv6 java joystick jpeg kde kipi kwallet lcms libglvnd libnotify libtirpc lm-sensors lm_sensors lto lvm mad matroska mng mp3 mp4 mpeg multilib ncurses nls nptl ogg opencl opengl openmp pam pango pcre pdf pdfimport pgo phonon physics plasma png policykit postproc ppds python qml qt5 rar readline samba sdl seccomp sound spell split-usr sqlite ssl startup-notification svg system-harfbuzz system-heimdal system-icu system-jpeg system-jsoncpp system-libevent system-libvpx system-llvm system-sqlite system-webp tcpd tiff truetype udev udisks unicode upower usb vaapi vdpau vlc vorbis vulkan widgets wxwidgets x264 x265 xattr xcb xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64 pc" INPUT_DEVICES="evdev" KERNEL="linux" L10N="he es en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="AArch64 ARM AMDGPU BPF" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python3_7" QEMU_SOFTMMU_TARGETS="x86_64 arm aarch64" QEMU_USER_TARGETS="x86_64 arm aarch64" RUBY_TARGETS="ruby26" SANE_BACKENDS="hp" USERLAND="GNU" VIDEO_CARDS="v4l amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS ================================================================= Package Settings ================================================================= sys-kernel/gentoo-sources-5.10.13::gentoo was built with the following: USE="experimental -build -symlink" ABI_X86="(64)" net-fs/nfs-utils-2.5.2::gentoo was built with the following: USE="ipv6 libmount nfsidmap nfsv4 nfsv41 tcpd uuid -caps -junction -kerberos -ldap -nfsdcld (-selinux)" ABI_X86="(64)"
Are you familiar with performing a git bisect ?
@daggystyle please follow this guide https://wiki.gentoo.org/wiki/Kernel_git-bisect
yes, I know how to preform it, will update when I know more.
Created attachment 685836 [details] bisect log
bisect found this: 00ee972739fb2526d3936f1e7ccfc8c91d250c60 is the first bad commit commit 00ee972739fb2526d3936f1e7ccfc8c91d250c60 Author: Chuck Lever <chuck.lever@oracle.com> Date: Fri Dec 18 12:28:41 2020 -0500 SUNRPC: Handle TCP socket sends with kernel_sendpage() again [ Upstream commit 4a85a6a3320b4a622315d2e0ea91a1d2b013bce4 ] Daire Byrne reports a ~50% aggregrate throughput regression on his Linux NFS server after commit da1661b93bf4 ("SUNRPC: Teach server to use xprt_sock_sendmsg for socket sends"), which replaced kernel_send_page() calls in NFSD's socket send path with calls to sock_sendmsg() using iov_iter. Investigation showed that tcp_sendmsg() was not using zero-copy to send the xdr_buf's bvec pages, but instead was relying on memcpy. This means copying every byte of a large NFS READ payload. It looks like TLS sockets do indeed support a ->sendpage method, so it's really not necessary to use xprt_sock_sendmsg() to support TLS fully on the server. A mechanical reversion of da1661b93bf4 is not possible at this point, but we can re-implement the server's TCP socket sendmsg path using kernel_sendpage(). Reported-by: Daire Byrne <daire@dneg.com> BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=209439 Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Sasha Levin <sashal@kernel.org> net/sunrpc/svcsock.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 85 insertions(+), 1 deletion(-) bisect log attached
also reported at https://bugs.archlinux.org/task/69491 and https://github.com/xbmc/xbmc/issues/19147
reverting this commit on 5.10.13 confirms it is the issue as nfs playback is working again
this: https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git/commit/?h=nfsd-5.11-3&id=bad4c6eb5eaa8300e065bd4426727db5141d687d might fix the issue
5.10.14 doesn't seem to have this fix in it
(In reply to DaggyStyle from comment #10) > this: > https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git/commit/?h=nfsd- > 5.11-3&id=bad4c6eb5eaa8300e065bd4426727db5141d687d might fix the issue applying this patch on 5.10.14 fixes the issue
(In reply to DaggyStyle from comment #12) > (In reply to DaggyStyle from comment #10) > > this: > > https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git/commit/?h=nfsd- > > 5.11-3&id=bad4c6eb5eaa8300e065bd4426727db5141d687d might fix the issue > > applying this patch on 5.10.14 fixes the issue Sorry, I've been out of pocket on a new build for my kernel building system but now I'm alive. I'll check this out and queue it up for release.
For me: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/net/sunrpc/svcsock.c?id=bad4c6eb5eaa8300e065bd4426727db5141d687d
Queued for next release commit f2b7022ccc048d545f23b383a7716a0cf9c7304f (HEAD -> 5.10, origin/5.10) Author: Mike Pagano <mpagano@gentoo.org> Date: Tue Feb 9 14:09:13 2021 -0500 SUNRPC: Fix NFS READs that start at non-page-aligned offsets See https://bugs.gentoo.org/768720 Thanks to DaggyStyle for reporting and testing Signed-off-by: Mike Pagano <mpagano@gentoo.org>
