Transocding always fails with "dl_fn->cuda_dl->cuInit(0) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected". Reproducible: Always Steps to Reproduce: 1. install jellyfin-bin on cuda enabled HW 2. set NVENC hw encoding in settings 3. try to play video with transcode (e.g. lowering stream resolution or playing browser unsupported format like xvid or such) Actual Results: when run as a systemd service "systemctl start jellyfin", transcoding fails. when run in bash under jellyfin user like "/opt/jellyfin/jellyfin --configdir /etc/jellyfin/ --webdir /opt/jellyfin/jellyfin-web/ --datadir /var/lib/jellyfin". transcoding works In attached logs, this is visible on lines 27-30(31). Expected Results: transcoding should work when started as systemd unit emerge --info Portage 3.0.66.1 (python 3.12.7-final-0, default/linux/amd64/23.0/desktop/plasma/systemd, gcc-14, glibc-2.40-r4, 6.11.0-gentoo x86_64) ================================================================= System uname: Linux-6.11.0-gentoo-x86_64-AMD_Ryzen_9_5950X_16-Core_Processor-with-glibc2.40 KiB Mem: 98774360 total, 65584680 free KiB Swap: 49386492 total, 49257980 free Timestamp of repository gentoo: Mon, 07 Oct 2024 08:18:31 +0000 Head commit of repository gentoo: b64c99c49f3644a652f1fa10dd9a956f37a454ad Timestamp of repository guru: Mon, 07 Oct 2024 02:48:16 +0000 Head commit of repository guru: 3dd5900a4fd0447399278911bf4686dc64d314ca Timestamp of repository science: Wed, 02 Oct 2024 09:19:09 +0000 Head commit of repository science: b7cc92184c09b98c0b25661fab9500ec8d33eea7 Timestamp of repository steam-overlay: Sun, 06 Oct 2024 22:03:15 +0000 Head commit of repository steam-overlay: e14708f6779dfa290feb8c8f391d34d483c73002 Timestamp of repository xarblu-overlay: Fri, 04 Oct 2024 19:48:18 +0000 Head commit of repository xarblu-overlay: 63aedff473499c465a52070389def05cfe1b99fb sh bash 5.2_p37 ld GNU ld (Gentoo 2.43 p2) 2.43.1 distcc 3.4 x86_64-pc-linux-gnu [disabled] app-misc/pax-utils: 1.3.8::gentoo app-shells/bash: 5.2_p37::gentoo dev-build/autoconf: 2.72-r1::gentoo dev-build/automake: 1.17-r1::gentoo dev-build/cmake: 3.30.4::gentoo dev-build/libtool: 2.5.3::gentoo dev-build/make: 4.4.1-r100::gentoo dev-build/meson: 1.5.2::gentoo dev-java/java-config: 2.3.4::gentoo dev-lang/perl: 5.40.0::gentoo dev-lang/python: 3.11.10_p1::gentoo, 3.12.7_p1::gentoo, 3.13.0_rc3::gentoo dev-lang/rust-bin: 1.81.0::gentoo sys-apps/baselayout: 2.15::gentoo sys-apps/sandbox: 2.39::gentoo sys-apps/systemd: 256.6::gentoo sys-devel/binutils: 2.43-r1::gentoo sys-devel/binutils-config: 5.5.2::gentoo sys-devel/clang: 18.1.8::gentoo, 19.1.1::gentoo sys-devel/gcc: 12.4.1_p20240919::gentoo, 14.2.1_p20240921::gentoo sys-devel/gcc-config: 2.11::gentoo sys-devel/llvm: 18.1.8-r4::gentoo, 19.1.1::gentoo sys-kernel/linux-headers: 6.10::gentoo (virtual/os-headers) sys-libs/glibc: 2.40-r4::gentoo Repositories: gentoo location: /var/db/repos/gentoo sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 volatile: False sync-git-verify-commit-signature: yes guru location: /var/db/repos/guru sync-type: git sync-uri: https://github.com/gentoo-mirror/guru.git masters: gentoo volatile: False local location: /var/db/repos/local masters: gentoo volatile: False science location: /var/db/repos/science sync-type: git sync-uri: https://github.com/gentoo-mirror/science.git masters: gentoo volatile: False steam-overlay location: /var/db/repos/steam-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/steam-overlay.git masters: gentoo volatile: False xarblu-overlay location: /var/db/repos/xarblu-overlay sync-type: git sync-uri: https://github.com/gentoo-mirror/xarblu-overlay.git masters: gentoo volatile: False Binary Repositories: binhost priority: 9999 sync-uri: https://distfiles.gentoo.org/releases/amd64/binpackages/23.0/x86-64 Installed sets: @esteam ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="@FREE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=znver3 -pipe " CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d" CXXFLAGS="-O2 -march=znver3 -pipe " DISTDIR="/var/cache/distfiles" EMERGE_DEFAULT_OPTS="--jobs 1 --load-average 15" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" FCFLAGS="-O2 -march=znver3 -pipe " FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-request-signature buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -march=znver3 -pipe " GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs" LEX="flex" LINGUAS="cs en en_US" MAKEOPTS="-j31" PKGDIR="/var/cache/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" SHELL="/bin/bash" USE="X a52 aac acl acpi activities alsa amd64 appstream bluetooth bluray branding brightness-control bzip2 cairo cdda cddb cdr cet crypt cuda cups dav1d dbus declarative dri drm dts dvd dvdr egl encode exif faad ffmpeg fftw flac fontconfig fuse gdbm gif gpg gpm grub gtk gui iconv icu ipv6 jpeg jpeg2k kde kf6compat kwallet lcms libdrm libnotify librtmp libtirpc live lm-sensors mad midi mng mp3 mp4 mpeg mplayer mtp multilib multimedia multipath musepack ncurses networkmanager nls numa nvenc nvidia ogg opencl opencv opengl openmp openvdb opus pam pango parted pcre pdf pipewire pkcs11 plasma png policykit powermanagement ppds pulseaudio pypy3 python qml qt5 qt6 qtmedia readline samba sasl screencast sddm sdl seccomp semantic-desktop sensors sound speex spell srt ssl startup-notification sudo svg synaptics systemd taglib test-rust theora tiff tray truetype twolame udev udisks unicode upower usb v4l vaapi vdpau virtfs vlc vorbis vulkan wayland webp websockets widgets wifi wxwidgets x11extras x264 x265 xattr xcb xft xml xv xvid xwayland zip zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 vpclmulqdq" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="pc" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput wacom synaptics evdev" KERNEL="linux" L10N="cs cs-CZ en en-US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_TARGETS="BPF NVPTX X86" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby32" VIDEO_CARDS="nvidia nvk nouveau d3d12" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Created attachment 905067 [details] jellyfin as systemd service - transcode not working
Created attachment 905068 [details] jellyfin run from terminal - transcode working
Hmm... As you use the jellyfin user for both invocations the only thing I can think of is that is has to do with the systemd sandboxing. I don't have any cuda HW to test with, and scanning the documentation I can't see any obvious solution. You can probably troubleshoot by overloading/replacing the jellyfin.service unit-file while modifying/removing the various sandboxing options until you find what option causes this failure. https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#Sandboxing
(In reply to Fredrik Eriksson from comment #3) > Hmm... As you use the jellyfin user for both invocations the only thing I > can think of is that is has to do with the systemd sandboxing. I don't have > any cuda HW to test with, and scanning the documentation I can't see any > obvious solution. > > You can probably troubleshoot by overloading/replacing the jellyfin.service > unit-file while modifying/removing the various sandboxing options until you > find what option causes this failure. > > https://www.freedesktop.org/software/systemd/man/latest/systemd.exec. > html#Sandboxing Hmmm, I already tried without any luck, now I wiped all restrictions of systemd unit (thus mimicking systemd unit from Arch, where it works), and it somehow started working.. Will be back after I find out which restriction is the culprit.
So, I found out that problem lies in the systemd unit, specifically part with " DeviceAllow=char-drm rw DeviceAllow=char-nvidia-frontend rw DeviceAllow=char-nvidia-uvm rw " Thing is that addition of DeviceAllow disables otherwise default DevicePolicy=auto, which in turn makes nvidia HW encoding dysfunctional. So, for correct behavior of HW encoding, commenting out all "DeviceAllow" parts works, but I'm unsure if this shouldn't be solved by finding out which "DeviceAllow" to add. That's sadly out of my powers, if you don't have any tip..
(In reply to Martin Rott from comment #5) > So, > > I found out that problem lies in the systemd unit, > specifically part with > " > DeviceAllow=char-drm rw > DeviceAllow=char-nvidia-frontend rw > DeviceAllow=char-nvidia-uvm rw > " > > Thing is that addition of DeviceAllow disables otherwise default > DevicePolicy=auto, which in turn makes nvidia HW encoding dysfunctional. I guess that makes sense considering the error message. > > So, for correct behavior of HW encoding, commenting out all "DeviceAllow" > parts works, but I'm unsure if this shouldn't be solved by finding out which > "DeviceAllow" to add. That's sadly out of my powers, if you don't have any > tip.. "The device node specifier is either a path to a device node in the file system, starting with /dev/, or a string starting with either "char-" or "block-" followed by a device group name, as listed in /proc/devices"[0] So the line to add is probably "DeviceAllow=char-<something from /proc/devices> rw". If you can't see anything obvious to try from the character device section of /proc/devices you could try to compare the content of /proc/devices with and without the nvidia module loaded; I assume it is a device group added by the nvidia driver that is missing. [0]https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html#DeviceAllow=
Great, exactly the push I needed! Included is the patch that seems to work... TLDR: addition of "DeviceAllow=char-nvidia rw" made it work..
Created attachment 906120 [details, diff] Patch
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=109e8f3aa0cd7465cafaeb798853896465884358 commit 109e8f3aa0cd7465cafaeb798853896465884358 Author: Fredrik Eriksson <feffe@fulh.ax> AuthorDate: 2024-10-17 15:37:44 +0000 Commit: Craig Andrews <candrews@gentoo.org> CommitDate: 2024-10-17 15:59:34 +0000 www-apps/jellyfin-bin: Fix NVENC HW acceleration Bump revision to make sure users receive the new systemd unit. While here: Clean up old versions. Closes: https://bugs.gentoo.org/941107 Suggested-by: Martin Rott Signed-off-by: Fredrik Eriksson <feffe@fulh.ax> Closes: https://github.com/gentoo/gentoo/pull/39020 Signed-off-by: Craig Andrews <candrews@gentoo.org> www-apps/jellyfin-bin/Manifest | 8 --- www-apps/jellyfin-bin/files/jellyfin.service | 1 + ....9.10.ebuild => jellyfin-bin-10.9.11-r1.ebuild} | 0 www-apps/jellyfin-bin/jellyfin-bin-10.9.9.ebuild | 77 ---------------------- 4 files changed, 1 insertion(+), 85 deletions(-)
