Hello. I have dash as eselect sh (so /bin/sh is symlink to /bin/dash). Every time the system boots, I see the message "/lib/rc/sh/openrc-run.sh: 258: ulimit: too many arguments" after the OpenRC log line "Setting up lightdm", so I'm supposing there is some wrong ulimit usage somewhere near lightdm rc script. The systems boots up normally and no erratic behavior is observed, but that error message bothers me.
Please post your `emerge --info sys-apps/openrc' output in a comment.
Portage 2.3.103 (python 3.7.9-final-0, default/linux/amd64/17.0/desktop, gcc-9.3.0, glibc-2.31-r6, 5.4.48-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-5.4.48-gentoo-x86_64-Intel-R-_Core-TM-_i3-2100_CPU_@_3.10GHz-with-gentoo-2.6 KiB Mem: 16385664 total, 9009884 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Sat, 22 Aug 2020 06:05:32 +0000 Head commit of repository gentoo: ff0eb849040d4004858809bb547a1683e06fbc39 sh dash 0.5.11.1 ld GNU ld (Gentoo 2.33.1 p2) 2.33.1 ccache version 3.7.10 [enabled] app-shells/bash: 5.0_p18::gentoo dev-java/java-config: 2.3.1::gentoo dev-lang/perl: 5.30.3::gentoo dev-lang/python: 2.7.18-r1::gentoo, 3.6.11-r2::gentoo, 3.7.9::gentoo, 3.8.5::gentoo dev-util/ccache: 3.7.10::gentoo dev-util/cmake: 3.18.1::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.18::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.33.1-r1::gentoo sys-devel/gcc: 6.5.0-r3::gentoo, 9.3.0-r1::gentoo sys-devel/gcc-config: 2.3.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.31-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: git sync-uri: https://github.com/gentoo-mirror/gentoo.git priority: -1000 localrepo location: /usr/local/portage masters: gentoo 0x4d4c location: /var/lib/layman/0x4d4c masters: gentoo priority: 50 AzP location: /var/lib/layman/AzP masters: gentoo priority: 50 Drauthius location: /var/lib/layman/Drauthius masters: gentoo priority: 50 deadbeef-overlay location: /var/lib/layman/deadbeef-overlay masters: gentoo priority: 50 didactic-duck location: /var/lib/layman/didactic-duck masters: gentoo priority: 50 dotnet location: /var/lib/layman/dotnet masters: gentoo priority: 50 fkmclane location: /var/lib/layman/fkmclane masters: gentoo priority: 50 gentoo-zh location: /var/lib/layman/gentoo-zh masters: gentoo priority: 50 hossie location: /var/lib/layman/hossie masters: gentoo priority: 50 lisp location: /var/lib/layman/lisp masters: gentoo priority: 50 raiagent location: /var/lib/layman/raiagent masters: gentoo priority: 50 reagentoo location: /var/lib/layman/reagentoo masters: gentoo priority: 50 science location: /var/lib/layman/science masters: gentoo priority: 50 sk-overlay location: /var/lib/layman/sk-overlay masters: gentoo priority: 50 tranquility location: /var/lib/layman/tranquility masters: gentoo priority: 50 vampire location: /var/lib/layman/vampire masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -mtune=native -O2 -fomit-frame-pointer -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=native -mtune=native -O2 -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs ccache compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.yandex.ru/gentoo-distfiles/" LANG="ru_RU.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en en_US ru ru_RU" MAKEOPTS="-j5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr clang cli cmake corefonts crypt cscope cuda cups curl dbus dga djvu dot dri dts dvd dvdr dvi elogind emboss encode evdev exif flac fontconfig fortran gdbm gif git graphviz gtk gtkstyle http2 iconv icu idn ipv6 jemalloc jit jpeg latex lcms ldap libglvnd libnotify libtirpc lldb lz4 lzma lzo mad mng mp3 mp4 mpeg multilib natspec ncat ncurses nls nptl ntlm ntp offensive ogg opencl opengl openmp pam pango pch pcre pdf png policykit ppds qt5 readline samba sdl seccomp sftp spell split-usr ssl startup-notification subversion svg t1lib tcpd tiff truetype udev udisks unicode upower usb vaapi valgrind vdpau vim-syntax vorbis vpx widevine wma wxwidgets x264 xattr xcb xinerama xml xps xscreensaver xv xvid zlib zsh-completion" ABI_X86="64" ADA_TARGET="gnat_2018" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en en-US ru ru-RU" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby25" SANE_BACKENDS="pixma" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= sys-apps/openrc-0.42.1::gentoo was built with the following: USE="ncurses netifrc pam (split-usr) unicode -audit -bash -debug -newnet (-prefix) (-selinux) -static-libs -sysv-utils" ABI_X86="(64)" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
Andrew, please adjust line #258 of /lib/rc/sh/openrc-run.sh so that it reads as follows. { set -x; ulimit ${rc_ulimit:-$RC_ULIMIT}; set +x; } The next time you start the offending service, it will print the ulimit command that is being executed. Having established which (erroneous) options are being used, it should then be possible to track down where they are being set and why.
Thanks Kerin. I've done this and it looks like this ulimit error happens in docker init script, namely the following: ulimit -c unlimited -n 1048576 -u unlimited
I see. POSIX only defines the -f option for setting RLIMIT_FSIZE, and its absence for querying it. dash supports many more options, but -u is not among them. Yet again, this is a case of things only being tested with bash. Really, the way in which support for resource limits is implemented by OpenRC is very weak, as it blindly passes options to the ulimit builtin, rendering it dependent on the /bin/sh implementation. Perhaps it ought to be implemented elsewhere, such as in start-stop-daemon. I don't use docker but a possible solution might be to install a file under /etc/security/limits.d for handling the definition of resource limits. However, that might not be feasible unless the service is expected to run under a specific user account, which appears not to be the case. Another option is to effectively sweep the issue under the carpet by installing openrc with the "bash" USE flag set. In any case, and given that this particular issue is assigned to the OpenRC team, I think that the least OpenRC could do is to provide better diagnostic information in the case that the ulimit call fails. I'll suggest a patch soon.
openrc doesn't provide any diagnostics for the RC_ULIMIT variable; the diagnostic is provided by the shell. If you want that diagnostic to be better, I would suggest opening a bug against the shell. If I provide diagnostics myself, I would have to parse RC_ULIMIT, and if I do that I would limit it to posix options and fail if something else was defined. I can't dforce the bash use flag on because we have users that do not use bash and that would break things for them. OpenRC's code is written for posix sh and we prefer that service scripts be posix sh.
Thanks for commenting, William. With all due respect, I think you are taking a rather narrow view of the situation. I did not propose that RC_ULIMIT options should be parsed. That would be absurd, not to mention unworkable. On the contrary, I suggested that the ability to set rlimits might be better implemented in, say, start-stop-daemon. Anyway, let's step back for a moment and try to look at it from Andrew's point of view. He was greeted with an error which:- 1) did not report the offending service (leaving him to incorrectly guess) 2) did not report the offending options Now, the current code does not even bother to check the exit status value! If it did, it would be trivial to augment the STDERR spew with a concise error message detailing both of these things. This information would help users such as Andrew not to misdirect their bug reports, and it would - on occasion - help developers to fix their poorly conceived runscripts. In this case, it would certainly have saved me from walking Andrew through the process of tracking down the offending runscript. That's what I mean by "better diagnostic information". OpenRC is in charge of running the ulimit builtin and knows full well which runscript it is dealing with, but it doesn't bother to report anything in the case that ulimit yields a non-zero status value, much less treat it as an error. I was intending to submit a patch that improves it in this regard. Regarding the bash USE flag, I was only explaining to Andrew that it was a workaround for his issue. Not a particularly desirable one, I might add. Regarding the point about OpenRC being intended to interoperate with POSIX sh, I do not know how you could read my commentary on this or, indeed, many of the other bugs I have submitted that touch upon this topic and come to the conclusion that I am not fully cognisant of this. Indeed, Gentoo is riddled with issues concerning POSIX conformance, not least in OpenRC itself (the use of the local keyword for instance).
I know that I just mentioned it in the docker bug, but I'll repeat it here because it directly concerns OpenRC and the subject of this bug. Aside from the obvious issue of option portability (only -f is standard), it's not reasonable to expect that shells have a ulimit builtin that allows for multiple option/value arguments. Certainly, dash does not allow for it, which is exactly why the "too many arguments" message came up for Andrew. Effectively, the handling of rlimits is crippled in OpenRC because it is non-portable by design (relying on the sh implementation) and has no expectation of potentially needing to run ulimit more than once, nor does it even know how to do so.
Created attachment 681355 [details, diff] openrc-raise-useful-warnings-if-ulimit-fails.patch Here is the promised patch. It does nothing to advance the means by which OpenRC allows for resource limit configuration. However, it does result in OpenRC printing genuinely useful warning message in the case that the call to ulimit fails.
The following commits, which will be in OpenRC 0.43, make the diagnostic more specific by identifying the service script that is affected. https://github.com/OpenRC/openrc/commit/9a669887 https://github.com/OpenRC/openrc/commit/170ce262 I am opening the below issue upstream to look into using system calls to set the limits directly in start-stop-daemon and supervise-daemon. https://github.com/OpenRC/openrc/issues/399 I would rather have one place to track that fix, so I am going to close this bug. Thanks, William
Aw, I just submitted a patch :( Will take a look anyway.
OK, I think my patch is more useful. I would appreciate a review, when you have the time.
Created attachment 681370 [details, diff] openrc-raise-useful-warnings-if-ulimit-fails-r1.patch Submitting one last time, as the previous patch had a typo in one of the ewarn message strings (rc_limit vs rc_ulimit). The logic remains the same.
I think this issue is simpler than it looks. Dash just doesn't support setting multiple limits with one call to "ulimit". For example, `man dash` says that both -s and -n should be supported. This works in bash, $ ulimit -s unlimited -n 2048 but not in dash, $ dash $ ulimit -s unlimited -n 2048 dash: 1: ulimit: too many arguments So the problem is really that having a single variable rc_ulimit that gets passed to "ulimit" requires bash when you want to set more than one limit. If I want to set two limits, a diagnostic doesn't help. There's nothing wrong with my conf.d file.
(In reply to Michael Orlitzky from comment #14) > I think this issue is simpler than it looks. Dash just doesn't support No, it's not that simple. I have already explained why it's not this simple, both here and in bug 763891. > setting multiple limits with one call to "ulimit". For example, `man dash` > says that both -s and -n should be supported. This works in bash, > > $ ulimit -s unlimited -n 2048 > > but not in dash, > > $ dash > $ ulimit -s unlimited -n 2048 > dash: 1: ulimit: too many arguments > > So the problem is really that having a single variable rc_ulimit that gets > passed to "ulimit" requires bash when you want to set more than one limit. You're not adding any new information here. The fact that specifying more than one argument isn't portable is something that I have already mentioned, both in comment #8 and in the sixth comment of bug 763891. To act as if this is the only issue would be fallacious, given that there is no portable option other than -f, as I have mentioned both here and in bug 763891. See for yourself: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ulimit.html Please do read bug 763891 because it also explains why you can't even rely on options being portable between bash and dash, let alone other sh(1) implementations. Good luck getting getting -u to work in dash, for instance; it has to be changed to -p instead. You know, it's almost as if non-standard options can't be relied upon to work in the same way - if at all - across different sh(1) implementations. To whit, the options are as follows: 1) Continue to continue to stick our heads in the sand and labour under the delusion that rc_ulimit suffices as a generalised mechanism of setting limits while runscripts continue to be executed by sh(1). 2) Give up on this pretence of being - or caring about being - POSIX-conformant and have runscripts always be executed by /bin/bash. 3) Implement another mechanism to control resource limits and encourage its use. I wouldn't recommend the first option and I doubt that that the second would gain any traction. From an engineering standpoint, the third appears to make the most sense and would be in line with several other popular supervisory suites. > > If I want to set two limits, a diagnostic doesn't help. There's nothing > wrong with my conf.d file. Nonsense. If your conf.d file specifies a superset of the options that are supported by POSIX sh, and your choice of /bin/sh implementation happens to be something that does not support them, then your conf.d file is problematic, by definition. By default, openrc-run targets sh(1). Either accept that and deal with it or campaign for a different approach. A diagnostic does help. Decorating the STDERR spew that various sh(1) implementations may produce does help. My patch demonstrably improves the situation by specifically reporting which shell is involved and by making a reasonable effort to determine whether the offending setting(s) were sourced from a service-specific conf.d file. In the event that they were, it reports the offending service name. How is that not an obvious improvement upon the present status quo? If it were my call, I'd go even further and make it a fatal error. Besides, it is apparent that William agreed that the diagnostic could be improved, because he committed a modest improvement to that end. Is it your conviction that openrc would be in no worse a position, were that commit to be reverted? I rather doubt it.
(In reply to Kerin Millar from comment #15) I'll grant you that I may not be adding any new information, but otherwise we seem to be in violent agreement. I say that nothing is wrong with my conf.d file because (a) my conf.d file is using the interface provided by openrc and knows nothing about the underlying shell; and (b) there's nothing I can do to my conf.d file to make it work the way it should. If there's no "right" way to use rc_ulimit, then rc_ulimit is the wrong interface. The problem is, in a sense, simple: rc_ulimit was always the wrong interface and we should have something portable instead. A bunch of service script variables like limit_stack, limit_nofile, etc. supported by openrc-run and implemented through setrlimit() sounds like a good idea to me.
(In reply to Michael Orlitzky from comment #16) > (In reply to Kerin Millar from comment #15) > > I'll grant you that I may not be adding any new information, but otherwise > we seem to be in violent agreement. I say that nothing is wrong with my > conf.d file because (a) my conf.d file is using the interface provided by > openrc and knows nothing about the underlying shell; and (b) there's nothing > I can do to my conf.d file to make it work the way it should. If there's no > "right" way to use rc_ulimit, then rc_ulimit is the wrong interface. I see. Put that way, I concur. > > The problem is, in a sense, simple: rc_ulimit was always the wrong interface > and we should have something portable instead. Right. Decorating ulimit failures does marginally improve the user experience in the case that they run up against the shortcomings of this interface. At the same time, these shortcomings are serious enough that a new interface is merited. I am relieved that this is understood.
