when using "group: files" in /etc/nsswitch.conf as suggested at https://wiki.gentoo.org/wiki/Glibc_2.26_porting_notes/nsswitch.conf_in_glibc-2.26 the bundled "getent" utility skips long(er) lines in output when querying /etc/group without a key ("getent group"). in my case lines with more than 762 characters (many group members) are not printed. no error message seen. when repeating the "oversized" line in /etc/group, it is printed N-1 times. when using a key ("getent group $groupname") the "oversized" line is printed correctly. after switching to "group: compat" all lines are being printed as normal. i assume this issue is related to the underlaying "getgrent()" function since perl behaves the same when using "getgrent()". does "compat" (beside the NIS stuff) mean that line length does not matter and "files" implies some limit? Reproducible: Always Steps to Reproduce: 1. put many members in /etc/group until the length of the corresponding line exceeds 762 characters; use "group: files" in /etc/nsswitch.conf 2. $ getent group; -- or -- 3. $ perl -e 'setgrent(); while( my @g = getgrent() ){ printf( "%s: %s\n", $g[0], $g[3] ); }; endgrent();'; Actual Results: missed lines of /etc/group with > 762 chars Expected Results: all lines of /etc/group without skipping some of them silently. if there is any limit, an error message should go to stderr. Portage 2.3.49 (python 3.6.5-final-0, default/linux/amd64/17.0/no-multilib, gcc-7.3.0, glibc-2.27-r6, 4.14.65-jason-default-x86_64-1540277370 x86_64) ================================================================= System uname: Linux-4.14.65-jason-default-x86_64-1540277370-x86_64-Intel-R-_Pentium-R-_CPU_G6950_@_2.80GHz-with-gentoo-2.4.1 KiB Mem: 10148240 total, 1221144 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Tue, 13 Nov 2018 13:00:01 +0000 Head commit of repository gentoo: 15aa69164a64d75fe7e472b1ea2c91abfc0fb444 sh bash 4.4_p12 ld GNU ld (Gentoo 2.30 p5) 2.30.0 app-shells/bash: 4.4_p12::gentoo dev-lang/perl: 5.24.3-r1::gentoo dev-lang/python: 2.7.15::gentoo, 3.6.5::gentoo dev-util/cmake: 3.9.6::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1-r2::gentoo sys-apps/openrc: 0.38.3::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.69-r4::gentoo sys-devel/automake: 1.11.6-r3::gentoo, 1.15.1-r2::gentoo sys-devel/binutils: 2.30-r4::gentoo sys-devel/gcc: 7.3.0-r3::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers) sys-libs/glibc: 2.27-r6::gentoo Repositories: gentoo location: /var/db/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-extra-opts: sync-rsync-verify-metamanifest: no sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 layer23 location: /dat/local/share/portage/db masters: gentoo ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -fomit-frame-pointer -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/apache2-php7.0/ext-active/ /etc/php/apache2-php7.1/ext-active/ /etc/php/apache2-php7.2/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cgi-php7.0/ext-active/ /etc/php/cgi-php7.1/ext-active/ /etc/php/cgi-php7.2/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/php/cli-php7.0/ext-active/ /etc/php/cli-php7.1/ext-active/ /etc/php/cli-php7.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -fomit-frame-pointer -pipe" DISTDIR="/tmp" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="de_DE.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j3" PKGDIR="/tmp" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl alsa amd64 berkdb bzip2 cli cxx dri fortran gd iconv imap jpeg ldap libtirpc mbox mysql ncurses nptl openmp pam pcre perl png readline sasl seccomp ssl syslog tiff truetype unicode xml zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias asis auth_basic auth_digest filter authn_core authn_file authn_anon authz_core authz_groupfile authz_host authz_owner authz_user access_compat cache cache_disk cgi deflate dumpio dav dav_fs dir env expires headers include info log_config macro mime mime_magic proxy proxy_balancer proxy_connect proxy_fcgi proxy_http ratelimit rewrite setenvif slotmem_shm so socache_shmcb status substitute userdir unique_id unixd vhost_alias" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NGINX_MODULES_HTTP="access auth_basic charset dav fastcgi geo gzip limit_req lua map memcached proxy rewrite ssl stub_status upstream_keepalive" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" QEMU_SOFTMMU_TARGETS="x86_64" RUBY_TARGETS="ruby23 ruby24" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
I failed to reproduce it in a chroot [1]. If you can reproduce it can you craft /etc/passwd and /etc/group files that exhibit the failure and attach it here along with /etc/nsswitch.conf? [1]: my attempt: # groupadd testgroup # getent group testgroup testgroup:x:30001: # for i in $(seq 1 100); do useradd testuser${i} -G testgroup; done # getent group testgroup # getent group testgroup testgroup:x:30001:testuser1,testuser2,testuser3,testuser4,testuser5,testuser6,testuser7,testuser8,testuser9,testuser10,testuser11,testuser12,testuser13,testuser14,testuser15,testuser16,testuser17,testuser18,testuser19,testuser20,testuser21,testuser22,testuser23,testuser24,testuser25,testuser26,testuser27,testuser28,testuser29,testuser30,testuser31,testuser32,testuser33,testuser34,testuser35,testuser36,testuser37,testuser38,testuser39,testuser40,testuser41,testuser42,testuser43,testuser44,testuser45,testuser46,testuser47,testuser48,testuser49,testuser50,testuser51,testuser52,testuser53,testuser54,testuser55,testuser56,testuser57,testuser58,testuser59,testuser60,testuser61,testuser62,testuser63,testuser64,testuser65,testuser66,testuser67,testuser68,testuser69,testuser70,testuser71,testuser72,testuser73,testuser74,testuser75,testuser76,testuser77,testuser78,testuser79,testuser80,testuser81,testuser82,testuser83,testuser84,testuser85,testuser86,testuser87,testuser88,testuser89,testuser90,testuser91,testuser92,testuser93,testuser94,testuser95,testuser96,testuser97,testuser98,testuser99,testuser100 # getent group testgroup | wc -c 1110
using your example i get the same results. but as i wrote in my example ("steps to reproduce"), the issue occurs when omitting the second argument (the <key>) and only using the db-name "group" as the only argument to "getent". then the specific line is simply skipped in output. similar behaviour using the mentioned perl-oneliner. sry for replying late. extending your example: # getent group | grep ^testgroup: | wc -l; 0 with "group: files" in /etc/nssswitch.conf 1 with "group: compat" in /etc/nssswitch.conf
Let's check if this is fixed in 2.28
Not a regression in 2.28 anyway. Did you file an upstream bug? If yes please link to it here.
(In reply to Nico Rittner from comment #2) > using your example i get the same results. but as i wrote in my example > ("steps to reproduce"), the issue occurs when omitting the second argument > (the <key>) and only using the db-name "group" as the only argument to > "getent". then the specific line is simply skipped in output. similar > behaviour using the mentioned perl-oneliner. sry for replying late. > > extending your example: > # getent group | grep ^testgroup: | wc -l; > 0 with "group: files" in /etc/nssswitch.conf > 1 with "group: compat" in /etc/nssswitch.conf Thank you! Managed to reproduce it on glibc-2.27 but not on glibc-2.29. Let's see if I can bisect it down to the fix.
Bisected it down to: 916124ed841745b7a1e0fbc43f9909340b47d373 is the first bad commit commit 916124ed841745b7a1e0fbc43f9909340b47d373 Author: Florian Weimer <fweimer@redhat.com> Date: Fri Jul 6 14:23:15 2018 +0200 nss_files: Fix re-reading of long lines [BZ #18991] Use the new __libc_readline_unlocked function to pick up reading at the same line in case the buffer needs to be enlarged. :100644 100644 51e6bdcefc5efeb34efb3f221513622cd4a92475 7a052c344a10cf3c9c89bf8ad1c8e5a535cab192 M ChangeLog :040000 040000 f03d96780ae1f73923e51333dcad792ac3bfb686 a737d442aacbd3fbbb76159aa8b9535f5a0e7aa2 M nss Looks like a legitimate fix: https://sourceware.org/PR18991 Fixed in glibc-2.28.
glibc-2.28 is stable everywhere.