'etc-update' doesn't work for configuration files with space in the file name, e.g. portage message: * IMPORTANT: config file '/etc/bareos/bareos-dir.d/fileset/Windows All Drives.conf' needs updating. * See the CONFIGURATION FILES and CONFIGURATION FILES UPDATE TOOLS * sections of the emerge man page to learn how to update config files. backupserver / # etc-update Scanning Configuration files... Skipping non-file /etc/bareos/bareos-dir.d/fileset/._cfg0000_Windows ... Skipping non-file All ... Skipping non-file Drives.conf ... Exiting: Nothing left to do; exiting. :) Reproducible: Always Steps to Reproduce: 1. # etc-update Actual Results: Scanning Configuration files... Skipping non-file /etc/bareos/bareos-dir.d/fileset/._cfg0000_Windows ... Skipping non-file All ... Skipping non-file Drives.conf ... Exiting: Nothing left to do; exiting. :) Expected Results: etc-update's menu Portage 2.3.51 (python 2.7.15-final-0, hardened/linux/amd64, gcc-7.3.0, glibc-2.27-r6, 4.14.81-gentoo x86_64) ================================================================= System uname: Linux-4.14.81-gentoo-x86_64-Quad-Core_AMD_Opteron-tm-_Processor_2350-with-gentoo-2.6 KiB Mem: 32903004 total, 1213016 free KiB Swap: 33554364 total, 33540284 free Timestamp of repository gentoo: Wed, 21 Nov 2018 07:45:01 +0000 Head commit of repository gentoo: e78fb2d070f4d937f3dff13605debce687c25d02 sh bash 4.4_p12 ld GNU ld (Gentoo 2.30 p5) 2.30.0 app-shells/bash: 4.4_p12::gentoo dev-lang/perl: 5.24.3-r1::gentoo dev-lang/python: 2.7.15::gentoo, 3.5.5::gentoo, 3.6.5::gentoo dev-util/cmake: 3.9.6::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.69-r4::gentoo sys-devel/automake: 1.15.1-r2::gentoo sys-devel/binutils: 2.30-r4::gentoo sys-devel/gcc: 7.3.0-r3::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.14-r1::gentoo (virtual/os-headers) sys-libs/glibc: 2.27-r6::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://wwwcache.cluster.local/gentoo-portage priority: -1000 sync-rsync-verify-jobs: 1 sync-rsync-verify-metamanifest: no sync-rsync-extra-opts: sync-rsync-verify-max-age: 24 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=native -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=native -O2 -pipe" DISTDIR="/usr/portage/distfiles" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_GB.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 berkdb bzip2 cli crypt cxx device-mapper dri gdbm hardened iconv ipv6 libtirpc mmx multilib ncurses nls nptl openmp openssl pam pcre pie readline seccomp sse sse2 ssl ssp systemd tcpd unicode urandom xattr xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby23 ruby24" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
NB: Original file does not exist since it was renamed to 'WindowsAll.conf' and edited: backupserver / # ls -la /etc/bareos/bareos-dir.d/fileset/ total 40 drwxr-xr-x 2 bareos bareos 4096 Nov 21 09:23 . drwxr-xr-x 15 bareos bareos 4096 Jun 28 18:11 .. -rw-r----- 1 bareos bareos 312 Jul 24 14:28 Catalog.conf -rw-r----- 1 root root 362 Nov 20 18:12 '._cfg0000_Windows All Drives.conf' -rw-r--r-- 1 bareos bareos 56 Jul 26 05:53 Empty.conf -rw-r----- 1 bareos bareos 1695 Jul 25 17:51 FS00_Backups.conf -rw-r----- 1 bareos bareos 888 Jul 26 05:59 FullSet.conf -rw-r----- 1 bareos bareos 1612 Jul 26 15:12 LinuxAll.conf -rw-r----- 1 bareos bareos 211 Sep 20 11:42 SelfTest.conf -rw-r----- 1 bareos bareos 421 Nov 21 09:23 WindowsAll.conf
The first problem I see is this for loop which uses whitespace to separate values of ${file}: > for file in $(find "${path}"/ "${find_opts[@]}" | > sed \ > -e 's://*:/:g' \ > -e "${scan_regexp}" | > sort -t"$b" -k2,2 -k4,4 -k3,3 | > LC_ALL=C cut -f1 -d"$b") > do We can fix it like this: while read -r -d ''; do stuff done < <(find ... -print0)
This still happens: ... * IMPORTANT: config file '/etc/testssl/DST Root CA X3.txt' needs updating. ... # etc-update Scanning Configuration files... Skipping non-file /etc/testssl/._cfg0000_DST ... Skipping non-file Root ... Skipping non-file CA ... Skipping non-file X3.txt ... Exiting: Nothing left to do; exiting. :) # ls -l /etc/testssl/._cfg0000_DST* -rw-r--r-- 1 root root 1200 Dec 6 02:00 '/etc/testssl/._cfg0000_DST Root CA X3.txt' # equery f testssl | egrep ' ' /etc/testssl/DST Root CA X3.txt # equery b /usr/sbin/etc-update * Searching for /usr/sbin/etc-update ... sys-apps/portage-3.0.56-r1 (/usr/sbin/etc-update) # equery l testssl * Searching for testssl ... [I-O] [ ] net-analyzer/testssl-3.2_rc3_p20231110:0 This is a bump of testssl vs what is in ::gentoo, and in it, that 'DST Root CA X3.txt' has changed in minor/trivial ways, but that's just one example, I think anything w/spaces in the path will fail. (In reply to Zac Medico from comment #2) > We can fix it like this: > > while read -r -d ''; do > stuff > done < <(find ... -print0) I think this is on the right track but that ... is doing a lot of work, bc the logic in the current find pipeline is complex. I don't have enough brain right now to flesh it out. I suppose other problems lurk, for example CONFIG_PROTECT is a space-separated list; how does it handle spaces in paths, and should it? Perhaps etc-update, CONFIG_PROTECT and the like should not try, and spaces in config paths & filename should just be banned; it would be up to ebuilds to rename files and patch things looking for them. I could not find any such rule in an exhaustive 30 seconds of Googling.
(In reply to Hank Leininger from comment #3) > I suppose other problems lurk, for example CONFIG_PROTECT is a > space-separated list; how does it handle spaces in paths, and should it? Should work with embedded quotes, since we use the shlex.split function. > Perhaps etc-update, CONFIG_PROTECT and the like should not try, and spaces > in config paths & filename should just be banned; it would be up to ebuilds > to rename files and patch things looking for them. I could not find any such > rule in an exhaustive 30 seconds of Googling. We really need to support it if other package managers do, or else it will be a pain. Looks like rpm supports it: https://rpm-list.redhat.narkive.com/7WUOZXa6/basic-question-space-in-file-name
Currently, PMS only says that CONFIG_PROTECT is space-delimited and says nothing about embedded quotes. We can use shlex.split then requote in " ".join(mylist) to support spaces in CONFIG_PROTECT in lib/portage/util/env_update.py here: > for var in space_separated: > mylist = [] > for myconfig in config_list: > if var in myconfig: > for item in myconfig[var].split(): > if item and not item in mylist: > mylist.append(item) > del myconfig[var] # prepare for env.update(myconfig) > if mylist: > env[var] = " ".join(mylist) > specials[var] = mylist I've checked the eselect.env module and it also does not quote spaces within an item here: > elif has ${var} ${SPACE_CLASS}; then > # Space separated > store=$(load_config "${tmpprofile}" ${var}) > if [[ -z ${store} ]]; then > store=$(load_config "${envfile}" ${var}) > else > items=$(load_config "${envfile}" ${var}) > for item in ${items}; do > has ${item} ${store} \ > || store="${store}${store:+ }${item}" > done > fi > else
(In reply to Zac Medico from comment #5) > I've checked the eselect.env module and it also does not quote spaces within > an item here: Obviously, the elements of a whitespace delimited list cannot contain whitespace. It is a limit of the spec, so nothing that eselect can do about it. We should really consider limiting names of configuration files to a sane character set, in order to avoid strange problems like this.
CONFIG_PROTECT and CONFIG_PROTECT_MASK usually contain directory paths. I think it is rare to specify the full path to individual files. We might therefore consider allowing file names to contain spaces, but ban it for directories. Or we can just live with the fact that there will be paths that are impossible to declare in CONFIG_PROTECT and CONFIG_PROTECT_MASK. Either way, etc-update should be fixed to function properly with any installed file name.
