# equery f tt-rss * Searching for tt-rss ... * Contents of www-apps/tt-rss-1.13: /.keep_www-apps_tt-rss-1.13 /conf /hooks /htdocs /htdocs/LICENSE /htdocs/README.md /htdocs/api /htdocs/api/index.php /htdocs/atom-to-html.xsl /htdocs/backend.php /htdocs/cache /htdocs/cache/.htaccess /htdocs/cache/export /htdocs/cache/export/.empty /htdocs/cache/images /htdocs/cache/images/.empty /htdocs/cache/js /htdocs/cache/js/.empty /htdocs/cache/simplepie /htdocs/cache/simplepie/.empty /htdocs/cache/upload /htdocs/cache/upload/.empty /htdocs/classes /htdocs/classes/.htaccess /htdocs/classes/api.php /htdocs/classes/article.php /htdocs/classes/auth /htdocs/classes/auth/base.php /htdocs/classes/backend.php /htdocs/classes/db /htdocs/classes/db.php /htdocs/classes/db/mysql.php /htdocs/classes/db/mysqli.php /htdocs/classes/db/pdo.php /htdocs/classes/db/pgsql.php /htdocs/classes/db/prefs.php /htdocs/classes/db/stmt.php /htdocs/classes/dbupdater.php [...] This could have backfired _massively_ It may well be the fault of the webapp eclass. Either way, this needs to be fixed quickly. If a single application has something under etc/, major breakage may happen, up to and including completely hosing installations. The Changelog in the eclass subdir has no entries about webapp-config, but that doesn't mean much. # emerge --info Portage 2.2.14 (python 2.7.8-final-0, default/linux/amd64/13.0, gcc-4.8.3, glibc-2.20, 3.18.0-rc2 x86_64) ================================================================= System uname: Linux-3.18.0-rc2-x86_64-Intel-R-_Core-TM-_i7-3770_CPU_@_3.40GHz-with-gentoo-2.2 KiB Mem: 32847176 total, 6218948 free KiB Swap: 0 total, 0 free Timestamp of tree: Sun, 09 Nov 2014 15:45:01 +0000 ld GNU ld (GNU Binutils) 2.24 app-shells/bash: 4.3_p30 dev-lang/perl: 5.20.1-r2 dev-lang/python: 2.7.8, 3.3.5-r1, 3.4.2 dev-util/cmake: 3.0.2 dev-util/pkgconfig: 0.28-r2 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.13.3 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.14.1 sys-devel/binutils: 2.24-r3 sys-devel/gcc: 4.8.3 sys-devel/gcc-config: 1.8 sys-devel/libtool: 2.4.3-r2 sys-devel/make: 4.1-r1 sys-kernel/linux-headers: 3.16 (virtual/os-headers) sys-libs/glibc: 2.20 Repositories: gentoo local ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=corei7" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=corei7" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms sign strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://gentoo.tiscali.nl/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j6" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync5.de.gentoo.org/gentoo-portage" USE="3dnow acl acpi amd64 apache2 bash-completion bcmath berkdb bzip2 calendar caps clamav cli cracklib crypt cscope curl cxx dri ethereal exif fortran ftp gd gdbm gif gmp gnutls iconv idn imagemagick imap imlib ipv6 jabber jpeg kdeenablefinal libwww maildir mbox mmap mmx mng modules mp3 mpeg mpi multilib mysql mysqli ncurses nls nptl nptlonly ogg oggvorbis openmp pam pcre perl php png posix python readline recode sasl session sockets spell sse sse2 ssl svg tcpd truetype unicode vhosts vim vim-pager vim-syntax vorbis xattr xetex zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias reqtimeout" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="mdraid dash btrfs ssh-client ifcfg network net" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3 python3_4" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="vga" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" USE_PYTHON="2.7 3.3" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(In reply to Tobias Klausmann from comment #0) > # equery f tt-rss > * Searching for tt-rss ... > * Contents of www-apps/tt-rss-1.13: > /.keep_www-apps_tt-rss-1.13 > /conf > /hooks > /htdocs > /htdocs/LICENSE > /htdocs/README.md Yeah this is bad. twitch153 has mostly been taking care of this stuff, but I don't think he did anything to trigger this. Do you know if its isolated ot tt-rss or does it affect other webapp-config stuff? Also, what version of webapp-config are you using?
Any hints how I could trigger that behaviour? I've never seen it on any of my tt-rss installations?
(In reply to Thomas Kahle from comment #2) > Any hints how I could trigger that behaviour? I've never seen it on any of > my tt-rss installations? This looks like vhost_root was set to "/" in /etc/vhosts/webapp-config. Ideally it should be something like "/var/www/${vhost_hostname}" If you changed your webapp-config from the default, please post it. Also, what version of webapp-config are you using?
My webapp-config is below, I don't remember ever changing it. $ grep -v ^# /etc/vhosts/webapp-config|sort -u allow_absolute="no" package_manager="portage" vhost_config_default_dirs="default-owned" vhost_config_dir="${vhost_root}/conf" vhost_config_virtual_files="virtual" vhost_default_gid="root" vhost_default_uid="root" vhost_hostname="localhost" vhost_htdocs_insecure="htdocs" vhost_htdocs_secure="htdocs-secure" vhost_perms_configowned_dir="0755" vhost_perms_configowned_file="0644" vhost_perms_defaultowned_dir="0755" vhost_perms_installdir="0755" vhost_perms_serverowned_dir="0775" vhost_perms_serverowned_file="0664" vhost_perms_virtualowned_file="o-w" vhost_root="/var/www/${vhost_hostname}" vhost_server="apache" WA_CONF_VERSION="7"
I'm using app-admin/webapp-config-1.53, installed 11:36:05 AM 09/07/2014. I'm not sure when this broke, since tt-rss is the only webapp-config'd package I use and I only noticed it when the upgrade to 1.14 happeneds. Normally, I'd suspect the eclass to have broken, but there are no changes there, either. As a result, I suspect some rule about variable expansion changed, maybe in Portage. Note that the docs to webapp.eclass claim that MY_HTDOCS is exported, but I see no code to that effect in it.
Some time ago there was another hiccup with config.php not being protected by webapp-config: bug #493520. We work around this now, but I find these webapp-config related issues impossible to debug :(
So I tried the same thing on one of my other machines: no problems at all. Things that I suspected were at fault (spoiler: none of them are): - Recent glibc update after which I hadn't rebooted. - Python versions - webapp-config somehow got corrupted on one machine[0] - Portage versions - Portage tree version/sync time - Different FEATURES - Different USE-Flags on webapp-config, portage, python. The global USE flags _are_ different since the two machines have very different purposes. [0] Compared MD5sums on both machines, they are the same aside from some of the pyo/pyc files. The machines have different processors, so... I'll keep digging, but I suspect emerge -1 --emptytree is going to be my only recourse. And it may not even help.
(In reply to Tobias Klausmann from comment #7) > So I tried the same thing on one of my other machines: no problems at all. I can't reproduce either. > > Things that I suspected were at fault (spoiler: none of them are): > > - Recent glibc update after which I hadn't rebooted. > - Python versions > - webapp-config somehow got corrupted on one machine[0] > - Portage versions > - Portage tree version/sync time > - Different FEATURES > - Different USE-Flags on webapp-config, portage, python. The global USE > flags _are_ different since the two machines have very different purposes. > > [0] Compared MD5sums on both machines, they are the same aside from some of > the pyo/pyc files. The machines have different processors, so... > > I'll keep digging, but I suspect emerge -1 --emptytree is going to be my > only recourse. And it may not even help. My guess is that its some variable not being exported somewhere. You could try to print out all the variables after config.parseparams() in main of webapp-config using config.get_config(<var>) and see if the variables are different on your two machines. (Exit right aftre that if you want to not actually go through and install the webapp.)
Some more endless digging. Most of it pointless. I added set -x in the eclass (at line 60) to see what falls out of the call to webapp-config --query. On the working machine, I get (amongst other stuff): VHOST_CONFIG_DIR="/var/www/localhost/conf" whereas the broken machine gives me: VHOST_CONFIG_DIR="${vhost_root}/conf" Note how the ${} is not interpolated. So I set out to understand the inner workings of webapp-config, specifically its configuration loading. It's in Python and I am quite well versed in that language. But boy is w-c a pile of excrement. I've put three hours into it, trying to understand how it works, including trying to make sense of its completely broken debug logging (yay, another hour wasted, I should have gone with print-debugging in the first place). Anyway, I still don't know what broke interpolation. I am amazed it ever worked. I'll keep digging.
I found it. It's a bug in webapp-config: it uses ConfigParser as shipped with dev-lang/python. However, it makes the mistake of assuming that the superclass calls _interpolate_some() during parsing. This is wrong since functions prefixed with _ are, by convention, private and should not be relied on. The add-on package dev-python/config-parser, which is a backport of the Python-3 stock versions does not call this method and thus installation of webapps fails in the manner described here. So at the very least w-c needs to have a !dep for d-p/configparser. Ideally, its use of ConfigParser should be fixed.
(In reply to Tobias Klausmann from comment #10) > I found it. > > It's a bug in webapp-config: it uses ConfigParser as shipped with > dev-lang/python. However, it makes the mistake of assuming that the > superclass calls _interpolate_some() during parsing. This is wrong since > functions prefixed with _ are, by convention, private and should not be > relied on. The add-on package dev-python/config-parser, which is a backport > of the Python-3 stock versions does not call this method and thus > installation of webapps fails in the manner described here. > > So at the very least w-c needs to have a !dep for d-p/configparser. Ideally, > its use of ConfigParser should be fixed. Nice work. For the time being I'm going to block against configparser until this is properly fixed.
I can confirm that this is only relevant with Python2.7. I'll be working on removing the internal interpolate_some() function over the next few days so we can remove the blocker. Thanks again for finding this weirdness.
After reviewing the code, I realized that the internal _interpolate_some() function did not need to be removed at all but instead what needed to be fixed was the automagic importing of the dev-python/configparser class. To fix this I forced webapp-config to import ConfigParser if the python version is less than 3.0 as opposed to attempting to import configparser (the py3.x variant) and then if this failed to fall back to the ConfigParser class. This makes the importing more explicit and leaves no room for error when having the backport package dev-python/configparser installed on the system. The change has been made and pushed the webapp-config's experimental branch and after review of the commits residing in that branch we'll merge them to master and release a new version of webapp-config.
*** Bug 508434 has been marked as a duplicate of this bug. ***
Could you please post a link to the experimental branch so I can test/use it?
(In reply to Harold Naparst from comment #15) > Could you please post a link to the experimental branch so I can test/use it? To get the experimental webapp-config code run these commands: 1.) git clone git://git.overlays.gentoo.org/proj/webapp-config.git 2.) cd webapp-config 3.) git checkout experimental Warning: This is an experimental branch for a reason. Some of the stuff on here might not work and stability is not guaranteed. However, we try our best to make sure webapp-config is fully functional. Also be warned that because this is a git repo a new commit could come at any time and your version of webapp-config will not be up to date with the current experimental HEAD unless you git sync. So any bugs in here shouldn't be reported via bugs.gentoo.org. Good luck :)
Created attachment 390884 [details] Ebuild based on the experimental branch This ebuild tracks the experimental branch of webapp-config. I gave it the revision 1.54_alpha because it is intended that the next release from the experimental branch may be called 1.54. This ebuild is not a proposal for portage. It is an ebuild for users who are currently blocked in their work by bug 528752, and have to have configparser and webapp-config both installed. This ebuild allows this. Please note that the syntax of webapp-config as a command has changed. The new syntax is given in the post install notes when this ebuild is run.
If you want to run the attached ebuild, you can do so from my overlay: layman -a hnaparst
This is fixed in webapp-config-1.54 which is currently on the tree. Marking as resolved.
