Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 471682 - dev-lang/php: bundled libmagic code can conflict with file libmagic.so leading to crashes
Summary: dev-lang/php: bundled libmagic code can conflict with file libmagic.so leadin...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: x86 Linux
: Normal normal with 5 votes (vote)
Assignee: PHP Bugs
URL: https://bugs.php.net/bug.php?id=66095
Whiteboard:
Keywords:
: 467756 482950 508386 520268 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-05-29 09:48 UTC by Frank Limpert
Modified: 2016-07-12 22:27 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
apache error_log (error_log-20130526.gz,150.21 KB, application/x-gzip)
2013-05-29 09:48 UTC, Frank Limpert
Details
emerge-info to comment #2 (emerge-info.txt,5.00 KB, text/plain)
2013-06-26 06:29 UTC, Palatonka
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frank Limpert 2013-05-29 09:48:38 UTC
Created attachment 349542 [details]
apache error_log

After update from sys-apps/file-5.11 to sys-apps/file-5.12-r1 apache crashed frequently when php pages (especially css) were requested. The first thing I noticed was my mediawiki looking ugly without any style sheet. Please see the attached error_log. After some days of re-compiling the whole system and rolling back different packages to former versions that ran stable on my system I found that my system became stable again after going back to sys-apps/file-5.11
Comment 1 SpanKY gentoo-dev 2013-05-30 01:14:10 UTC
i don't suppose you could find a simple php file that triggered the crash ?  apache serving css shouldn't involve php.

also, please post `emerge --info` as an attachment in all of your bug reports.
Comment 2 Palatonka 2013-06-26 06:28:34 UTC
I have the same problem that after the update from sys-apps/file-5.11 to sys-apps/file-5.12-r1 the CSS in Mediawiki could not be loaded anymore.

Also in a non hardened env.

The only way to get Mediawiki running is to mask the newer sys-apps/file.

I will attach my emerge info.
Comment 3 Palatonka 2013-06-26 06:29:46 UTC
Created attachment 351984 [details]
emerge-info to comment #2
Comment 4 Palatonka 2013-06-26 06:36:05 UTC
Here is a CSS reference link which crashes:

http://<MY_IP>/w/load.php?debug=false&lang=de&modules=mediawiki.legacy.commonPrint%2Cshared|skins.vector&only=styles&skin=vector&*

(In reply to Palatonka from comment #2)
Comment 5 nebojsa 2013-07-06 20:27:51 UTC
Having the same problem on a non hardened armv5. 
gdb shows that php is calling fileinfo using libmagic, and than glibc complains about free and causes seg fault.
Please revert back file-5.12-r1 to unstable until this is fixed.
Also owncloud has issues with file-5.12-r1, making it impossible to activate plugins (they are zip archives, and again fileinfo is used).
Simply removing (and thus disabling file) /usr/share/misc/magic.mgc "fixes" issues with Mediawiki.
Comment 6 Joschi Kuphal 2013-07-11 21:06:39 UTC
The issue has also been reported as a PHP bug (https://bugs.php.net/bug.php?id=64836) occuring in conjunction with -D SVN as Apache startup parameter. Downgrading to sys-apps/file-5.11 seems to solve the issue in this case.
Comment 7 KevinOfOz 2013-08-06 14:58:02 UTC
(This is my first bug report)

I am also having problems with apache seg-faulting.

This script seg-faults with file-5.12-r1

<?php
$finfo = finfo_open();
$fn = __FILE__;
echo "File ".$fn." is of type ".finfo_file($finfo,$fn);
finfo_close($finfo);
?>

using apache-2.2.25, php-5.4.17 & sys-apps/file-5.12-r1

Downgrading to sys-apps/file-5.11 solves the issue.

emerge --info

Portage 2.1.12.2 (default/linux/amd64/13.0, gcc-4.6.3, glibc-2.15-r3, 3.8.13-gentoo x86_64)
=================================================================
System uname: Linux-3.8.13-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5504_@_2.00GHz-with-gentoo-2.2
KiB Mem:     2054224 total,   1111676 free
KiB Swap:     262140 total,    185676 free
Timestamp of tree: Tue, 06 Aug 2013 01:00:01 +0000
ld GNU ld (GNU Binutils) 2.23.1
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.5, 3.2.5-r1
dev-util/cmake:           2.8.10.2-r2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.11.8
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.9.6-r3, 1.11.6, 1.12.6
sys-devel/binutils:       2.23.1
sys-devel/gcc:            4.6.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.7 (virtual/os-headers)
sys-libs/glibc:           2.15-r3
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA dlj-1.1"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/www/localhost/htdocs/b2evolution/conf /var/www/localhost/htdocs/joomla /var/www/localhost/htdocs/mediawiki /var/www/localhost/htdocs/phpBB"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="ftp://ftp.filearena.net/pub/gentoo ftp://mirror.pacific.net.au/linux/Gentoo http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j10"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl amd64 apache2 bzip2 c++ cli cracklib crypt curl cxx gd gdbm gif gs hal hddtemp iconv imagemagick jpeg jpeg2k mmx modules mudflap multilib mysql mysqli ncurses nptl nss openssl pam pcre pdo php png python readline session sqlite sqlite2 sqlite3 sse sse2 ssl subversion tcpd threads threadsonly tiff tokenizer truetype unicode usb userlocales vhosts webdav-neon webdav-serf xml xmlreader zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-3 php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 8 Christopher Crammond 2013-08-25 23:39:35 UTC
I am experience a similar issue on a non hardened x64 hardware; however, downgrading to file-5.11 didn't resolve my issue.

www-servers/apache-2.2.25:2
sys-apps/file-5.11:0
dev-lang/php-5.4.17:5.4
dev-lang/php-5.5.2:5.5
Comment 9 Andrey Niahajchyk 2013-08-29 08:40:52 UTC
See also comment in https://bugs.gentoo.org/show_bug.cgi?id=467188#c11
Comment 10 Palatonka 2013-10-21 05:43:41 UTC
After a portage update, there is no sys-apps/file-5.11 anymore to downgrade to and the mask '>sys-apps/file-5.11' doesn't work also.

emerge output:
The following mask changes are necessary to proceed:
# required by sys-apps/file
=sys-apps/file-5.12-r1

BUT the CSS still cannot be loaded.
Comment 11 Palatonka 2013-10-21 05:48:00 UTC
# eix sys-apps/file
[D] sys-apps/file
     Available versions:  [m]5.12-r1 [m]~5.15 [m]**9999 {python static-libs zlib PYTHON_TARGETS="pypy2_0 python2_6 python2_7 python3_2 python3_3"}
     Installed versions:  5.12-r1(10:58:27 18.10.2013)(python zlib -static-libs)
     Homepage:            http://www.darwinsys.com/file/
     Description:         identify a file's format by scanning binary data for patterns

# emerge =sys-apps/file-5.11
emerge: there are no ebuilds to satisfy "=sys-apps/file-5.11".
Comment 12 Palatonka 2013-10-21 08:09:10 UTC
How to force or manually emerge sys-apps/file-5.11?
Comment 13 Palatonka 2013-10-21 08:10:54 UTC
Also the unstable sys-apps/file-5.15 doesn't fix this problem.
Comment 14 Palatonka 2013-10-21 10:25:58 UTC
(In reply to Palatonka from comment #12)

I've now unmerged file-5.12-r1 and emerged file-5.11 with local portage overlay.

But what about the dependencies now?
Comment 15 hactar 2013-12-06 14:40:26 UTC
Thanks to the tips here I could finally get both SVN and mediawiki to function. I can confirm that the only solution is to downgrade to file-5.11, which is no longer in the portage tree :(. file-5.16 does not resolve this issue.
Comment 16 J.C. Wren 2013-12-26 23:19:03 UTC
Same here. I ran into this issue with file-5.12 came out, and backtracked it to the file-5.12 upgrade breaking it. Going back to 5.11 fixed it. 

I upgraded my system over the weekend, file-5.16 was installed, and both svn and mediawiki broke. I had forgotten about the fix (although I had masked 5.12 and 5-12.r1, I didn't mask anything greater). 

After spending 3 hours tracking it down, creating a portage overlay, yada yada yada, I got 5.11 re-installed, and all is well.

Quick list:
  mkdir -p /usr/local/portage/metadata
  mkdir -p /usr/local/portage/sys-apps/file
  echo 'masters = gentoo' >/usr/local/portage/metadata/layout.conf
  echo 'PORTDIR_OVERLAY="/usr/local/portage"' >>/etc/portage/make.conf
  echo '>=sys-apps/file-5.12' >/etc/portage/package.mask/file
  wget http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/sys-apps/file/file-5.11.ebuild?revision=1.10 -O /usr/local/portage/sys-apps/file/file-5.11.ebuild
  ebuild /usr/local/portage/sys-apps/file/file-5.11.ebuild manifest
  emerge -q file

(Adjust accordingly if make.conf is somewhere else, you already have an overlay directory, etc).
Comment 17 Daniel Minder 2014-02-14 15:18:47 UTC
My bug looks a bit different, but is triggered by the same code as in https://bugs.php.net/bug.php?id=64836#1391894588

*** Error in `/usr/sbin/apache2': free(): invalid pointer: 0x00007f67acfc1068 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7fc56)[0x7f67e886fc56]
/lib64/libc.so.6(+0x80a80)[0x7f67e8870a80]
/usr/lib64/libmagic.so.1(+0x5277)[0x7f67deef9277]
/usr/lib64/php5.5/apache2/libphp5.so(finfo_resource_destructor+0x29)[0x7f67de236309]
/usr/lib64/php5.5/apache2/libphp5.so(list_entry_destructor+0x79)[0x7f67de4efaa9]
/usr/lib64/php5.5/apache2/libphp5.so(zend_hash_del_key_or_index+0x209)[0x7f67de4ed1e9]
/usr/lib64/php5.5/apache2/libphp5.so(_zend_list_delete+0xb6)[0x7f67de4efcd6]
/usr/lib64/php5.5/apache2/libphp5.so(zif_finfo_close+0x77)[0x7f67de2362a7]
/usr/lib64/php5.5/apache2/libphp5.so(+0x551317)[0x7f67de5aa317]
/usr/lib64/php5.5/apache2/libphp5.so(execute_ex+0x5b)[0x7f67de5633eb]
/usr/lib64/php5.5/apache2/libphp5.so(zend_execute_scripts+0x1c2)[0x7f67de4ddf62]
/usr/lib64/php5.5/apache2/libphp5.so(php_execute_script+0x214)[0x7f67de463ae4]
/usr/lib64/php5.5/apache2/libphp5.so(+0x5554e6)[0x7f67de5ae4e6]
[...]

The interesting thing here is that it is triggered in libmagic.so.1 although I thought that PHP5.5 includes its own libmagic (compiled in statically) and does not depend on libmagic.so.

The last call in libphp5.so is finfo_resource_destructor() where magic_close() is called (verified by backtrace offset).

Asm code is:
  1dd304:       e8 67 e5 f6 ff          callq  14b870 <magic_close@plt>

From there is goes here:
000000000014b870 <magic_close@plt>:
  14b870:       ff 25 92 d8 b1 00       jmpq   *0xb1d892(%rip)        # c69108

which goes to the GOT. readelf --relocs shows:
000000c69108  13a300000007 R_X86_64_JUMP_SLO 00000000001e8890 magic_close + 0

and indeed at location 1e8890 we find the implementation of magic_close in statically included libmagic. It calls file_ms_free also via the PLT but at the end we should still be in libphp5.so

However, the error is triggered in libmagic.so. Offset 0x5277 is the following code in file-5.15 apprentice.c, exact location marked with arrow:
protected void
file_ms_free(struct magic_set *ms)
{
        size_t i;
        if (ms == NULL)
                return;
        for (i = 0; i < MAGIC_SETS; i++)
                mlist_free(ms->mlist[i]);
        free(ms->o.pbuf);  <----
        free(ms->o.buf);
        free(ms->c.li);
        free(ms);
}

Why does this happen? Can this also be the reason for the error of the others which results in a seg fault?
Comment 18 Dennis Schridde 2014-02-25 19:21:58 UTC
I suspect that PHP's bundled libmagic plays some role in this (see upstream bugreport). Is it possible to unbundle libmagic from php?

See-Also: https://bugs.php.net/bug.php?id=66095
Comment 19 SpanKY gentoo-dev 2014-03-28 06:46:57 UTC
(In reply to Dennis Schridde from comment #18)

it sounds like a pretty good explanation
Comment 20 Fabio Rossi 2014-12-26 17:08:02 UTC
any news on this? sys-apps/file-5.11 is not anymore in portage...
Comment 21 Ole Markus With (RETIRED) gentoo-dev 2014-12-26 19:03:42 UTC
(In reply to Fabio Rossi from comment #20)
> any news on this? sys-apps/file-5.11 is not anymore in portage...

You can upgrade to PHP 5.6.x, which has bundled 5.17
Comment 22 Fabio Rossi 2014-12-28 11:44:43 UTC
(In reply to Ole Markus With from comment #21)
> (In reply to Fabio Rossi from comment #20)
> > any news on this? sys-apps/file-5.11 is not anymore in portage...
> 
> You can upgrade to PHP 5.6.x, which has bundled 5.17

I have just tried upgrading to PHP 5.6.4 but the problem is confirmed and not solved (both with system file-5.17 and file-5.21). Tested with mediawiki and php script at comment #7.

Moreover GLSA 201412-48 states the vulnerability of file < 5.21
Comment 23 Ole Markus With (RETIRED) gentoo-dev 2014-12-28 14:18:17 UTC
(In reply to Fabio Rossi from comment #22)
> (In reply to Ole Markus With from comment #21)
> > (In reply to Fabio Rossi from comment #20)
> > > any news on this? sys-apps/file-5.11 is not anymore in portage...
> > 
> > You can upgrade to PHP 5.6.x, which has bundled 5.17
> 
> I have just tried upgrading to PHP 5.6.4 but the problem is confirmed and
> not solved (both with system file-5.17 and file-5.21). Tested with mediawiki
> and php script at comment #7.
> 
> Moreover GLSA 201412-48 states the vulnerability of file < 5.21

Those who are affected by this should take the issue upstream (as some already has). The only thing we can do is add dependency blockers.

A workaround to this issue is to use fastcgi/FPM instead of mod_php.
Comment 24 Michael Orlitzky gentoo-dev 2015-11-07 02:28:28 UTC
*** Bug 467756 has been marked as a duplicate of this bug. ***
Comment 25 Michael Orlitzky gentoo-dev 2015-11-07 02:29:30 UTC
*** Bug 482950 has been marked as a duplicate of this bug. ***
Comment 26 Michael Orlitzky gentoo-dev 2015-11-07 02:29:44 UTC
*** Bug 508386 has been marked as a duplicate of this bug. ***
Comment 27 Michael Orlitzky gentoo-dev 2015-11-07 02:29:56 UTC
*** Bug 520268 has been marked as a duplicate of this bug. ***
Comment 28 Michael Orlitzky gentoo-dev 2015-11-07 02:37:36 UTC
Here's another upstream bug, this one still open:

https://bugs.php.net/bug.php?id=66095

It might be possible to unbundle their libmagic and use the system version, but that would require more autotools-fu than I currently possess.
Comment 29 Michael Orlitzky gentoo-dev 2016-07-12 22:27:03 UTC
I've learned PHP's autotools in the meantime, but their bundled libmagic contains some php-specific changes, so unbundling is not an option at the moment.

The upstream bug in the URL has a patch that sounds promising. Basically, the reporter took the libmagic code and inlined it into fileinfo. Then he overrode the bundled symbols so that they aren't public, and thus don't conflict with the symbols from our system libmagic.

The patch is outdated, but conceptually simple. It might make a good Sunday project for somebody =)

PHP upstream is pretty good at accepting pull requests on github.