I have installed dev-lang/php-5.2.6-r7 with USE="apache2 bcmath berkdb bzip2 calendar cli crypt ctype curl curlwrappers exif fastbuild filter flatfile ftp gd gdbm gmp hash iconv inifile iodbc json ldap ldap-sasl mhash mysql ncurses nls odbc pcntl pcre pdo posix postgres readline reflection session snmp soap sockets spell spl sqlite ssl suhosin sysvipc threads tidy tokenizer truetype unicode xml xmlreader xmlrpc xmlwriter xpm xsl zip-external zlib" , dev-php5/suhosin-0.9.27, and www-servers/apache-2.2.9-r1 with APACHE2_MPMS="worker" APACHE2_MODULES="actions alias asis auth_basic auth_digest authn_anon authn_default authn_file authz_default authz_groupfile authz_host authz_user autoindex cache deflate dir disk_cache env expires file_cache filter headers imagemap include log_config logio mem_cache mime mime_magic negotiation rewrite setenvif userdir vhost_alias" USE="ldap sni ssl threads" . My test program looks as follows: -----BEGIN pcretest.php----- <?php echo preg_replace('/\p{Lu}/u', 'x', "TEST\n"); ?> -----END pcretest.php----- Executing php pcretest.php on the command line produces the output xxxx , as it should. But running the script on a local webserver via the Apache module results in an error with the following message in the server error log (removed irrelevant parts): [error] PHP Warning: preg_replace() [<a href='function.preg-replace'>function.preg-replace</a>]: Compilation failed: unknown property name after \\P or \\p at offset 5 in pcretest.php on line 1 My php.ini files for apache2-php5 and cli-php5 only differ in logging settings and ‘register_argc_argv’, which is ‘Off’ for apache2-php5, but ‘On’ for cli-php5. /etc/php/*-php5/ext* are equal for apache module and CLI, and include only suhosin.ini and zip.ini. I expect both ways of executing the script (via apache module, and via CLI) to behave equally.
I see "unicode related" in the title -- could you go into a little more detail about this?
Might be related to the fact that Apache uses PCRE internally as well, but in theory PHP should use its own copy (and iirc they are doing some tricks to avoid symbol collisions). Will look into it in the next days. Just for reference: FastCGI works like CLI as well.
(In reply to comment #1) > I see "unicode related" in the title -- could you go into a little more detail > about this? The bug does not appear when using PCRE functionality that is not UTF-8 related, as in the following example: -----BEGIN pcretest2.php----- <?php echo preg_replace('/[A-Z]/', 'x', "TEST\n"); ?> -----END pcretest2.php----- This example works fine from the command line and via the Apache module, and prints "xxxx\n" both times. (In reply to comment #2) > Might be related to the fact that Apache uses PCRE internally as well, Probably.
I have a similar problem of unicode chars handling by php on a phpBB board since upgrade to php-5.2.6-r7. Same days libpcre was upgraded, it seems to be the source of problem, I downgraded from dev-libs/libpcre-7.8 to dev-libs/libpcre-7.7-r1, rebuild apache & php, and now it works again. Your script pcretest.php works also after that downgrade.
First, sorry for the delay. Can you please try =dev-lang/php-5.2.6-r8 from the php-testing overlay available via layman? This package should use the system version of pcre (you can verify by using ldd `which php` | grep pcre; it should list some libpcre*.so). Then, please test whether PCRE works properly in mod_php at all (i.e. by using pcre_match() functions and friends from within PHP), check whether your bug (unicode-related) is still present and it would also be cool if you could try to use PCRE from within Apache (e.g. by using mod_rewrite). I cannot promise that we are going to include this fix in the main tree, I still fear it might break things with Apache. When I joined Gentoo for maintaining PHP, I also aimed for getting rid of the bundled PCRE library, but I was told that there were severe problems w/ using the system copy of PCRE in case of Apache/mod_php.
I've tried =dev-lang/php-5.2.6-r8 on amd64 without success. I have the following components/versions/USE flags: emerge -pv apache php libpcre These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] dev-libs/libpcre-7.8 USE="bzip2 cxx unicode zlib -doc" 0 kB [ebuild R ] www-servers/apache-2.2.9-r1 USE="ldap ssl threads -debug -doc (-selinux) -sni -static -suexec" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias -asis -auth_digest -authn_dbd -cern_meta -charset_lite -dbd-dumpio -ident -imagemap -log_forensic -proxy -proxy_ajp -proxy_balancer -proxy_connect -proxy_ftp-proxy_http -substitute -version" APACHE2_MPMS="worker -event -itk -peruser -prefork" 0 kB [ebuild R ] dev-lang/php-5.2.6-r8 USE="apache2 berkdb bzip2 cgi cli crypt ctype gdbm iconv inifile ipv6 ldap mysql mysqli ncurses nls pcre readline reflection session spl sqlite ssl threads unicode zlib (-adabas) -bcmath (-birdstep) -calendar -cdb -cjk -concurrentmodphp -curl -curlwrappers -db2 -dbase (-dbmaker) -debug -discard-path -doc (-empress) (-empress-bcs) (-esoob) -exif -fastbuild (-fdftk) -filter (-firebird) -flatfile -force-cgi-redirect (-frontbase) -ftp -gd -gd-external -gmp -hash -imap -interbase -iodbc (-java-external) -json -kerberos -kolab -ldap-sasl -libedit -mcve -mhash -msql -mssql -oci8 -oci8-instant-client -odbc -pcntl -pdo -pic -posix -postgres -qdbm -recode-sapdb -sharedext -sharedmem -simplexml -snmp -soap -sockets (-solid) -spell -suhosin (-sybase) (-sybase-ct) -sysvipc -tidy -tokenizer -truetype -wddx -xml -xmlreader -xmlrpc -xmlwriter -xpm -xsl -yaz -zip -zip-external" 0 kB [1] Total: 3 packages (3 reinstalls), Size of downloads: 0 kB Portage tree and overlays: [0] /usr/portage [1] /usr/portage/local/layman/php-testing After upgrading to the -r8 I restarted Apache → no change So I recompiled Apache + restarted it → no change I've checked PHP to link against the correct PCRE libs: ldd `which php` | grep pcre libpcre.so.0 => /usr/lib/libpcre.so.0 (0x00007fe4e1fb2000) But does this really matter, whether /usr/bin/php is linked against this? As it worked on CLI before anyways - what doesn't work is mod_php. Anything I forgot? Regards, Elias P.
(In reply to comment #6) > I've tried =dev-lang/php-5.2.6-r8 on amd64 without success. Too bad. :/ > After upgrading to the -r8 I restarted Apache → no change > So I recompiled Apache + restarted it → no change > I've checked PHP to link against the correct PCRE libs: > > ldd `which php` | grep pcre > libpcre.so.0 => /usr/lib/libpcre.so.0 (0x00007fe4e1fb2000) > > But does this really matter, whether /usr/bin/php is linked against this? As it > worked on CLI before anyways - what doesn't work is mod_php. Heh, you are right. I was just assuming that if the CLI version of PHP properly links to PCRE, then mod_php should as well. This would be more exact: ldd /usr/lib/apache2/modules/libphp5.so | grep pcre But I highly doubt that the output will differ. So... I'm out of ideas at the moment, I really thought it was the fact that PHP used the internal version of PCRE...
> Heh, you are right. I was just assuming that if the CLI version of PHP > properly links to PCRE, then mod_php should as well. > This would be more exact: > ldd /usr/lib/apache2/modules/libphp5.so | grep pcre > > But I highly doubt that the output will differ. So... I'm out of ideas at > the moment, I really thought it was the fact that PHP used the internal > version of PCRE... It does actually make a difference - libphp5.so isn't linked against PCRE at all: ldd /usr/lib/apache2/modules/libphp5.so linux-vdso.so.1 => (0x00007fff9bffe000) libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007f7d93586000) libsqlite.so.0 => /usr/lib/libsqlite.so.0 (0x00007f7d9332a000) libhistory.so.5 => /lib/libhistory.so.5 (0x00007f7d93121000) libreadline.so.5 => /lib/libreadline.so.5 (0x00007f7d92ee0000) libncurses.so.5 => /lib/libncurses.so.5 (0x00007f7d92c82000) libpanel.so.5 => /usr/lib/libpanel.so.5 (0x00007f7d92a7d000) libmcrypt.so.4 => /usr/lib/libmcrypt.so.4 (0x00007f7d9284d000) libltdl.so.3 => /usr/lib/libltdl.so.3 (0x00007f7d92645000) libldap-2.4.so.2 => /usr/lib/libldap-2.4.so.2 (0x00007f7d92403000) liblber-2.4.so.2 => /usr/lib/liblber-2.4.so.2 (0x00007f7d921f4000) libdb-4.5.so => /usr/lib/libdb-4.5.so (0x00007f7d91ed5000) libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0x00007f7d91cce000) libbz2.so.1 => /lib/libbz2.so.1 (0x00007f7d91abe000) libz.so.1 => /lib/libz.so.1 (0x00007f7d918a9000) libresolv.so.2 => /lib/libresolv.so.2 (0x00007f7d91694000) libm.so.6 => /lib/libm.so.6 (0x00007f7d91413000) libdl.so.2 => /lib/libdl.so.2 (0x00007f7d9120f000) libnsl.so.1 => /lib/libnsl.so.1 (0x00007f7d90ff7000) libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0x00007f7d90da9000) libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0x00007f7d90a2c000) libmysqlclient_r.so.15 => /usr/lib64/mysql/libmysqlclient_r.so.15 (0x00007f7d906b5000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007f7d9049a000) libc.so.6 => /lib/libc.so.6 (0x00007f7d9015b000) libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0x00007f7d8ff40000) /lib64/ld-linux-x86-64.so.2 (0x00007f7d93f4c000)
I watched the output of ./configure now and found this interesting line: checking pcre install prefix... no Doesn't look really good to me as '--with-pcre-dir=/usr' is explicitly called in the ebuild (line 328).
> I watched the output of ./configure now and found this interesting line: > > checking pcre install prefix... no > > Doesn't look really good to me as '--with-pcre-dir=/usr' is explicitly called > in the ebuild (line 328). Downgraded libpcre to 7.7-r1 now - the line "checking pcre install prefix... no" stays the same, but it works. I'll keep libpcre-7.8 masked for now on my production servers - don't have the slightest clue why it doesn't work with 7.8.
(In reply to comment #8) > It does actually make a difference - libphp5.so isn't linked against PCRE at > all: D'oh, I fail at ebuild logic. It worked properly in case of USE=fastbuild, but it worked only for the first go of the three-part build process when building USE=-fastbuild, and as such mod_php and php-cgi did not get the proper configure arguments, while php-cli did. May I ask you to re-sync the php-testing overlay (at least rev 3542) and try again? It should at least link properly to libpcre now, and this will hopefully solve the problem as well... (In reply to comment #10) > Downgraded libpcre to 7.7-r1 now - the line "checking pcre install prefix... > no" stays the same, but it works. > > I'll keep libpcre-7.8 masked for now on my production servers - don't have the > slightest clue why it doesn't work with 7.8. Probably it's simply the fact that the two copies of libpcre don't run seperated, but they share some parts (address space? whatever) and this does not work properly if versions differ.
Updated the overlay and re-emerged PHP and unmasked libpcre-7.8 before. It works now as expected - this fix should go into the official tree ASAP. Thanks a lot for fixing this! Regards, Elias P.
reopening
php-5.2.8, which I just committed, has this change and should work as expected as such (in fact, it is bundling a newer version of pcre as well, so vanilla php-5.2.8 should work as well; I decided to go with the system copy anyway, firstly because of security reasons, secondly as this issue is likely to come up after some time again otherwise). Leaving open until this hits stable.
Maybe the php-5.2.8 ebuild should check or mention that the dev-libs/libpcre ebuild is or has to be compiled with the +unicode use-flag. If it is not the behavior of the pcre-functions in php changes. They do not support unicode strings anymore and throws an error (or warning).
(In reply to comment #15) > Maybe the php-5.2.8 ebuild should check or mention that the dev-libs/libpcre > ebuild is or has to be compiled with the +unicode use-flag. If it is not the > behavior of the pcre-functions in php changes. They do not support unicode > strings anymore and throws an error (or warning). Thanks for the suggestion, of course this should be checked. Added in php-5.2.8-r1. We are targetting this version for stabilization per bug 249875.
