Apache 2.0.58 has problems displaying indexes. In my case, I tested different indexes containing: a) 31 files and folders, b) 95 files and folders, c) 161 folders, d) 581 folders In case of a), apache was able to index it in a few seconds. In case of b) indexing took a little longer 5-10 seconds. In case of c) indexing took about 2-3 minutes. In case of d) indexing is very slow, even after 5 minutes the page was not displayed and apache used all the rest of available CPU. In all cases, no swap memory has been used and free memory has been available (512MB total). The setup is a gentoo headless system with the latest kernel (Linux pluto 2.6.16-gentoo-r11) and ebuilds. Workaround: downgrading to apache-2.0.55-r1 reemerge apache2 after masking version 2.0.58 ( echo "=net-www/apache-2.0.58">> /etc/portage/package.mask ) I downgraded to apache 2.0.55-r1 and did not change anything else of the setup. While back on apache 2.0.55-r1 even in case of d) indexing takes only a few seconds. Normal behaviour, even the high cpu usage has gone. Feel free to contact me and help me to debug the bug.
Besides the default configuration, I tested with the following modules: LoadModule cache_module modules/mod_cache.so LoadModule disk_cache_module modules/mod_disk_cache.so LoadModule mem_cache_module modules/mod_mem_cache.so LoadModule file_cache_module modules/mod_file_cache.so Withh or without the above modules, no difference. Apache 2.0.58 still very slow in indexing. Besides the default module configuration: I tested only with the modules enabled below: # The following Modules are not commonly loaded for Apache # LoadModule case_filter_module modules/mod_case_filter.so LoadModule case_filter_in_module modules/mod_case_filter_in.so LoadModule echo_module modules/mod_echo.so LoadModule mime_magic_module modules/mod_mime_magic.so LoadModule speling_module modules/mod_speling.so LoadModule unique_id_module modules/mod_unique_id.so LoadModule vhost_alias_module modules/mod_vhost_alias.so
I tried the modules cillybabe listed and didn't get the slowdown that she found. I loaded a page w/ 112 directories in less than 2 seconds but it took 110 seconds to index 1186 file in /usr/share/doc and was ~90% CPU. Without the caching stuff it took 2:15 seconds and 100% CPU. The machine is doing other things so these number aren't definative and they are odd in that while tweaking things it took between 2:00 - 2:15 (min:sec) to load the 1186 items regardless of the settings of the modules she listed above. (Including disabling all of them where my first run was 110 seconds). ???? I know nothing. :(
I tested both versions of apache on 581 items to index: apache 2.0.58 581 folders > 5 minutes apache 2.0.55-r1 581 folders ~ 12 seconds same setup, with all the modules enabled above.
I tested apache 2.0.58-r2 and it is slow as hell in indexing large indexes. The index-option is not usuable for large indexes.
*** Bug 146966 has been marked as a duplicate of this bug. ***
Additional Info: Use the following Index-Options: IndexOptions FancyIndexing IgnoreCase ScanHTMLTitles NameWidth=80 DescriptionWidth=32 FoldersFirst VersionSort
Update: This bug is reproduceable with apache 2.0.59, apache 2.0.58 AND apache 2.0.55-r2.
I was able to trace back: The issue is NOT reproduceable with apache 2.0.55-r1. This means the bug was introduced with apache 2.0.55-r2 and a look in the changelog explains probably the reason why autoindex it kind of broken in apache 2.0.55-r2 and later: *apache-2.0.55-r2 (31 Mar 2006) 31 Mar 2006; Michael Stewart <vericgar@gentoo.org> +apache-2.0.55-r2.ebuild: Replace the patch for AddDirectoryIndex with a rewritten one. This should fix the segfaults reported in bug 106808 Thx for the help of Maxi, pZYchO, dok and others.
Hello, looks like AddDiretoryIndex is the problem. After removing it (and expand DirectoryIndex) everything works fine. Greets pZYchO
Digging through some code..... mod_autoindex is running slower after the patch becuase it does a subrequest on every item in that directory. Every directory that apache goes through (from / down to wherever your documents are), apache does a merge routine, to merge the configurations from the parent directory to the child directory. Before the patch was added, this merge was simple for the DirectoryIndex directive: if DirectoryIndex is set in child, then use that DirectoryIndex, otherwise use DirectoryIndex from parent. Now that AddDirectoryIndex exists, this logic gets a bit more complicated, if the child doesn't set a DirectoryIndex, then the parent's DirectoryIndex, parent's AddDirectoryIndex and child's AddDirectoryIndex are merged together to create the DirectoryIndex for that directory. If the child's DirectoryIndex is set, then the child's DirectoryIndex and AddDirectoryIndex are merged together to create the DirectoryIndex for the directory. This is causing every subrequest to take a bit longer then it usually would, and when there are many directories, it adds up to a long time. Anyone whos knows C++ is free to take a look at the patch and offer improvements on how to make this faster. I'm not an experience C++ programmer, so I may have missed some obvious optimization. The current patch is at http://sources.gentoo.org/viewcvs.py/*checkout*/apache/trunk/dist/2.0/patches/07_all_mod_dir_incremental.patch Otherwise, if you need speed when generating indexes, you have a few options: 1. don't use AddDirectoryIndex (i.e. comment out any directives and put thier contents on the end of DirectoryIndex) 2. write a custom script to do it
Workaround: Replace the single DirectoryIndex line in httpd.conf with: <IfDefine !PHP5> DirectoryIndex index.html index.html.var index.htm index.htm.var </IfDefine> <IfDefine PHP5> DirectoryIndex index.html index.html.var index.htm index.htm.var index.php index.phtml </IfDefine> and comment the line AddDirectoryIndex in modules.d/70_mod_php5.conf.
Additional info to my last comment: http://bugs.gentoo.org/show_bug.cgi?id=139451#c11 The "DirectoryIndex $args" has to be a single line.
After commenting the AddDirectoryIndex line in modules.d/70_mod_php5.conf and restarting Apache, I'm noticing indexes loading in 1-2 seconds where they took about 30-45 seconds previously. Apache's CPU usage is greatly reduced as well. I'm using Apache 2.0.58.
Seconding results found above re. AddDirectoryIndex patch. Scans all subdirs for indexes. The detail I'm adding is that it doesn't even seem to matter if mod_autoindex is added or if autoindexes are enabled, it still does this scan. takes _forever_ in dirs with many subdirs. the problem seems to be in mod_dir (as this seems to be where the horrible AddDirectoryIndex patch lives). Having the AddDirectoryIndex command anywhere in apache config will cause this subdir scan. There seems to be absolutely no need for this patch- it's a horrendous performance hit. Can't you guys do things with IfDefines like all other distros? Remove this patch! bleaaah!
(In reply to comment #14) > Seconding results found above re. AddDirectoryIndex patch. Scans all subdirs > for indexes. The detail I'm adding is that it doesn't even seem to matter if > mod_autoindex is added or if autoindexes are enabled, it still does this scan. > takes _forever_ in dirs with many subdirs. You must be mistaken, the code doesn't work that way. The added patch does no directory scanning at all. Some module you are using most be doing that if mod_autoindex isn't. The patch only does a few API calls that merge arrays. > Can't you guys do > things with IfDefines like all other distros? We are not other distros. We like to give you the choice as to which add-on modules you want. Other distros package these add-on modules with apache, or expect you to edit your configuration after installing if you want a new DirectoryIndex. We like to have things Just Work after they are built and installed. IfDefines require every package to modify a single file. In portage, this file would now be "owned" by every module as well as apache. There are not allowed in Gentoo as they cause problems when updating and unmerging. This is the solution we are using to work around the need for modules to add thier own DirectoryIndex and at this point in time it will not be changing. We came to this solution as the best option back at the beginning of last year as we were redesigning the apache configuration and creating eclasses to help maintain modules. Improvements to the AddDirectoryIndex patch are welcome. Demanding that we remove that patch are not. A patch that fixes mod_autoindex so that it doesn't do a subrequest on every directory would also be nice, though that may require re-writing much of it.
*** Bug 150947 has been marked as a duplicate of this bug. ***
*** Bug 151410 has been marked as a duplicate of this bug. ***
I request to remove the patch, since it breaks more than it fixes.
> We are not other distros. We like to give you the choice as to which add-on > modules you want. Other distros package these add-on modules with apache, or > expect you to edit your configuration after installing if you want a new > DirectoryIndex. We like to have things Just Work after they are built and > installed. Noble sentiment, but in my case this patch caused things to Just Break after they were built and installed. My production server was sidelined for nearly two months because of bug #151410. > Improvements to the AddDirectoryIndex patch are welcome. Demanding that we > remove that patch are not. Is it possible to prevent this directive from being used in .htaccess files? Or at least, could a big warning be added to the ebuild? Once again, please see bug #151410 for details. I believe that problem is unacceptable due to the potential for a DoS attack, which I'd be happy to demonstrate to any willing party with a Gentoo server running Apache. :-)
I tried apache 2.2.3 and guess what: autoindex now skips those dirs, which means they aren't listed at all
Do not reopen this bug. The workarounds are listed, and work. The patch does absolutely *nothing* if you don't have an AddDirectoryIndex directive anywhere in your config. If you didn't have the patch, then that directive wouldn't exist, and so if you want it ripped out, I assume you know how to make things work on your system without that directive, so just quit using that directive. The patch is NOT going away.
*** Bug 169989 has been marked as a duplicate of this bug. ***
I think I've found a simple solution that will please everyone: limit the use of the AddDirectoryIndex directive to the global server config only. This seems sufficient for the modular purpose the for which the directive is intended. All of the associated security and performance bugs I've seen seem to be related to having the directive in a per-directory context instead. And an AddDirectoryIndex directive in a VirtualHost context is still applied server-wide anyway, so there's no need for it to be there either. I'd be happy to discuss any questions, concerns, or criticisms. Feel free to /msg depquid on freenode if you want to have a live conversation. I'll be attaching a modified patch that implements my recommendation.
Created attachment 115204 [details, diff] Fixed AddDirectoryIndex patch This is a tweaked version of 07_all_mod_dir_incremental.patch from the Gentoo Apache patchset. It is intended to replace the current patch.
Reopen.
(In reply to comment #23) > I think I've found a simple solution that will please everyone: limit the use > of the AddDirectoryIndex directive to the global server config only. This > seems sufficient for the modular purpose the for which the directive is > intended. All of the associated security and performance bugs I've seen seem > to be related to having the directive in a per-directory context instead. And > an AddDirectoryIndex directive in a VirtualHost context is still applied > server-wide anyway, so there's no need for it to be there either. > > I'd be happy to discuss any questions, concerns, or criticisms. Feel free to > /msg depquid on freenode if you want to have a live conversation. > > I'll be attaching a modified patch that implements my recommendation. Yup, and that's what I just added to our svn repository.
(In reply to comment #27) > Yup, and that's what I just added to our svn repository. And this went now into the tree with apache-2.2.4-r5.
Why is this marked as NEEDINFO?
*** Bug 183280 has been marked as a duplicate of this bug. ***
Reopen to resolve properly.
FIXED in >=apache-2.2.4-r5
Created attachment 137458 [details, diff] rewritten patch Several months later... Nobody has noticed that the bug actually isn't fixed. The the problem is in the following lines in fixup_dir(): if (d->index_names_incr) { if (!d->index_names) { d->index_names = d->index_names_incr; } else { apr_array_cat(d->index_names, d->index_names_incr); } } Changes made to d->index_names are persistent and on high-load servers the list grows very fast. And when index_names has several thousands elements its processing takes much time. I've attached rewritten 02_all_gentoo_mod_dir_incremental.patch. I hope it doesn't change the semantics of AddDirectoryEntry. It's tested it on my server and seems to work.
I've read history of the patch: http://sources.gentoo.org/viewcvs.py/apache/trunk/dist/2.2/patches/02_all_gentoo_mod_dir_incremental.patch?rev=152&r1=128&view=log The bug was introduced in revision 69, and in revision 73 was backported to apache 2.0. I think that reverting to revision 68 can be the way to go.
(In reply to comment #33) > Several months later... Nobody has noticed that the bug actually isn't fixed. > > The the problem is in the following lines in fixup_dir(): Are you experiencing problems, or merely trying to propose a more ideal solution? IIRC, the growth of the d->index_names array only occurred if AddDirectoryIndex was processed in non-global context (which it should never have been allowed to in the first place, as it has a server-wide effect). Therefore, the problem seems to be fixed (both practically and ideologically) now that the directive is not allowed in any other context. (My server's highly improved performance suggests the same.) Therefore, I say don't fix what is already working acceptably. On the other hand, if you are actually experiencing performance issues related to this, what version of Apache are you using?
(In reply to comment #35) > IIRC, the growth of the d->index_names array only occurred if > AddDirectoryIndex was processed in non-global context (which it should never > have been allowed to in the first place, as it has a server-wide effect). It occurred with default gentoo configuration files (AddDirectoryIndex was in modules.d/70_mod_suphp.conf). > Therefore, the problem seems to be fixed (both practically and ideologically) > now that the directive is not allowed in any other context. (My server's > highly improved performance suggests the same.) Therefore, I say don't fix > what is already working acceptably. But I think that users should be able to use AddDirectoryIndex in .htaccess (as they can use DirectoryIndex). > On the other hand, if you are actually experiencing performance issues related > to this, what version of Apache are you using? I was experiencing it on 2.2.6. Commenting lines in modules.d/70_mod_suphp.conf fixed the problem.
(In reply to comment #36) > But I think that users should be able to use AddDirectoryIndex in .htaccess (as > they can use DirectoryIndex). Well, as long as the directory context is handled properly, and thoroughly tested for regressions, I'm happy with that. > I was experiencing it on 2.2.6. Commenting lines in modules.d/70_mod_suphp.conf > fixed the problem. Hmm, could this be in anyway related to mod_suphp? I wouldn't think so, but I'm not using it, and, as I said, I've had no problems since AddDirectoryIndex was restricted to global context.
I took apache 2.2.6 with gentoo patches and added the following lines to fixup_dir(): if (d->index_names) ap_log_rerror(__FILE__, __LINE__, APLOG_DEBUG, 0, r, "index_names count: %d", d->index_names->nelts); else ap_log_rerror(__FILE__, __LINE__, APLOG_DEBUG, 0, r, "empty index_names"); I have AddDirectoryIndex only global: $ grep DirectoryIndex /etc/apache2 /var/www/localhost -R /etc/apache2/modules.d/00_autoindex.conf:# a file matching those listed in the DirectoryIndex directive. /etc/apache2/modules.d/00_default_settings.conf:# DirectoryIndex: sets the file that Apache will serve if a directory /etc/apache2/modules.d/00_default_settings.conf:# To add files to that list use AddDirectoryIndex in a custom config /etc/apache2/modules.d/00_default_settings.conf: DirectoryIndex index.htm /etc/apache2/modules.d/00_default_settings.conf: AddDirectoryIndex index.html logs: [Sat Dec 01 21:51:59 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 2 [Sat Dec 01 21:52:01 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 2 [Sat Dec 01 21:52:01 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 3 [Sat Dec 01 21:52:02 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 3 [Sat Dec 01 21:52:13 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 4 [Sat Dec 01 21:52:14 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 4 [Sat Dec 01 21:52:16 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 5 [Sat Dec 01 21:52:17 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 5 [Sat Dec 01 21:52:27 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 6 [Sat Dec 01 21:52:29 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 6 [Sat Dec 01 21:52:30 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 7 [Sat Dec 01 21:52:31 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 7 [Sat Dec 01 21:52:32 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 8 [Sat Dec 01 21:52:33 2007] [debug] mod_dir.c(205): [client 127.0.0.1] index_names count: 8 (I think it increases every second time, because apache has two processes) So you can see that when AddDirectoryIndex is in global scope the bug still occurres.
(In reply to comment #38) > I took apache 2.2.6 with gentoo patches and added the following lines to > fixup_dir(): And this test system doesn't have anything like suphp or even php.
And about limiting AddDirectoryIndex to global scope: Two DirectoryIndex entries can coexist in one scope, eg. when you have: DirectoryIndex index.html DirectoryIndex index.php then either file can be index, not only index.php. Thus AddDirectoryIndex is really useful only in sub-scopes (DirectoryIndex in sub-scope causes inherited values to be discarded). And because DirectoryIndex can be used in sub-scopes, I can't see ideological reasons not to allow AddDirectoryIndex.
Bump... Joshua, did you manage to reproduce my results ? Or maybe you can confirm that the bug isn't present on your system ?
(In reply to comment #41) > Joshua, did you manage to reproduce my results ? Or maybe you can confirm that > the bug isn't present on your system ? > Sorry, I've been pre-occupied with other projects (gotta keep the paying clients), so I haven't been able to look at this again yet. Unfortunately, I don't know how soon I'll be able to thoroughly test this. In the mean time, could you double-check your assertion about two DirectoryIndex directives in the same scope? I was under the impression that the latter declaration would overwrite the former. After all, this is the reason that (at least according to comment #15) the AddDirectoryIndex directive was created: so that conf files for mods could add filenames to the directory index list without having to change httpd.conf. Therefore, the ability to use AddDirectoryIndex in non-global context is apparently not needed for Gentoo purposes, so the Gentoo devs may not want to try to extend it's functionality in that way. After all, we're already having enough trouble getting the bugs out of it as is.
I have compiled apache without the 02_all_gentoo_mod_dir_incremental.patch patch and it works as I said: mareklin /etc/apache2 # grep DirectoryIndex . -R modules.d/00_default_settings.conf: DirectoryIndex index.htm modules.d/00_default_settings.conf: DirectoryIndex index.html modules.d/70_mod_php5.conf: DirectoryIndex index.php index.phtml And all these index files work.
(In reply to comment #43) > And all these index files work. I have confirmed this ... DirectoryIndex in the same scope will be merged, that's all we need, i will remove the patch from 2.2.8-r1 as soon as all packages using it have switched over (php and mod_suphp) php herd: please replace AddDirectoryIndex with DirectoryIndex (no need for new revision, replace it in all config files)
*** Bug 207946 has been marked as a duplicate of this bug. ***
(In reply to comment #44) > I have confirmed this ... DirectoryIndex in the same scope will be merged, > that's all we need, i will remove the patch from 2.2.8-r1 as soon as all > packages using it have switched over (php and mod_suphp) > > php herd: please replace AddDirectoryIndex with DirectoryIndex (no need for new > revision, replace it in all config files) Done for dev-lang/php. Seems like you already did mod_suphp. Also, as pointed out on IRC already, docs[1] need an update regarding AddDirectoryIndex. [1] http://www.gentoo.org/doc/en/apache-developer.xml
(In reply to comment #46) > Also, as pointed out on IRC already, docs[1] need an update regarding > AddDirectoryIndex. > > [1] http://www.gentoo.org/doc/en/apache-developer.xml yep, this is already tracked in bug 200617 patch removed in 2.2.8-r1, closing ...
I just emerged 2.2.8-r1, fixed my PHP.conf and can confirm that the CPU-hogging of browsing indexed directories is gone. THANK YOU Gentoo devs!
*** Bug 209033 has been marked as a duplicate of this bug. ***
Yes, thank you for finally fixing this, instead of stonewalling :)
