Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 457784

Summary: /data/cached/buglist-*.html missing last-modified timestamps
Product: Gentoo Infrastructure Reporter: Daniel Santos <daniel.santos>
Component: BugzillaAssignee: Bugzilla Admins <bugzilla>
Severity: normal    
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Package list:
Runtime testing required: ---
Bug Depends on: 457782    
Bug Blocks:    

Description Daniel Santos 2013-02-16 03:02:56 UTC
Entries in /data/cached/buglist-*.html do not include a last-modified timestamp.  Therefore, if the bugs were actually accessible (see bug #457782), bots would have to read each bug report each time it crawled the site, resulting in much greater server load than necessary.

Perhaps a better solution would be to generate a compressed sitemap.xml to replace the current buglist-*.html files.  This has many advantages, one of which is that robots will know which bug reports are stale in its cache and which are still good and, thus, only download bugs reports which have been modified.  Additionally, the bandwidth, I/O & CPU load on the server will be reduced, as the compressed file will be smaller than an uncompressed html.
Comment 1 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2021-08-25 21:11:24 UTC
This is 8 years old, please re-open if its an actual problem for you.