It would be nice for bgo to have a sitemap. A robots.txt exists already: https://bugs.gentoo.org/robots.txt planet.gentoo.org, however, is missing both. The wiki is fine. I didn't check other sites. Reproducible: Always