Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 294791 - repoman should (optionally) check for 301 code for HOMEPAGE
Summary: repoman should (optionally) check for 301 code for HOMEPAGE
Status: RESOLVED DUPLICATE of bug 295194
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All All
: Lowest trivial (vote)
Assignee: Portage team
Depends on:
Blocks: 295335
  Show dependency tree
Reported: 2009-11-27 07:25 UTC by Martin Walch
Modified: 2010-03-11 23:48 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---

both versions of the small bash script from comment #3 (,811 bytes, text/plain)
2009-11-28 18:41 UTC, Martin Walch

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Walch 2009-11-27 07:25:27 UTC
The homepage link

is redirected (HTTP/1.1 301 Moved Permanently) to

Both are valid URIs (rfc2396), so I suggest adding the trailing slash in order
to avoid unnecessary redirection.
Comment 1 Allen Brooker (AllenJB) 2009-11-27 08:05:11 UTC
I'd suggest NOT making this change for 2 reasons:
1) It doesn't matter - the current URI works fine and invisibly redirects. This bug is just creating unnecessary load on the Gentoo developers.

2) The current URI allows upstream to invisibly move between a directory and a page without the need for the URI to be changed.
Comment 2 Martin Walch 2009-11-27 08:44:42 UTC
1) rfc2616 is clear about status code 301:
> The requested resource has been assigned a new permanent URI and any
> future references to this resource SHOULD use one of the returned URIs.

So it is primarily about conforming to HTTP/1.1 or not.

2) For this purpose, there are status codes 302 and 307.
Comment 3 Martin Walch 2009-11-28 18:35:13 UTC
I thought about this whole thing and came to the conclusion that it is indeed quite time consuming (especially my time, as during this year I have checked thousands of packages). Also it causes quite much noise with one bug report per 301 response.

So I have now written a (more or less) one liner in bash that checks each package for return code 301.

FORMAT="<category>/<name> <homepage>\n" eix | while read var; do name=`echo $var | cut -d\  -f1 -s`; value=`echo $var | cut -d\  -f2- -s`; for i in $value; do if [ ${i:0:4} = http ] && [ $(host $(echo $i | cut -d: -f2 | cut -d/ -f3)>/dev/null; echo $?) = 0 ] && [ ! "$(httping -g $i -c1 -G -m -o301 -e1)" = "1" -a $? = 0 ]; then echo $name: $i; fi; done; done

As it has become hard to read, I have tried unscrambling it a bit:

FORMAT="<category>/<name> <homepage>\n" eix | while read var
  do name=$(echo $var | cut -d\  -f1 -s)
     value=$(echo $var | cut -d\  -f2- -s)
     for i in $value
       do \
         host=$(echo $i | cut -d: -f2 | cut -d/ -f3)
         if [ ${i:0:4} = http ] \
         && [ $(host $host>/dev/null; echo $?) = 0 ] && \
         [ ! "$(httping $i -c1 -G -m -o301 -e1)" = "1" -a $? = 0 ]
           then echo $name: $i

It uses app-portage/eix, host from net-dns/bind-tools and >=net-analyzer/httping-1.30 (earlier versions of httping had an ugly bug).

Unfortunately, it is probably a bad idea to replace every link that is answered with "301 Moved Permanently" with the URI in the Location field, as some sites that have disappeared might now be redirected to placeholders or spam sites.

Maybe there is a maintainer who is willing to check all links in one pass and change (most of) the HOMEPAGE links accordingly? This will be much more efficient and will avoid much noise in the bug tracker.

Is this the right place for this suggestion?
Comment 4 Martin Walch 2009-11-28 18:41:13 UTC
Created attachment 211471 [details]
both versions of the small bash script from comment #3

Sorry, code in comment #3 broke when posting.
Comment 5 Diego Elio Pettenò (RETIRED) gentoo-dev 2009-11-29 17:41:42 UTC
Open a feature request for repoman to check for that (optionally) might be a better idea.
Comment 6 Sebastian Pipping gentoo-dev 2010-03-11 22:53:04 UTC
+  11 Mar 2010; Sebastian Pipping <> asunder-1.9.1.ebuild:
+  Fix homepage (bug #294791)

Now turning into repoman check as Diego suggested.
Comment 7 Sebastian Pipping gentoo-dev 2010-03-11 23:48:13 UTC

*** This bug has been marked as a duplicate of bug 295194 ***