Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bugzilla DB migration completed. Please report issues to Infra team via email via infra@gentoo.org or IRC

Bug 157543

Summary: torrents is dead
Product: Gentoo Infrastructure Reporter: Michael Cummings (RETIRED) <mcummings>
Component: OtherAssignee: Gentoo Infrastructure <infra-bugs>
Status: RESOLVED FIXED    
Severity: normal CC: djqfrey, gentoo-bugzilla2, m3q, paapaa125
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://torrents.gentoo.org
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: PHP script to fix database schema

Description Michael Cummings (RETIRED) gentoo-dev 2006-12-08 15:11:12 UTC
Looks like almost all of the torrents are stale - confirmed when trying to use the x86 one :/
Comment 1 Xavier Neys (RETIRED) gentoo-dev 2007-02-13 15:06:55 UTC
*** Bug 166665 has been marked as a duplicate of this bug. ***
Comment 2 paapaa125 2007-02-13 15:32:32 UTC
Just want to report that Gentoo torrents are still dead - 2 months after this bug has been opened. 

I think curtis119@gentoo.org is maintaining the torrents but his last post to Gentoo forums is from September 11th 2006 - that is over 5 months ago. 

So is he really maintaining the torrents anymore?
Comment 3 m3q 2007-02-14 07:35:27 UTC
Firstly hello in bugzilla! - my first reply here.

Coming to error - It doesn't happened to me. I've started downloading torrents (both live amd64 & live x86) at 2007-01-16, and i'm still uploading them. Maybe problem is not tracker related, but you have problems with too many sources for torrent files?

Ps. I'm seeding them on Win 2k3 server, but this should be no difference.
Comment 4 paapaa125 2007-02-14 11:25:59 UTC
Could you please try removing the torrent completely and downloading it again from the scratch so that we get comparable results? Try amd64 minimal 2006.1 for example:

http://torrents.gentoo.org/torrents/install-amd64-minimal-2006.1.torrent

I'm trying to download it right now and no established connections after 5 minutes of usage. At the same time Ubuntu ISOs work just fine. And I want to repeat that this is happening to other users besides me aswell.

PS. Where is curtis119?
Comment 5 m3q 2007-02-14 14:28:26 UTC
I've just added that torrent, but I need to leave now. I'll check tomorrow is sth happened.

Ps. I don't know curtis119 (but I read previous post in this bug).

Ps2 It is downloading (after 3 minutes), but there is only 3 seeds available, so maybe your problem is related to your connection? Do you have active (public) IP? Maybe there is no other active seeds available.
Comment 6 paapaa125 2007-02-14 17:43:51 UTC
Then it is not working properly. The tracker reports 20 seeds and so does torrents.gentoo.org. Yet you are connecting only to 3 of them. Could you please try this one too, it should have over 2300 seeds.

http://torrents.gentoo.org/torrents/livedvd-i686-installer-2006.1.torrent

After 5min I could connect only to 2 seeds and 0 peers. Yet I see that there are almost 2400 seeds in the swarm.

And this happens only with Gentoo torrents. But please try this torrent too and tell me how fast you can download and how many seeds you actually connect with?

Thanks for your help.
Comment 7 paapaa125 2007-02-14 21:55:11 UTC
Also, please try this:

http://torrents.gentoo.org/torrents/install-x86-minimal-2006.1.torrent 

I can't connect to ANY of the 60 seeders this torrent has. If you succeed, please try with Azureus also and post your logs from the beginning to the point you get a few established connections.
Comment 8 paapaa125 2007-02-16 19:53:15 UTC
Nevermind, I made a poll in Gentoo forums:

http://forums.gentoo.org/viewtopic-t-540055-highlight-.html

As you can see, 70% of users are definitely having problems with Gentoo torrents. (And posible the other 30% don't just know it because the tracker misleadingly reports itself as "OK"). These problems are also 100% evident by looking at this page:

http://linuxtracker.org/index.php?sort=8

Just compare the seeds/peers of Gentoo and other torrents. I'll file a new bug requesting that the Gentoo BitTorrent link should be removed from www.gentoo.org and a new maintainer found as Curtis is MIA. 

Maybe this is not an infrastructure issue as nobody seems to be interested at all?
Comment 9 Xavier Neys (RETIRED) gentoo-dev 2007-02-17 00:08:52 UTC
(In reply to comment #8)
> Nevermind, I made a poll in Gentoo forums:
> 
> http://forums.gentoo.org/viewtopic-t-540055-highlight-.html
> 
> As you can see, 70% of users are definitely having problems with Gentoo

70% of 10 is not that many :)

> Just compare the seeds/peers of Gentoo and other torrents. I'll file a new bug
> requesting that the Gentoo BitTorrent link should be removed from
> www.gentoo.org and a new maintainer found as Curtis is MIA.

Don't bother. You've aready filed one dupe.
I was waiting to see some feedback before commenting out the link on www.g.o's front page and on the get gentoo page. I'll do it soon(-ish)

> Maybe this is not an infrastructure issue as nobody seems to be interested at
> all?

Considering the number of complaints and of users on your poll, hardly anybody seems interested indeed.

IMO, it should be fixed or shut down until someone can take care of it.
Comment 10 m3q 2007-02-17 08:41:13 UTC
I've downloaded i686 livedvd at 2007-01-16 and I'am seeding it since then (There are some peers downloading from me almost all the time, now 3 of 19 peers total are downloading from me (10 in queue) ). Now I'm seeing 2359 usable seeds. What about your IP? Is it public? Maybe your ISP has network throttling enabled and is filtering p2p? I know that you told that there is no problem with other torrents, but sometimes "cannot start" issue is related to limits not in speed, but in session limit, and if there is too many seeds/peers that might cause problems. Try to change maximum number of connections in you torrent client. 
There might be also other reason - I think most people are seeding no only gentoo torrents.
Hope you find source of your problem.
Comment 11 paapaa125 2007-02-17 08:44:45 UTC
(In reply to comment #9)

> 70% of 10 is not that many :)

You can make that 72% now :-) (Or actually 80% because of 4 voices on another non-Gentoo forum...) 

> Don't bother. You've aready filed one dupe.

True. The reason I'm over-active is that this relly affects mostly _new_ users - users who most likely won't even bother to register to forums commenting "Gentoo torrents suck big time!". So thanks for reacting.

> I was waiting to see some feedback before commenting out the link on www.g.o's
> front page and on the get gentoo page. I'll do it soon(-ish)

That is great. Could you also remove curtis119's contact information and maybe put a message (on the torrent page) that Gentoo torrent tracker needs a new maintainer? Of course, mentioning this in GWN would make things proceed faster :)

> Considering the number of complaints and of users on your poll, hardly anybody
> seems interested indeed.

Yes, I guess all the seeders (or peers) don't even know that the torrents are not working (b/c tracker="OK") so that is why they are not complaining.
Comment 12 paapaa125 2007-02-17 08:54:55 UTC
(In reply to comment #10)
> I've downloaded i686 livedvd at 2007-01-16 and I'am seeding it since then
> (There are some peers downloading from me almost all the time, now 3 of 19
> peers total are downloading from me (10 in queue) ). Now I'm seeing 2359 usable
> seeds. 

You really should try it yourself: delete your torrents completely and start from the scratch. The tracker is broken and giving false IPs. That should be evident by now - this affects many users, not just me as you can see from the poll. And like I said, the tracker indeed reports 2400 seeds but they are not working! You can't connect almost any of them. Try it yourself. Also, turn of all kinds of "distributed tracker" functionality as it will bias the results.

> What about your IP? Is it public? Maybe your ISP has network throttling
> enabled and is filtering p2p? I know that you told that there is no problem
> with other torrents, but sometimes "cannot start" issue is related to limits
> not in speed, but in session limit, and if there is too many seeds/peers that
> might cause problems. Try to change maximum number of connections in you
> torrent client. 

This concerns _only_ the Gentoo tracker. There is no reason to assume that I have a connection problem as every single other torrent out there works 100% fine - no matter how many connections, inbound or outbound connections, how many seeds/peers etc. They all work. But despite that I have tried with all possible config setting with no luck. And no, the ISP is not blocking anything here in Finland. And despite that I'm not using port 6881 anymore but 50000. (Yes, I tried this with port 6881 also, without luck).
Comment 13 m3q 2007-02-17 09:01:37 UTC
I've deleted it, disabled dht and even peer exchange, restarted client and after 1 minute I'm downloading from scratch ~60KB/s. I've only leaved *.torrent file. So I think there is no problem for me. We will see if there will be problems with finishing this torrent. 
Comment 14 m3q 2007-02-17 09:04:50 UTC
BTW There is no need to block certain port, because there is no problem with detecting "what's inside" certain connections. The only work aroun this technology is to use protocol encryption, but not many people use it.
Comment 15 paapaa125 2007-02-17 09:21:14 UTC
AFAIK some ISPs also block/throttle bittorrent ports. But of course this is not the case as all my other torrents work perefectly. Interesting that you get correct IPs. Could it be that previously working IPs were cached and you got those instead of a new list from the tracker - Azureus does this. Could you try a new .torrent you have NOT been using in the past? ie. download it from the torrent.gentoo.org and try it. Pick something with at least 10 seeds.

My guess is that the tracker is not updating the IP list but sending old IPs to clients. Some of them work - some of then don't.
Comment 16 m3q 2007-02-17 09:58:29 UTC
As I stated before I'm seeding/downloading torrents on Win2K3 server, I'm not using Azureus, uTorrent web interface (beta). 

I've disabled all distributed features and started everything from beginning. After 15 minutes there was no progress in downloading (downl. doesn't started), so I think you were right. 

Now I enebled DHT and peer exchange and it started. So I think there is problem of some kind. Maybe there are no other people compleining about gentoo torrents because most people doesn't change settings and by default DHT/Peer Exchange is enabled, so maybe no one saw problem before (except you :) ).
Comment 17 Xavier Neys (RETIRED) gentoo-dev 2007-02-17 12:54:04 UTC
(In reply to comment #11)
> > I was waiting to see some feedback before commenting out the link on www.g.o's
> > front page and on the get gentoo page. I'll do it soon(-ish)
> 
> That is great. Could you also remove curtis119's contact information and maybe
> put a message (on the torrent page) that Gentoo torrent tracker needs a new
> maintainer? Of course, mentioning this in GWN would make things proceed faster

Sorry, I can't touch torrents.g.o

> Yes, I guess all the seeders (or peers) don't even know that the torrents are
> not working (b/c tracker="OK") so that is why they are not complaining.

It doesn't work for me, but the problem could me on my side. I never use bittorrent. http://torrent.fedoraproject.org/ does work, though.
Comment 18 paapaa125 2007-02-17 13:29:25 UTC
> I've disabled all distributed features and started everything from beginning.
> After 15 minutes there was no progress in downloading (downl. doesn't
> started), so I think you were right. 

Thanks for verifying this! You are correct that the problem might not be easily seen by those who have been seeding for a long time. But the issue rises when someone begins to download.
Comment 19 paapaa125 2007-02-17 13:30:37 UTC
> Sorry, I can't touch torrents.g.o

Do you know who can, (besides curtis119 who is not reachable)?

> It doesn't work for me, but the problem could me on my side. I never use
> bittorrent. http://torrent.fedoraproject.org/ does work, though.

This is 100% a tracker issue, not on client side.
Comment 20 Daniel Frey 2007-02-20 05:31:43 UTC
(In reply to comment #19)

I was searching around on the forums regarding a problem with my Azureus client and came across a thread regarding the Gentoo tracker being not functional.

I'm the one who wrote the tracker scripts in question. I had discovered an interesting bug after upgrading mysql on one of my boxes. The problem lies in how mysql treats certain datatypes, and how it applies charset information. 

What's happening is, the compact announces (which contain IP and port information) are stored in a CHAR field. At some point, the mysql ebuild's character collation set had changed, so when I updated one of my servers, I noticed (after much head scratching) that the compact announce information was getting corrupted. Mysql was applying character collation, which was screwing up the format the compact announces were encoded in.

In short, the tracker spews out invalid IP addresses. It's usually pretty obvious, as there will be a sequence of bits (in the case of a multibyte charset) that continually repeat, so it looks like there's a lot of similar ports and patterns in the IP addresses.

This is solely a Mysql data type problem.

I do know how to fix this, and it requires a script to be run on the server to truncate the peer tables and change the appropriate data types. The tracker will then be able to correctly report IP addresses as the bittorrent clients check in afterwards.

I'll hang around on IRC as danomac in #gentoo and #gentoo-mirrors if a dev wants to speak with me. I'm usually on IRC 24/7 so please be patient if you message me (I'm GMT-8).

Comment 21 paapaa125 2007-02-22 10:07:17 UTC
Danomac, thanks for finding the problem!

Another problem is that there is no Gentoo BitTorrent maintainer - Curtis119 has not been around for over 5 months and he's not responding to any messages sent to him. 

We need a new one or at least Danomac should be given a temporal access to fix this annoying issue.
Comment 22 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2007-02-22 10:11:58 UTC
I'll be working on this thanks to the pointers here and privately by danomac.
However it will only be AFTER I do the cvs.g.o migration (as announced on -core and -dev earliy this week). Torrents is less important the CVS ;-).
Comment 23 m3q 2007-02-22 10:35:00 UTC
Danomac, nice findings about MySQL DB! They are only confirming that before DB upgrade one should backup all data.

After enabling DHT/Peer exch/etc I've finally downloaded those torrents (which I stated ealier), but it took much longer that when I first downloaded them at 2007-01-16.
Comment 24 paapaa125 2007-03-11 09:17:30 UTC
If fixing this will take a long time it would at least be nice if someone put a reminder to torrents.gentoo.org saying that the tracker is not working - as that is still the case. Also the address of curtis119 should be removed there.
Comment 25 Daniel Frey 2007-03-18 03:27:40 UTC
Created attachment 113615 [details]
PHP script to fix database schema

This needs to be edited for database user/password/host and database name. It also needs to be run in a shell.

The user will require ALTER TABLE permission on the tracker database. It will go through all the tables and fix them and the statistics.
Comment 26 Daniel Frey 2007-03-18 21:06:28 UTC
I should also mention I'm willing to maintain the tracker. I'm almost always on IRC unless on holidays. I do not know how to apply for that position, if there is a way to apply.
Comment 27 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2007-04-02 02:51:58 UTC
danomac: thanks, I ran your script and reviewing it, and it worked perfectly, after just setting the auth details. I might come back to you with specific tracker questions while we figure out doing the distribution of the 2007.0 release.