Examining Bittorrent

ToyKeeper and other wrote in with this: "The Register published a detailed analysis of BitTorrent traffic and user habits today, focusing on four aspects: availability, integrity, download speeds, and ability to withstand flash crowds. BitTorrent carries 53% of all P2P traffic (or ~35% of all 'net traffic), and this paper helps explain why. Also included are data about torrent lifetime, network poisoning, response during downtime or attacks, and lots of pretty charts. A few performance problems are revealed, which will hopefully be addressed in future p2p systems." The original paper (pdf) is available.

35%

35% of all 'net traffic

That's enormous!

I guess this proves that BitTorrent is the perfect vector for the largest files, be they Linux distros or movies (public-domain movies, of course). As the article says, BitTorrent is not perfect and will probably be surpassed in the future. But the fact that 35% of all 'net traffic is being carried by one program is simple awesome, and a great credit to BitTorrent's creators.

Also, with such a volume of traffic, surely it would be impossible for an **AA sniffer to track it all? Or at least, your chances of being caught and sued are pathetic small.

All of this is great news for BitTorrent. Long may it continue!

Re:35%

I have the complete statistics:

35% = BitTorrent
40% = Spam
15% = Slashdottings
10% = Porn Browsing

Re:It's you who are to blame

Poster wrote:

Yeah, right. Only an insignificant fraction of torrent traffic is legit. You really think that the scheme will remain legal because of these few users?

That's all it takes - see the Betamax decision. However, you might also want to take a look at the stats (below) for why people get high-speed internet.

... again ...

BitTorrent and the likes will be shut down in 2005. Mark my words. Since most of the traffic I see (I am an admin) is illegal, I'll shed no tears. It's you who violate copyrights who are to blame for the crackdown and the eventual clampdown on the internet - not RIAA, MPAA or any other corporation.

How are we supposed to "mark your words" when you post as an AC? Also, you seem to think that downloading music is illegal everywhere, when it's not. Not everyone lives in the US of BushCo. Also, the servers holding the torrent files are not breaking any laws.

From the article:

A few performance problems are revealed, which will hopefully be addressed in future p2p systems.

Well, since, according to El Reg http://www.theregister.co.uk/2004/12/08/brit_net_f ilth/ [theregister.co.uk] One in four Brits on net for Porn, there's a demand for "clittorrent".

The stats:

According to a survey conducted by British ISP Homecall, 23 per cent of Britons are getting broadband for the porn, and it's by far the most important factor in getting wired. 12 per cent cited access to music videos, 8 per cent access to movie trailers, and a gratifying 9 per cent for radio, which is undergoing a renaissance in the UK. Sometimes new media can be the best thing to happen to old media.

All the above are LEGAL.

Re:It's you who are to blame

• Yeah, right. Only an insignificant fraction of torrent traffic is legit.

Yup. All it takes is any.

The legal principal is this: if the {object, device, chemical, drug} has a purpose for which it is legal, then the thing should be legal.

The exceptions to this (guns, marijuana, and other things we've allowed to be banned) prove the rule. The pressure to legalize or ban something evinces arguments about its legitimate uses, and it's these arguments that are persuasive. Saying "We'll do it anyway" is unproductive.

In this case, since downloading Free software is so much more efficient with P2P, it's inappropriate to ban it even if that software is only a small percentage of the service's traffic.

Legal Torrents

aside from movie and music piracy there are legal uses for bittorrent p2p too, like when Linux distros are released the demand is much greater than the file servers can handle and thats where bittorrent plays an important role, i prefer to get my Linux ISOs via bittorrent because it helps others get their ISOs too, for example FedoraCore-3 was released and it came on 4 CDs plus a fifth rescue CD making for a HUGE download, and also offered resume so if you have to log off or have a network problem you don't lose all that data and have to start your download over...

Sites that list legal torrents...

There are also sites that list legal torrents, try File Soup [filesoup.com] or Legal Torrents [legaltorrents.com] for example. These are just two that I remember offhand, I'm sure there are many others as well. Remember, BitTorrent, like any other P2P application, has plenty of legitimate uses. Don't get sucked in by the *AA propaganda machine (not directed towards the parent, just saying that in general).

Re:Legal Torrents

There are other big legal downloads available via BT as well. For example, I set my web server up as a seed for the Project Gutenberg DVD-ROM and CD-ROM images [gutenberg.org], about as legal a set of files as you can get. So far I have served up over half a terabyte of those two images to people. I also seed a couple freeware games and some Creative Commons-licensed video to the tune of a couple hundred gigabytes of traffic, not a single byte of illegal or unauthorized content there.

Hosting the 3.85GB Gutenberg DVD image would be a bit costly for the Gutenberg folks. Without BT or something like it, it would be much less convenient for volunteers like me to help them out by spreading the load around.

Such an unused potential

There is such a powerfull distribution mechanism in P2P network, if only the studios/majors/etc would understand it and use it instead of fighting it, their market could explode, while having no distribution costs, their custermers would provide the distribution mechanisms.

But I'm afraid they are not going to get it in time.

My dream about a P2P PVR:
http://www.oberle.org/blog/2004/08/02/a-p2p-video- recorder-box/ [oberle.org]

Bartering?

the downloaders of a file barter for chunks of it by uploading and downloading them in a tit-for-tat-like manner to prevent parasitic behavior. Each peer is responsible for maximizing its own download rate by contacting suitable peers, and peers with high upload rates will with high probability also be able to download with high speeds.

Does this actually work? I find that when there are limited seeds, those first in line essentially transmit as fast as they recieve, and increasing upload doesn't really affect total speed much. When there are lots of seeders there's plenty of bandwidth to go around so it's always fast. Does anyone notice that restricting upload significantly affects download speed?

BT is great, but:

There are a few things that i would count as it's downsides. For instance, once the object that is being distributed been downloaded by the masses - you won't get a decent speed downloading it. So unless you grabbed it while it was "hot" - you will have to deal with much lower speeds. Also i often find that i upload almost as much as i download, not being greedy or anything, but here in New Zealand broadband is still capped either on speed or on traffic. And quotas are pretty stingy, counting both uploads and downloads... but that is more isp/country specific i guess:)

They missing the most important quality

...ability to withstand flash crowds

How about the ability to withstand lawsuits? Isn't that more important than flash crowds?

I'd like another name

A few performance problems are revealed

Yeah, performance problems should be fixed, but fix the name too. Name the next generation P2P client something like FuckTheRIAADickheadCunts. It would be interesting to see it get mentioned in the news each time RIAA sues something related to that P2P network. Call the "servers" instead "ejaculators" or something worse, and go on like that to introduce terms that violate various taboos. Soon enough, it can't get mentioned in the news anymore and (...now I get to my point, and now you will understand I'm not crazy, now you will see how this idea will triumph and free information once and for all...) RIAA's plans to scare customers by getting sue news in the newspapers won't work anymore!

HA HA HA!

Are you listening RIAA!?

We have you now!!!

THE NERDS HAVE YOU!

No, no no.

You're on the right lines, but we should call it something really positive, something they couldn't possibly want to ban. They're pretty hard hearted, they're already happy being know as people who want to ban sharing. But lets see them try to ban JesusKittenShare (the premier opensource implementation of the RespectYourElders protocol) and www.cutebabies.org, the popular .behappy listing site.

Re:No, no no.

I think this is the basic idea behind the PATRIOT Act, the Clear Skies initiative, the Healthy Forests Proposal, No Child Left Behind act, CAN-SPAM act, etc. The forces of evil are way ahead of you on this.

irony

The irony is that a web site dedicated toward serving a p2p protocol expressly designed to rememdy the slashdot effect gets slashdotted.

So why don't they just use Bittorrent to distribute their mirrors?

I don't infringe copyrights

As a solid, upstanding citizen of the United States (a country which has the best government that money can buy), I firmly believe in strongly adhering to all the laws of this fine country.

That's why I always go to thepiratesbay.org.

They are located in Finland, of course, where US Copyright Law doesn't apply. So it's legal for them to offer files for downloading.

And, of course, in the US it's legal to download files. What is illegal is to offer more than $1000 worth of them for uploading.

So, please, let us all keep our Bittorrent downloads legal, folks. Thank you. ;)

On Finland-Sweden, and funny legal threats

Isn't it Sweden?

Yes.

Also, everyone should take a look at their hilarious responses to the letters from lawyers
here [thepiratebay.org].

It's therapeutic to see the slimeball lawyers really getting what is coming to them. These guys have really got a daring attitude :-).

Don't forget the psychology!

Bittorrent did more than get the swapping strategy correct, it fixed the social psychology of p2p. Before, you traded files with other faceless users. This meant you had little investment in the uploads of others. People would join the network and not share files, cap their upload speeds, etc. Generally, this made downloading a slow and painful process. (Not to mention that it was difficult to tell if two similarly named files are the same ... there's too much diversity to get a good spread in file sources).

But Bittorrents have organized around websites. These sites typically require registration and monitor the share ratio of users. Users can no longer leach. There's social stigma attached to it. Also, you have some investment in making sure others have a copy of the file. If you liked it enough to d/l it, you probably want to share. Better yet, the action of the users of the site are focused on the same files, so resources are allocated fairly. Generally, it works better all around.

This leaves out the boost in nerd status of those who have large share ratios and upload lots of torrents. That helps with file availability too.

for those who didn't read the article...

from right below figure 4: In order to test the integrity of meta-data, we donated to Suprnova an account for hosting a mirror. By installing spyware in the HTML code, we have registered each .torrent download and could have easily corrupt the meta-data. We conclude that using donated resources for hosting meta-data entails substantial integrity and privacy risks.

Single point of failure

The study shows how vulnerable BitTorrent is to failures of Suprnova mirrors and trackers. Kill a large .torrent host and you effectively kill the network. Kill a large tracker and you severely cripple it. In comparision, ed2k network is much more resilient to attacks.

First, you don't need servers to distribute ed2k links. A short ASCII string effectively replaces a large .torrent file that needs to be hosted on a large server. You can send an ed2k link by e-mail, IM or post it on Slashdot. Furthermore, ed2k has excellent search capabilities - both via servers (very fast and very efficient) and via distributed Kad[emlia] system (fast and efficient). With the ability to check the filenames and comments for a certain file, you are relatively safe against fakes even when you can't use verified links. Of course, here I deliberately ignore the fact that both networks need "community portals" to inform users about released files, to provide forums for discussion, etc.

Second, the servers play only a secondary role, even if many servers would go down, that would have a small impact on the network because of source exchange. And using Kad it's even possible to operate entirely without servers.

I do not hate BitTorrent, really. Even though I am a long time eMule user and even though I am very annoyed by the apparent popularity of BitTorrent here on /. and elsewhere (as if other networks don't exist), I still don't hate BT. Actually, I tried it again recently and was very satisfied with the download speeds. I don't wish BitTorrent bad. But with the recent developments with police raids on torrent sites and ed2k link sites in Europe the networks will be tested and I am not sure BitTorrent is best prepared for it. Suprnova appears to be safe because of its geographic location, but it still remains a single point of possible failure. I don't think ShareReactor was as critical to the edonkey2000 network.

Interesting stats

Here's a link for the central internet exchange for Finnish ISP's to link together. Coralized FICIX stats [nyud.net].

Compare the stats from week ago, and today. Guess what changed?
Most telling is the last graph indicating traffic for the whole year.
The largest Finnish torrent site, Finreactor got busted by [p2pnet.net] Keskusrikospoliisi [poliisi.fi] (roughly the same as FBI of USA).

I guess they weren't sharing just Linux images ;P

Traffic estimate is suspect

This paper and all of the recent news articles that provide an estimate for BitTorrent protocol traffic use the same source. A single slide in a presentation by someone from Cache Logic shows BT using 1/2 of all P2P traffic at a "tier 1 ISP." Other sources cite P2P traffic at 66% of all 'net traffic. Therefore, BT is 33%.

I think any estimate made without measurements at many major routers would be suspect. While there is no doubt that BT is quite popular, the evidence presented thus far for the amount of traffic using BT protocol is extremely flimsy. I would take it with a grain of salt.

burris

Slashdot effect salve

The real breakthru for distributed P2P tech will come when someone publishes a BitTorrent content distributor that can be plugged transparently in front of an HTTPD. So I hit http://www.whatever.com , and get my HTTP response, with cache and timing headers intact. But behind the scenes, the "www" host is really the entry point to a distributed server network, a pool of interconnected "torrent" servers that transparently balance the traffic throughout the capacitance of the protocol network. Those servers actually tap the "real" HTTPD behind that network only to check for updated content, which is distributed to the network on demand, to be passed through to requesting clients. The clients speak only HTTP, and can't tell the difference between the real HTTPD and the distributed network proxies.

As long as I'm asking Santa, I'll be more specific. That "www" host has its DNS resolved by the nameserver at whatever.com , which hands out IP#s of the other "torrent" servers distributed around the "Web". torrent servers get the IP# of the real host at whatever.com, so they get content. There are problems: HTTPS requires each serialized object requested/replied to be encrypted with/for the actual private key of the requesting client, unknown until the request is made. And "CGI" or other dynamic content creates a huge space of permuted object states. But, Santa, Google figured out how to deal with all this in a centralized datacenter, and they're damn fast. Get the elves on this, and children around the world will sleep with visions of sugarplums streaming to their download directories.

My response to the author:

I have a few comments on your analysis of the BitTorrent protocol... My main criticism is that you are analyzing BitTorrent in combination with pirate web pages as a P2P file sharing system, when BitTorrent's real purpose is to be a file DISTRIBUTION system.

BitTorrent is designed to replace and enhance the performance of a standard http or ftp download server. Where even ten simultaneous downloads can slow the performance of most inexpensive server setups to a crawl, BitTorrent can easily handle ten thousand or more, and in this it is an enormous success.

One necessary element of a true BitTorrent distribution is a dedicated seed server. This server ought to be always working, and should have a significant amount of bandwidth behind it; I'd recommend 30KB/s minimum, but more is better. You complain that seeders are "punished" and this is why torrents die, but while long-term seeders are nice, they aren't necessary. It is better for me as a content distributor to allow people to close their torrent and play with their new download as soon as they'd like to. Having torrents die off when interest fades is an artifact of misuse of this specification.

You worry about pollution on Suprnova.org, and so do I; there's no reason why it wouldn't exist. But as BitTorrent was normally intended this isn't a problem at all. People visiting Blizzard's website to download content via BitTorrent (actually Blizzard uses a modified downloader, but the concept would be the same if users received a standard .torrent file) would obviously receive a genuine .torrent file, and the data in that file verifies the data received in the download. It's only torrent file redistributors like Suprnova.org where you'd need to be concerned about pollution.

You're also concerned about tracker availability. I recommend content distributors run their own trackers, which is an easy task given the numerous types of trackers available, including script-based trackers. There's no reason for a tracker to go down unless the web server goes down, in which case no one would probably be able to get a copy of the .torrent file anyway, and a standard download would also be blocked.

As a sharing method BitTorrent indeed has some deficiencies, but it simply wasn't designed for that. That BitTorrent is being misused for that purpose only testifies to its effectiveness. Perhaps a sharing system with elements taken from BitTorrent will someday arise; I know Suprnova.org is attempting to create one with "Exeem". But please don't badmouth BitTorrent. :-)

Re:I work for..

Two words - avoid BitTorrent. HUGE investigations are going on to bust bittorrent users

Wrong. There are no investigations going on to bust bittorrent users. There are investigations going on to bust people doing illegal file sharing, and some of them happen to be using bittorrent.