In Japan, copyright infringement is a criminal offense (unlike here, although our corporate fascist leanings might push it there in due time), so file sharing is a dangerous game. Because of this, sophisticated anonymizing and encryption techniques have been built into Japanese peer-to-peer programs for at least 5 years now.

The Japanese have a habit of latching onto odd western imports. The first time I was there, everyone spontaneously decided that they were cowboys. The trend-setters and trend-followers are never quite far behind, leading to interesting results like floppy socks or intentionally forgoing braces to be more yaeba. The flip side is that the Japanese adopt technical things that don't quite catch elsewhere, like Laserdisc, Yahoo!, and WinMX.

WinMX was the first p2p file sharing program with mainstream popularity in Japan, but there was a problem: the police could figure out who you were and what you were uploading. The successor, developed by a Japanese researcher in a Computer Engineering department, was WinNY (M+1, X+1), which exploded into popularity.

WinNY is more similar to anonymizing networks like Freenet than WinMX. Users join the network by having an encrypted address of other users on the network. Every time you connect to a node, your peer list is updated, so after long periods of connectivity you have a pretty good idea of most of the machines in the network. You can gain new information about other nodes on the networks only via their encrypted address, and user validation is entirely through a username/tripcode system (a username attached to a password hash). Each node choses a number of "cluster keywords" that describe the content they are providing (or searching for); as you cycle through the nodes in your cached version of the network, your searches are prioritized based on how the nodes match your cluster keywords. The important part is that most everything is encrypted and sent through the network in a way that anonymizes the uploader.

Or did. WinNY's anonymizing was cracked, and arrests were made as long ago as 2003. An engineer immediately picked up where WinNY left off, and created another (closed source) program called Share. Share is similar to WinNY in many ways (used sha1 for file hashes instead of md5), and similarly was cracked in 2006.

Where WinNY and Share left off, a new leader "Perfect Dark" (yes, like the video game) has emerged. Perfect Dark's interface is quite a departure from WinNY and Share's. The basic structure is that of a large distributed filesystem where file block lookups are done via a distributed hash table. Every user on the network has at least 100Kb/sec devoted to downloading or uploading data into a 40 gig minimum network cache. When a user starts to download a file, various criteria are checked (the upload speed of his node, the amt. of network storage he has cached, etc) and then pieces come in from nodes that have that block cached. The pieces, once uploaded, are shuffled around the network constantly via each node's cache until they expire (based on the number and frequency of downloads), at which point they are discarded.

This leads to a system where it is nearly impossible (and currently is) to figure out who was the original uploader of the content; the only thing you can see is that someone is connected to the network. Technically, it would be impossible to connect to the network and be sure you are not assisting in copyright violation, since your cache is encrypted and you can't figure out what's in it, but it should also be impossible to figure out who the original violator was. Thusfar, nobody has been arrested or brought to trial for being a part of a distributed p2p network in Japan.

Over time, various methods of combating these networks have arisen. The major way that files are announced as available is via the 2ch message boards, which themselves are anonymous. The ultimate identification of a file is via a hash; if people upload a duplicate to the cache, the blocks will all be identified as the same and delivered in the same fashion. The relative weakness of md5 has led to hash spoofing attacks on the WinNY network.

Another popular attack is the flooding of fake data, similar to p2p attacks done by the MPAA and RIAA in the US. Content providers masquerading as nodes can upload large volumes of fake content and use network resources (cache space & bandwidth) that would otherwise be taken by "real" file transfers. Perfect Dark has a built in protection against this; a distributed voting system whereby hashes can be up or down voted to hasten or delay expiry. Only peers that have downloaded the content are allowed to vote on it, which can potentially delay ballot stuffing techniques.

In the end, the sophistication of Japanese p2p software is pretty interesting to study. Met with a real world (although ethically dubious) problem, a large, ad-hoc, block-level distributed network storage mechanism has been created with near guaranteed anonymity and user moderated cache expiry. In contrast, p2p in the western world has been aimed at convenience and the timeliness of delivery.

Jul 14 2008