Distributed p2p technologies have always been fascinating to me. Bittorrent is cool not because you can download pirated content for free, but because it’s an amazing piece of technology.
At some point I read and researched a lot about how DHTs (distributed hash tables) work. DHTs are not part of the original bittorrent protocol, but after trackers were increasingly under threat to be closed for copyright infringment, “trackerless” features were added to the protocol. A DHT is distributed among all peers and holds information about which peer holds what data. Once you are connected to a peer, you can query it for their knowledge on who has what.
During my research (which was with no particular purpose) I took a note on many resources that I thought useful for understanding how DHTs work and possibly implementing something ontop of them in the future. In fact, a DHT is a “shared database”, “just like” a blockchain. You can’t trust it as much, but proving digital events does not require a blockchain anyway. My point here is – there is a lot more cool stuff to distributed / p2p systems than blockchain. And maybe way more practical stuff.
It’s important to note that the DHT used in BitTorrent is Kademlia. You’ll see a lot about it below.
Anyway, the point of this post is to share the resources that I collected. For my own reference and for everyone who wants to start somewhere on the topic of DHTs.
- Bittorrent DHT protocol – a nice explanation how DHT is used in bittorent (here’s a list of all bittorrent protocol enhancements)
- Kademlia: design specification – if you want in-depth understanding of how Kademlia works
- MLDHT – a Java library for the Mainline bittorrent DHT
- OpenKAD – Java implementation of the Kademlia DHT (copied from Google Code))
- TTorrent – a Java bittorrent library, useful to understand the bittorrent protocol. It doesn’t support DHT, but this old fork does. There’s an old pull request for DHT support as well.
- Kademlia DHT JS implementation – javascript implementation of kademlia
- Vuze mlDHT plugin – Vuze (Azureus) is using its own DHT, but it has a plugin that allows connecting to the mainline DHT. It’s useful because it’s a jar and can be used as a dependency
- Azureus Core – Vuze (Azureus) source code, including DHT implementation (just ignore the username/password request)
- Tampering with distributed hash tables – looking at the security of DHTs and demonstrating how one can store random data on other people machines without them knowing
- Kademlia over HTTP – another JavaScript implementation that allegedly brings DHT to mobile
- WebTorrent – browser-based torrent client, seen at webtorrent.io
- Some more Java BitTorrent clients, not necessarily supporting DHT, but useful for understanding the protocol: LWJTorrent, jlibtorrent, born-again-shark
I hope the list is interesting and useful. It’s not trivial to think of other uses of DHTs, but simply knowing about them and how they work is a good thing.
| Published on Java Code Geeks with permission by Bozhidar Bozhanov, partner at our JCG program. See the original article here: Resources on Distributed Hash Tables Opinions expressed by Java Code Geeks contributors are their own. |
