DistribuStream

Ruby
1

DistribuStream is an open peer-to-peer communications system which aims
to
become the successor to BitTorrent. You can read about it at
http://distribustream.org

Unlike BitTorrent, DistribuStream facilitates streaming progressive
downloads, making it an open alternative to proprietary protocols like
Joost.

DistribuStream is written on top of the Ruby/EventMachine library which,
at
least on Linux, allows it to scale to thousands of concurrent
connections.

It was developed collaboratively by ClickCaster, Inc. and the University
of
Colorado Computer Science program. The underlying protocol (Peer
Distributed Transfer Protocol or PDTP) has received an IANA port
assignment
of 6086. All source code is available under the GNU General Public
License
v3. We don’t plan a Ruby-licensed version at this time, sorry.

The protocol is philosophically different from BitTorrent, which relies
on
emergent, swarm-like behavior for traffic scheduling. All client/server
and
peer-to-peer communications can be modeled as simple state machines.
The
onus of traffic scheduling is placed entirely on the server.

Client/server communication is accomplished with a lightweight JSON
asynchronous messaging format (presently running over TCP).
Peer-to-peer
communication is accomplished with HTTP/1.1 using the Mongrel HTTP
server.
The entire protocol behaves as an ad hoc HTTP caching proxy network.

If you’d like to check it out, it’s available as a RubyGem:

gem install distribustream

Check out the README file for help getting started:

http://distribustream.rubyforge.org/svn/trunk/README

An example configuration file is available here:

http://distribustream.rubyforge.org/svn/trunk/conf/example.yml

Enjoy!

2

Tony A. wrote:

DistribuStream is an open peer-to-peer communications system which aims to
become the successor to BitTorrent.

I’m quite interested in the “ad hoc HTTP caching proxy network” part.
Some of the folks I work with in the CE dept at Berkeley want to use
HTTP for services in ad hoc vehicle networks (VANET). In another
project, I’m using ruby for HTTP proxying over mobile wireless networks
using UDP broadcast on the wireless link (DSRC), instead of TCP, to
avoid the setup time. Caching is valuable because the networks are so
limited in bandwidth, latency, and connection duration.

However: there’s gotta be a better name than DistribuStream. Why call it
“distribu” stream if its main difference from BitTorrent is that it is
more centralized? (“The onus of traffic scheduling is placed entirely on
the server.” Isn’t that going to be a bottleneck, anyway? Or can the
servers be distributed? How do they cooperate?)

If you’ll forgive the suggestions, what about…?

Distributary (a branching river)
Bitstributary
Roost (like Joost, but “R” for ruby)
Distribrew
Brew
Brewst

Ignore the last few. I’m probably just feeling bad about not drinking
beer with all the great ruby minds.

Thanks for announcing this.

3

On 10/31/07, Joel VanderWerf [email protected] wrote:

I’m quite interested in the “ad hoc HTTP caching proxy network” part.
Some of the folks I work with in the CE dept at Berkeley want to use
HTTP for services in ad hoc vehicle networks (VANET). In another
project, I’m using ruby for HTTP proxying over mobile wireless networks

It doesn’t provide a caching HTTP proxy network in the traditional
sense.
The protocol works by the server directing clients to perform successive
HTTP GET and PUT requests to assigned peers, with each peer running a
local
HTTP server and authenticating traffic against the central server. It
effectively provides a little botnet you can use for the purposes of
HTTP
caching and control the way you see fit.

If you wanted to use it to set up a mini-CDN on a wireless network, the
protocol would facilitate it but the current implementation currently
isn’t
set up to do that. With a little hacking it’d be pretty easy to do…

using UDP broadcast on the wireless link (DSRC), instead of TCP, to

avoid the setup time. Caching is valuable because the networks are so
limited in bandwidth, latency, and connection duration.

The plan is to go UDP, because overall it’d be a big win for improving
latency and lowering overhead, like you expressed. The entire protocol
can
be modeled as state machines to begin with so it’s not hard to add
timeouts/retries and ditch TCP altogether. There are no particularly
large
datagrams in the entire protocol, not do the size of the packets scale
with
the size of the data being transferred or anything like that.

However: there’s gotta be a better name than DistribuStream. Why call it

“distribu” stream if its main difference from BitTorrent is that it is
more centralized?

Haha, BitTorrent wasn’t really a consideration when we chose that name.
While the transfer control is centralized, the purpose is to provide
distributed content streaming across a peer network.

"The onus of traffic scheduling is placed entirely on

the server." Isn’t that going to be a bottleneck, anyway? Or can the
servers be distributed? How do they cooperate?

It won’t provide any more of a bottleneck than BitTorrent trackers now
in
terms of bandwidth, and due to the way the protocol is designed
messaging
between “peer” and “tracker” will have substantially lower overhead. If
there’s a bottleneck it’s going to be an algorithmic one in the server’s
network regulation.

There’s no clustering now, although that could easily be added with DRb.

If you’ll forgive the suggestions, what about…?

I don’t think we’ll be changing the name at this point, sorry… :slight_smile: