# Ruby-dipus 0.1.2 **URL:** \ **Category:** Ruby\ **Created:** [December 27, 2005, 8:07am UTC](https://www.ruby-forum.com/t/ruby-dipus-0-1-2/77044 "2005-12-27T08:07:21Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Ilmari\_H](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/a587f6/32.png) [@Ilmari\_H](https://www.ruby-forum.com/u/Ilmari_H)\ **Post date:** [December 27, 2005, 8:07am UTC](https://www.ruby-forum.com/t/ruby-dipus-0-1-2/77044/1 "2005-12-27T08:07:21Z")
# DIPUS - Distributed IPC by Proxying UNIX Sockets Sockets with user authentication, file permissions, encryption, service discovery and network transparent addressing. ## Using services: host1$ dcat lame-svc foo.wav | dcat madplay.host3 host1$ dcat sep://example.com/image/jpeg/to/image/png foo.jpeg | dcat ssh://removed\_email\_address@domain.invalid/image/png/to/text/plain | less ## Creating services: host3$ dipus-local-pipe ‘madplay -’ madplay host2$ dipus-export-pipe ‘lame - -’ lame-svc ## Some performance numbers: Local throughput around 50MBps (or 100MBps, depending whether you use socat as a client or not), unencrypted network throughput 9.8MBps in a 100Mbps LAN, CPU-limited when using SSH (my other computer is slow.) ## In what languages can I use and create services? Anything that supports a) UNIX domain sockets, and b) listing filesystem directories. So, just about every language available. If you install the helper scripts in bin, you don’t even need those, just the ability to read from stdin and write to stdout. Download: [http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2.tar.gz](http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2.tar.gz) RDocs: [http://www.cs.helsinki.fi/u/irkheikk/dipus-rdoc/](http://www.cs.helsinki.fi/u/irkheikk/dipus-rdoc/) Darcs repo: [http://dark.fhtr.org/repos/ruby-dipus](http://dark.fhtr.org/repos/ruby-dipus) Readme: [http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/readme.txt](http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/readme.txt) Design: [http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/doc/dipus-design.txt](http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/doc/dipus-design.txt) Protocol spec: [http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/doc/dipus-specification.txt](http://www.cs.helsinki.fi/u/irkheikk/ruby-dipus-0.1.2/doc/dipus-specification.txt) ^^^^^^^^^^^^^^^^^^^^^^^^^^ Hi, designed and implemented a distributed IPC system that takes lessons from UNIX (keep it simple) and HTTP (keep it simple.) Opinions on the design or anything else? I’m dying to get some feedback, writing stuff in a bubble can take you only so far. ## And, now, with the first version of this December project out of the way, I’m off to librend land~ Ilmari H. vvvvvvvvvvvvvvvvvvvvvvvvvv ## What is this for? You have a program. You want to access it remotely with the minimum hassle. It’d be nice to access it efficiently locally too. Without rewriting your connection code. And it’d be nice to put some connections over SSH. And be notified when a service dies or a new one is created. And not expose any of the individual services to the network, or even to other users on the machine running them. And it’d be cool to be able to have a different name and protocol for each service without having to shoehorn FTP over XML-RPC. Oh, also, display a real-time list of services running on the local network and on explicitly defined machines that aren’t on the local network. And it’d be especially cool if all the software could just use existing libraries like sockets and files. So, take UNIX sockets and sprinkle some sugar to make them usable for distributed IPC. ## Supported systems Should work on Linux and \*BSD, likely on OS X too. No Windows support, although it’s theoretically possible. ## Required Ruby version 1.8.2+ (one released in 2005) cat ## Recommended OpenSSH version \> 3.9 (for connection sharing support (ControlPath)) ssh-agent setup (for maximum convenience) socat More than one computer. ## How does it work? To create a service, make a listening socket in a subdir of /tmp/dipus, so that the path is like: /tmp/dipus/exported/audio/x-wav/to/audio/mpeg/lame.mybox…0 To access a service, open its socket. To make a service only accessible to members of group ‘audio’: chgrp audio socket\_name chmod 770 socket\_name Listing services that you can see: tree /tmp/dipus DIPUS provides helper methods and daemons to manipulate the sockets in /tmp/dipus, e.g. create proxies to remote sockets, delete dead sockets, provide network access to exported sockets, find sockets by their service name, hostname, or protocol. ## Quick usage example # (optional) start dipusd to automatically discover and import services, # export services, and delete dead sockets. # [kig@desktop] dipusd [kig@encode] dipusd # Create local madplay service for playing MP3s on # the computer connected to speakers. # The first argument is the command to run for each incoming connection. # The second argument is service\_name/protocol, e.g. my\_www/http. # # The following command creates a socket with the path # /tmp/dipus/local/audio/mpeg/to/audio/out/madplay.stereo…0 # and runs ‘madplay -’ for each incoming connection, setting the # connection to madplay’s stdin and stdout. # [kig@stereo] dipus-local-pipe ‘madplay -’ madplay/audio/mpeg/to/audio/out # Create an exported lame service for encoding wavs to MP3s. # [kig@encode] dipus-export-pipe ‘lame - -’ lame/audio/x-wav/to/audio/mpeg # Manually import the services from stereo. # Import over SSH to gain access to all services and not just exported ones. # [kig@desktop] dipus-import ssh://stereo # Dipusd should’ve discovered and imported the lame service from encode, # so we don’t need to manually import that. But if that didn’t happen, we # can import it by doing `dipus-import sep://encode`, sep being # Socket Export Protocol (see doc/dipus-specification.txt) # Now, let’s pipe some wavs to the stereo. # First encode the wav to MP3, using any service with a protocol that says # that the service converts audio/x-wav to audio/mpeg. # Play using a service named madplay running on stereo. # [kig@desktop] dcat audio/x-wav/to/audio/mpeg \*.wav | dcat madplay.stereo # We could also access the services without importing them by giving the # service URI as the service name. # # Here’s the above piping example using service URIs. # Dcat lists the services matching the URI and uses the first match. # [kig@desktop] dcat sep://encode/audio/x-wav/to/audio/mpeg \*.wav | dcat ssh://stereo/audio/mpeg/to/audio/out/madplay.stereo…0 ## List of DIPUS commands Pass --help as an argument to get the full USAGE. - dls – list services dls sep://foobar/audio dls ssh://barfoo - dcat – pipe stdin and/or given files to a DIPUS socket dcat lame foo.wav bar.wav \> foobar.mp3 dcat ssh://foobar/image/png/to/image/jpeg cat.png \> cat.jpg - dipus-import – imported all found service from an URI dipus-import ssh://foobar/audio dipus-import sep://barfoo - dipusd – start exporter, importer, advertiser, discoverer, dead socket monitor, and default services - dipus-exporter – start exporter - dipus-discoverer – discover exporters on the LAN and print their addresses - dipus-service-monitor – monitor for service changes on localhost, used by SSH importers - dipus-ssh-importer – import services over SSH to user’s DIPUS subdirectory (/tmp/dipus/username), can use discovery to find exporter-running hosts on localnet - dipus-local-pipe – create a local DIPUS service that forks the given command for the connection dipus-local-pipe ‘head’ head/text - dipus-export-pipe – create an exported DIPUS service like above dipus-export-pipe ‘madplay -’ madplay/audio/mpeg/to/audio/out ## Using socat to create and connect to DIPUS services # proxy a web server mkdir -p /tmp/dipus/exported/http socat UNIX-LISTEN:/tmp/dipus/exported/http/foo.foobar…0,fork TCP4-CONNECT:foo.example.com:80 rm /tmp/dipus/exported/http/foo.foobar…0 # using a lame service to encode a wav socat foo.wav UNIX-CONNECT:/tmp/dipus/local/audio/x-wav/to/audio/mpeg/lame.foobar…0 \> foo.mp3 # or by using dls to find the filename socat foo.wav UNIX-CONNECT:`dls -1 -f lame` \> foo.mp3 # proxy a DIPUS service to a TCP socket socat TCP4-LISTEN:1234,fork EXEC:‘dcat lame’ ## Possibilities Carry around a mobile phone with an Internet connection. Use it to run a media player. The media player sees files on your home computer and the files on the local network. Then it pipes the files in its playlist to the closest encode server, and from there to the wanted output (headphones, speakers, file, streaming server…) Send your 3D animation file over the Internet to a thousand idle machines to render. Walk into an auditorium and display slides on the overhead projector, streamed from a network mount, controlled from your mobile. Run a search engine, giving URIs to remote services, making automatable the task of finding and utilizing a network service. ---
**Author:** ![Ilmari\_H](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/a587f6/32.png) [@Ilmari\_H](https://www.ruby-forum.com/u/Ilmari_H)\ **Post date:** [December 27, 2005, 10:04am UTC](https://www.ruby-forum.com/t/ruby-dipus-0-1-2/77044/2 "2005-12-27T10:04:34Z")
Quoting Ilmari H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): Wow, that’s exactly what I was looking for! Thank you! Just one “but”: I think using a known path under /tmp/dipus may be a security risk. ---
**Author:** ![Ilmari\_H](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/a587f6/32.png) [@Ilmari\_H](https://www.ruby-forum.com/u/Ilmari_H)\ **Post date:** [December 27, 2005, 9:03pm UTC](https://www.ruby-forum.com/t/ruby-dipus-0-1-2/77044/3 "2005-12-27T21:03:41Z")
On 12/27/05, Pau Garcia i Quiles [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Quoting Ilmari H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): > > Wow, that’s exactly what I was looking for! Thank you! Thank you for trying it out, let me know if / when you find bugs 😉 > Just one “but”: I think using a known path under /tmp/dipus may be a security > risk. How do you mean?