[ANN] Ebb Web Server


#1

Hello Mongrel Users,

I’m writing a web server called Ebb. It’s written in C, makes use of
the Mongrel HTTP parser, and uses libev its event loop. The goal is to
be small, fast, and language independent server that can host web
frameworks. I have written a small Ruby binding which provides a Rack
handler - this will allow Ebb to host Rails, Merb, and other Ruby
frameworks. In the future I will write a Python WSGI binding.

The design is similar to the evented Mongrel web server. Connections
are processed as follows:

  1. libev loops and waits for incoming connections.
  2. When Ebb can read from a client socket, it passes the buffer into the
    mongrel state machine which parses the headers into name value pairs.
  3. Ebb starts a new thread and passes the request information and peer
    socket
    to a user supplied callback. The thread lasts only for the length of
    that
    callback.
  4. The included Ruby binding, supplying this callback transforms the
    request
    into a Rack compatible “env” variable and passes it on a Rack
    adapter.

The code measures in at less than 1000 lines of C code.

There is much work to do; it is not ready for use. I am soliciting
help from the community for testing and development. You may browse
the git repository at http://repo.or.cz/w/ebb.git or check out the
code with this command:
git clone git://repo.or.cz/ebb.git
I release Ebb under the MIT license.
It is very fun to program Ebb so I suggest you do too :slight_smile:

Ry Dahl


#2

On Jan 14, 2008 2:43 PM, ry dahl removed_email_address@domain.invalid wrote:

Hello Mongrel Users,

I’m writing a web server called Ebb. It’s written in C, makes use of
the Mongrel HTTP parser, and uses libev its event loop. The goal is to
be small, fast, and language independent server that can host web
frameworks. I have written a small Ruby binding which provides a Rack
handler - this will allow Ebb to host Rails, Merb, and other Ruby
frameworks. In the future I will write a Python WSGI binding.

Nice ry!

There is much work to do; it is not ready for use. I am soliciting
help from the community for testing and development. You may browse
the git repository at http://repo.or.cz/w/ebb.git or check out the
code with this command:
git clone git://repo.or.cz/ebb.git
I release Ebb under the MIT license.
It is very fun to program Ebb so I suggest you do too :slight_smile:

A svn repo clone? (git-svn)? Not all jumped into the git bandwagon :slight_smile:


Luis L.
Multimedia systems

A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams


#3

Good luck with your new server! I also suggest looking at Thin which
has been developed very recently and seems quite promising:

http://code.macournoyer.com/thin/

According to the benchmarks it’s the fastest Ruby web server currently.


#4

Hey Ry,

Your project looks amazing. I had a look at rev to use libev, but it
seems to be Ruby 1.9 only and I’m far from a C expert. So right now
Thin uses EventMachine.

I really like your idea and I think we could help each other in
building something very cool.
If you’d like to participate in Thin and have some suggestions on how
to make it better we should have a talk.
Send me an email if you’re interested.

Marc


#5

A svn repo clone? (git-svn)? Not all jumped into the git bandwagon :slight_smile:

Sorry - my hosting abilities are limited. I also find the multitude of
revision control software annoying (but git seems functional!)
apt-get install git

ry


#6

On Jan 14, 2008 4:56 PM, ry dahl removed_email_address@domain.invalid wrote:

A svn repo clone? (git-svn)? Not all jumped into the git bandwagon :slight_smile:

Sorry - my hosting abilities are limited. I also find the multitude of
revision control software annoying (but git seems functional!)
apt-get install git

no apt-get on Windows. And git support on it is a bit flacky
(I hear another git versus *nix versus windows discussion starting).

I’m on Bazaar, which is the same used by Ubuntu :slight_smile:

http://bazaar-vcs.org/BzrVsGit

Anyway, have git but not on my Windows box.

Will check your code this weekend :wink:


Luis L.
Multimedia systems

A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams


#7

On Jan 14, 2008, at 8:43 AM, ry dahl wrote:

are processed as follows:
callback.
code with this command:
git clone git://repo.or.cz/ebb.git
I release Ebb under the MIT license.
It is very fun to program Ebb so I suggest you do too :slight_smile:

Ry Dahl

Hey Ry-

Is glib-2.0 a dependency? It seems like 2.0 is an old version of
glib. So ebb doesn’t compile on leopard out of the box. What do I need
to install to make it work?

~/ebb > make
gcc -g -Wall pkg-config --cflags glib-2.0 -I/opt/libev-2.01/include -
L/opt/libev-2.01/lib -c tcp.c -o tcp.o
Package glib-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `glib-2.0.pc’
to the PKG_CONFIG_PATH environment variable
No package ‘glib-2.0’ found
tcp.c:18:16: error: ev.h: No such file or directory
tcp.c:19:18: error: glib.h: No such file or directory
In file included from tcp.c:23:
tcp.h:48: error: syntax error before ‘GQueue’
tcp.h:48: warning: no semicolon at end of struct or union
tcp.h:53: error: syntax error before ‘*’ token
tcp.h:53: warning: type defaults to ‘int’ in declaration of
‘accept_watcher’
tcp.h:53: warning: data definition has no type or storage class
tcp.h:55: error: syntax error before ‘}’ token
tcp.h:72: error: syntax error before ‘ev_io’
tcp.h:72: warning: no semicolon at end of struct or union
tcp.c: In function ‘tcp_client_write’:
tcp.c:33: error: dereferencing pointer to incomplete type
tcp.c:34: error: dereferencing pointer to incomplete type
tcp.c:36: warning: implicit declaration of function ‘g_log’
tcp.c:36: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:36: error: (Each undeclared identifier is reported only once
tcp.c:36: error: for each function it appears in.)
tcp.c: At top level:
tcp.c:46: warning: ‘struct ev_io’ declared inside parameter list
tcp.c:46: warning: its scope is only this definition or declaration,
which is probably not what you want
tcp.c: In function ‘tcp_client_on_readable’:
tcp.c:48: error: dereferencing pointer to incomplete type
tcp.c:52: error: ‘EV_ERROR’ undeclared (first use in this function)
tcp.c:53: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:57: error: dereferencing pointer to incomplete type
tcp.c:58: error: dereferencing pointer to incomplete type
tcp.c:59: error: dereferencing pointer to incomplete type
tcp.c:60: error: dereferencing pointer to incomplete type
tcp.c:62: error: dereferencing pointer to incomplete type
tcp.c:64: error: dereferencing pointer to incomplete type
tcp.c:64: error: dereferencing pointer to incomplete type
tcp.c:67: warning: implicit declaration of function ‘g_debug’
tcp.c:81: error: dereferencing pointer to incomplete type
tcp.c:81: error: dereferencing pointer to incomplete type
tcp.c:81: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_client_new’:
tcp.c:95: warning: implicit declaration of function ‘g_new0’
tcp.c:95: error: syntax error before ‘tcp_client’
tcp.c:97: error: dereferencing pointer to incomplete type
tcp.c:99: error: dereferencing pointer to incomplete type
tcp.c:99: error: dereferencing pointer to incomplete type
tcp.c:99: error: dereferencing pointer to incomplete type
tcp.c:100: error: dereferencing pointer to incomplete type
tcp.c:101: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:105: error: dereferencing pointer to incomplete type
tcp.c:105: error: ‘TRUE’ undeclared (first use in this function)
tcp.c:107: error: dereferencing pointer to incomplete type
tcp.c:113: error: dereferencing pointer to incomplete type
tcp.c:115: error: dereferencing pointer to incomplete type
tcp.c:115: error: syntax error before ‘struct’
tcp.c:116: error: dereferencing pointer to incomplete type
tcp.c:117: warning: implicit declaration of function ‘ev_init’
tcp.c:117: error: dereferencing pointer to incomplete type
tcp.c:118: warning: implicit declaration of function ‘ev_io_set’
tcp.c:118: error: dereferencing pointer to incomplete type
tcp.c:118: error: dereferencing pointer to incomplete type
tcp.c:118: error: ‘EV_READ’ undeclared (first use in this function)
tcp.c:118: error: ‘EV_ERROR’ undeclared (first use in this function)
tcp.c:119: warning: implicit declaration of function ‘ev_io_start’
tcp.c:119: error: dereferencing pointer to incomplete type
tcp.c:119: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_client_stop_read_watcher’:
tcp.c:131: error: dereferencing pointer to incomplete type
tcp.c:134: error: dereferencing pointer to incomplete type
tcp.c:136: warning: implicit declaration of function ‘ev_io_stop’
tcp.c:136: error: dereferencing pointer to incomplete type
tcp.c:136: error: dereferencing pointer to incomplete type
tcp.c:137: error: dereferencing pointer to incomplete type
tcp.c:138: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_client_free’:
tcp.c:146: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_client_close’:
tcp.c:153: error: dereferencing pointer to incomplete type
tcp.c:155: error: dereferencing pointer to incomplete type
tcp.c:156: error: dereferencing pointer to incomplete type
tcp.c:156: error: ‘FALSE’ undeclared (first use in this function)
tcp.c: In function ‘tcp_server_new’:
tcp.c:165: error: syntax error before ‘tcp_server’
tcp.c:167: error: dereferencing pointer to incomplete type
tcp.c:168: error: dereferencing pointer to incomplete type
tcp.c:170: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:175: error: dereferencing pointer to incomplete type
tcp.c:189: error: dereferencing pointer to incomplete type
tcp.c:189: warning: implicit declaration of function ‘ev_loop_new’
tcp.c:191: error: dereferencing pointer to incomplete type
tcp.c:191: warning: implicit declaration of function ‘g_queue_new’
tcp.c:192: error: dereferencing pointer to incomplete type
tcp.c:192: error: ‘FALSE’ undeclared (first use in this function)
tcp.c: In function ‘tcp_server_free’:
tcp.c:203: warning: implicit declaration of function ‘g_queue_free’
tcp.c:203: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_server_close’:
tcp.c:211: error: dereferencing pointer to incomplete type
tcp.c:214: warning: implicit declaration of function ‘g_queue_pop_head’
tcp.c:214: error: dereferencing pointer to incomplete type
tcp.c:214: warning: assignment makes pointer from integer without a cast
tcp.c:217: error: dereferencing pointer to incomplete type
tcp.c:218: error: dereferencing pointer to incomplete type
tcp.c:219: error: dereferencing pointer to incomplete type
tcp.c:221: error: dereferencing pointer to incomplete type
tcp.c:222: error: dereferencing pointer to incomplete type
tcp.c:223: error: dereferencing pointer to incomplete type
tcp.c:225: error: dereferencing pointer to incomplete type
tcp.c:227: error: dereferencing pointer to incomplete type
tcp.c:227: error: dereferencing pointer to incomplete type
tcp.c:228: error: dereferencing pointer to incomplete type
tcp.c:229: error: dereferencing pointer to incomplete type
tcp.c:231: warning: implicit declaration of function ‘ev_unloop’
tcp.c:231: error: dereferencing pointer to incomplete type
tcp.c:231: error: ‘EVUNLOOP_ALL’ undeclared (first use in this function)
tcp.c:232: warning: implicit declaration of function ‘ev_loop_destroy’
tcp.c:232: error: dereferencing pointer to incomplete type
tcp.c:233: error: dereferencing pointer to incomplete type
tcp.c:235: error: dereferencing pointer to incomplete type
tcp.c:236: error: dereferencing pointer to incomplete type
tcp.c:236: error: ‘FALSE’ undeclared (first use in this function)
tcp.c: At top level:
tcp.c:242: warning: ‘struct ev_io’ declared inside parameter list
tcp.c: In function ‘tcp_server_accept’:
tcp.c:244: error: dereferencing pointer to incomplete type
tcp.c:247: error: dereferencing pointer to incomplete type
tcp.c:248: error: dereferencing pointer to incomplete type
tcp.c:251: error: ‘EV_ERROR’ undeclared (first use in this function)
tcp.c:252: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:258: warning: implicit declaration of function ‘g_queue_push_head’
tcp.c:258: error: dereferencing pointer to incomplete type
tcp.c:258: error: ‘gpointer’ undeclared (first use in this function)
tcp.c:258: error: syntax error before ‘client’
tcp.c:260: error: dereferencing pointer to incomplete type
tcp.c:261: error: dereferencing pointer to incomplete type
tcp.c:261: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_server_listen’:
tcp.c:276: error: dereferencing pointer to incomplete type
tcp.c:277: error: dereferencing pointer to incomplete type
tcp.c:280: error: dereferencing pointer to incomplete type
tcp.c:281: error: dereferencing pointer to incomplete type
tcp.c:283: error: dereferencing pointer to incomplete type
tcp.c:284: error: dereferencing pointer to incomplete type
tcp.c:284: error: dereferencing pointer to incomplete type
tcp.c:285: error: ‘G_LOG_LEVEL_ERROR’ undeclared (first use in this
function)
tcp.c:288: error: dereferencing pointer to incomplete type
tcp.c:288: error: dereferencing pointer to incomplete type
tcp.c:298: error: dereferencing pointer to incomplete type
tcp.c:298: error: dereferencing pointer to incomplete type
tcp.c:298: error: dereferencing pointer to incomplete type
tcp.c:304: error: dereferencing pointer to incomplete type
tcp.c:310: error: dereferencing pointer to incomplete type
tcp.c:310: error: ‘FALSE’ undeclared (first use in this function)
tcp.c:311: error: dereferencing pointer to incomplete type
tcp.c:311: error: ‘TRUE’ undeclared (first use in this function)
tcp.c:313: error: dereferencing pointer to incomplete type
tcp.c:313: error: syntax error before ‘struct’
tcp.c:314: error: dereferencing pointer to incomplete type
tcp.c:315: error: dereferencing pointer to incomplete type
tcp.c:316: error: dereferencing pointer to incomplete type
tcp.c:318: error: dereferencing pointer to incomplete type
tcp.c:319: error: dereferencing pointer to incomplete type
tcp.c:319: error: dereferencing pointer to incomplete type
tcp.c:319: error: ‘EV_READ’ undeclared (first use in this function)
tcp.c:319: error: ‘EV_ERROR’ undeclared (first use in this function)
tcp.c:320: error: dereferencing pointer to incomplete type
tcp.c:320: error: dereferencing pointer to incomplete type
tcp.c:321: warning: implicit declaration of function ‘ev_loop’
tcp.c:321: error: dereferencing pointer to incomplete type
tcp.c: In function ‘tcp_server_address’:
tcp.c:331: error: dereferencing pointer to incomplete type
tcp.c:332: error: dereferencing pointer to incomplete type
tcp.c:335: warning: control reaches end of non-void function
make: *** [tcp.o] Error 1

Thanks

  • Ezra Z.
    – Founder & Software Architect
    – removed_email_address@domain.invalid
    EngineYard.com

#8

Hi Erza,

I’m not sure if glib is installed by default on macs. (Seems like it
should be but I don’t see it on mine.) If you use fink you can do
apt-get install glib2-dev
The makefile is pretty hacky - make sure you have libev in your
LIBRARY_PATH and ev.h in your CPATH

ry


#9

An alternative might be github.com. It looks like they provide svn
mirrors of your git repos.

On Jan 14, 2008 1:50 PM, Luis L. removed_email_address@domain.invalid wrote:

Multimedia systems


Cheers,

Kevin W.
http://www.bantamtech.com/


#10

Ezra,

libev will be in MacPorts later tonight or tomorrow, and with glib2
installed, Ebb builds cleanly on Leopard.

best,
Brett

Ezra Z. wrote:

On Jan 14, 2008, at 8:43 AM, ry dahl wrote:

Hey Ry-

Is glib-2.0 a dependency? It seems like 2.0 is an old version of
glib. So ebb doesn’t compile on leopard out of the box. What do I need
to install to make it work?

Thanks

  • Ezra Z.
    – Founder & Software Architect
    – removed_email_address@domain.invalid
    EngineYard.com

#11

On Jan 14, 2008 11:43 AM, ry dahl removed_email_address@domain.invalid wrote:

are processed as follows:
request
It is very fun to program Ebb so I suggest you do too :slight_smile:

Ry Dahl

Ry,

Awesome, great idea!

I will definitely be checking this out hopefully this weekend

~Wayne


#12

On Jan 21, 2008 6:34 PM, ry dahl removed_email_address@domain.invalid wrote:

Hey All,

Just one more quick message to entice contributors to take a look at
Ebb. I was able to run some preliminary benchmarks for the first time
today against evented Mongrel and Thin. They’re all running a small
Camping application through Rack.

http://s3.amazonaws.com/four.livejournal/20080121/ebb.png

Hey ry, did you tweaked the HttpParser like Thin guy did?

I was thinking about the modifications he did:

He removed all the if conditions evaluating http_field,
request_method, request_uri and others (most the ones that are
allocated/initialized in HttpParser_alloc).

But since I couldn’t find time to ask it in a separate mail, is time
that boost the question using your thread :wink:

The code for that benchmark (and chart generation) can be found in the Ebb repo:
http://repo.or.cz/w/ebb.git?a=commit;h=c2fecde0a04603727949ec0b05d694be89a464d2

svn mirror is coming, right? :smiley:


Luis L.
Multimedia systems

A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams


#13

On Jan 21, 2008 1:55 PM, Luis L. removed_email_address@domain.invalid wrote:

Hey ry, did you tweaked the HttpParser like Thin guy did?

I was thinking about the modifications he did:

He removed all the if conditions evaluating http_field,
request_method, request_uri and others (most the ones that are
allocated/initialized in HttpParser_alloc).

I don’t think those conditions were removed, I think they were
moved, but I’ve only glanced at the C code and would barely know
what I was looking at anyway. :slight_smile:


Cheers,

Kevin W.
http://www.bantamtech.com/
http://www.almostserio.us/
http://kevwil.com/


#14

On Jan 21, 2008 7:00 PM, Kevin W. removed_email_address@domain.invalid wrote:

moved, but I’ve only glanced at the C code and would barely know
what I was looking at anyway. :slight_smile:

No, he removed them:

http://pastie.caboo.se/141633

The ragel code will generate the http11_parser.c file, and http11.c
file defines all the functions pointers:

http11.c:207…225:

VALUE HttpParser_alloc(VALUE klass)
{
VALUE obj;
http_parser *hp = ALLOC_N(http_parser, 1);
TRACE();
hp->http_field = http_field;
hp->request_method = request_method;
hp->request_uri = request_uri;
hp->fragment = fragment;
hp->request_path = request_path;
hp->query_string = query_string;
hp->http_version = http_version;
hp->header_done = header_done;
http_parser_init(hp);

obj = Data_Wrap_Struct(klass, NULL, HttpParser_free, hp);

return obj;
}

Theory indicates that allocation happens way before the ragel code is
getting executed, so there is no need to evaluate the conditions all
the time, ending with a speed boost.


Luis L.
Multimedia systems

A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams


#15

Hey All,

Just one more quick message to entice contributors to take a look at
Ebb. I was able to run some preliminary benchmarks for the first time
today against evented Mongrel and Thin. They’re all running a small
Camping application through Rack.

http://s3.amazonaws.com/four.livejournal/20080121/ebb.png

The code for that benchmark (and chart generation) can be found in the
Ebb repo:
http://repo.or.cz/w/ebb.git?a=commit;h=c2fecde0a04603727949ec0b05d694be89a464d2

ry


#16

yes I removed those conditions and looks like Ry did the same (by
looking quickly at his code).


#17

I didn’t know about those mods. I pulled the parser code from Thin.

ry


#18

svn mirror is coming, right? :smiley:

Hi Luis,

I’ll put it on github when they open (or another service?) Apparently
their svn bridge isn’t fully functional yet.

ry


#19

On Mon, 21 Jan 2008 21:34:22 +0100
“ry dahl” removed_email_address@domain.invalid wrote:

http://repo.or.cz/w/ebb.git?a=commit;h=c2fecde0a04603727949ec0b05d694be89a464d2
Grrrr, you need a Y axis Ry. What is that measuring. Hell, shoot me
the raw data and I’ll do the graph with R (since I’m curious).


Zed A. Shaw


#20

On Jan 21, 2008 7:46 PM, ry dahl removed_email_address@domain.invalid wrote:

svn mirror is coming, right? :smiley:

Hi Luis,

I’ll put it on github when they open (or another service?) Apparently
their svn bridge isn’t fully functional yet.

What you can do is ask for a rubyforge project and use the svn
available there as mirror.

I’m still waiting to get bzr-git with write support so I can use
bazaar all the way :slight_smile:


Luis L.
Multimedia systems

A common mistake that people make when trying to design
something completely foolproof is to underestimate
the ingenuity of complete fools.
Douglas Adams