How to chat


Seeing as how everyone who works out a Rails technique should blog about
(and seeing as how I decline to research how to build a gem, plugin, or
blog), this post will sketch a system I developed to chat.

If anyone likes this, please put it on your blog, so other blogs can
to it and increase its Google pagerank, etc.

If anyone /doesn’t/ like it, speak up now, or forever … chat about it

== Requirements ==

From the user’s viewpoint, a chat panel should work like this:

  • hit your page
  • all prior chat utterances appear in a scrolling DIV
  • user enters text into a long thin field
  • the text appears at the bottom of the DIV
  • the DIV scrolls down to display it
  • anyone else viewing that page sees it, too
  • you see their chats

That’s the core of chat. Here’s a naive implementation:

  • create a Chat database record and its model
  • put a DIV on your page
  • add a JavaScript DOM timer to your page
  • the timer sends AJAX “refresh” request
  • the server returns all the chat as HTML
  • the browser pushes the HTML into that DIV
  • put a form_remote_for :chat on your page
  • wire it to call an action, “utter” that adds
    a chat to the database
  • when the timer goes off, it “refreshes” this new chat

I hope to improve upon that implementation, but note that my
might be naive too. It will religiously observe the guideline “Premature
Optimization is the Root of all Hourly Billing Strategies”. I hope it is
least more accurate and responsive than some of the chat sample code I
downloaded. For example: It won’t top-post!

== Analysis ==

So let’s guess that sending a long chat history once per timer will be
This implies that each batch of chat should append to the end of the

Let’s also guess we should not set our timer to run faster than every
seconds. This implies each chat batch could contain more than one line.

We need a system to delimit a batch of chat, so the server can avoid
the same batch of chat over and over again. We must store a “high water
mark” variable, and only send chat that exceeds this value. And we
store it in a session variable, because they don’t scale. Session
are essentially scaffolding; to use in a pinch, before we figure out a
stateless solution.

So we add these lines to that algorithm:

  • add a hidden last_chat_id input field
  • the “refresh” timer sends the last_chat_id
  • the server fetches chat with id > last_chat_id
  • the “refresh” response contains the highest
    chat id
  • the browser pushes this into the hidden field

That system ass-u-me-s that our database will always increase and never
roll-over the variable. We could replace the variable with a
timestamp; the rest of the algorithm will work the same. Another
would be to occassionally “accidentally” erase the entire chat table and
reset its index ID. (That solution has the added benefit of insulating
from compliance with court orders requesting proof that someone once
something naughty!)

To install a “high water mark”, we need one variable’s value,
to travel thru these layers:

  • a hidden form field
  • the ActiveForm helper that builds AJAX
  • the JavaScript inside that helper
  • the wire
  • the server’s param[] bundle of joy
  • the ActiveRecord parameters
  • the SQL statement
  • the SQL’s data store
  • the returned recordset (the maximum one in this batch)
  • the returned HTML inside the wire
  • the AJAX JavaScript handler
  • our browser’s page’s DOM elements
  • something that pushes last_chat_id back into its hidden field!

Gods bless N-tier architectures, huh?

But focus on “something”. If our chat action returned raw HTML (such as
TABLE full of rows, each containing a chat utterance), then our AJAX
would push this into the DIV tag containing our chat history. Not into
FORM tag containing our hidden field.

This implies our HTML package should return a SCRIPT tag containing
JavaScript that updates that field to our hidden value.

That sounds like a plan, but here’s another problem.

Suppose our timer goes off, and our browser launches a “refresh”
Then the server slows down under load, and the timer gets bored and
another one. Each will have the same last_chat_id variable. Slowing down
timer is not an option, because if everyone starts chatting the server
slow down enough to encounter this problem.

A chat batch should come back to the browser containing enough
to censor duplicate lines. So this implies the chat batch is not HTML,
all JavaScript. It’s a series of addChat() calls, targetting a function
our application.js file with this signature:

addChat(panel_id, chat_id, fighter_name, utterance)

That will first check if our browser already contains a SPAN with an ID
to ‘chat_id’. If so, this is a duplicate chat, so just bounce out. (In a
stateless, high-latency system, sending a little more data than we need
always more efficient than risking sending a little less!)

If the chat utterance is not in the history DIV yet, push it in. (Code
will show how.)

That JavaScript function has one more variable, and explaining it will
finish our requirements.

== Productization ==

The ‘panel_id’ is a number indicating which chat panel we target. Our
application could have more than one chat - sometimes more than one on
page! So each DIV, FORM, and SPAN we have discussed will need an ID with
this number inside it, such as <div id=‘panel_<%= panel_id
%>_chat_history’…>, to make it unique. That’s so all our excessive
JavaScript can find the correct panel when it goes to harvest data out
plug data back in.

So henceforth, if I leave out the excessive <%= panel_id %> baggage, and
only write panel_1_, just imagine it’s there, because it’s easy to code.

== The System ==

So here’s the complete rig:

  • a chat model/table with

    • id
    • user_id → the usual user/login account system
    • panel_id → optional panel model/table
    • uttered_at datetime
    • utterance
  • the chat-endowed page has

    • ‘application.js’, containing
      addChat(panel_id, chat_id, fighter_name, utterance)
      scrollMe(panel_id) ← calls div.scrollTop = 0xffff;
      setLastChatId(panel_id, chat_id)
    • a DIV called ‘panel_1_chat_history’
    • a FORM to be called ‘panel_1_chat_form’
      with a big edit field and a submit button
      with a HIDDEN field called panel_1_last_chat_id
      and an ONSUBMIT with magic AJAXy stuff in it
      that targets :action => ‘utter’
      and updates this:
    • a SPAN called ‘panel_1_new_content_script’
    • a periodically_call_remote() that
      updates ‘panel_1_new_content_script’
      at :frequency => 5 seconds
      :with => ‘panel_1_last_chat_id’
      targets :action => ‘utter’
  • the chat-endowed page’s controller has (or imports*)

    • an action ‘utter’ which
      finds a chat in the params and saves it
      returns a SCRIPT containing
      JavaScript code that updates the ‘panel_1_chat_history’

(*Note there’s a slight gap between canonical MVC and Rails’s convenient
implementation. True MVC makes elements in each module pluggable, so
elements in Views can plug into elements in Controllers, independent of
other. Rails, by contrast, uses “Controller” to mean “thing that serves
logic for one coupled View”. [Unless I’m incredibly mistaken.] This is
a very good pattern - it enforces the most important separation; between
Layer concerns and Representation Layer concerns. Yet if this were true
then my partial chat View could plug into a truly decoupled Controller.
a Controller that couples with every page that contains chat panels. But

Now notice that the same action, ‘utter’, serves both saving chat and
refreshing the history. This is because we have two AJAX conversations,
we might as well use both to reflect the latest chat back to the user.
allows the user to see their own chat, much sooner than our 5 second
will allow. This preserves the illusion that other chats arrive

== Implementation ==

The following code snippets illustrate the stations on that cycle that
hard to code. The easy ones (like, uh, lots of RHTML, DDL, etc), are
left as
an exercise for the reader. The easy tests are left too (and, yes, I

Create a chat controller/view with one target action, ‘_index.rhtml’. Go
the your page that will host a chat panel, and add this line, wherever
want chat:

<%= render :partial => ‘chat/index’,
:locals => { :panel_id => 1 }

Make the :panel_id unique, somehow. If you only have a few chats, simply
hard-code each ones ID. Always remember: Magic Numbers are a perfectly
and valid coding practice if they are less than 10.

The first part of ‘_index.rhtml’ is the DIV that will contain our
chat history:

That sets the font size a little smaller than normal (a trick I learned

cough< Google Chat), and it sets the overflow style to automatic


goes over the wire empty, and your browser will populate it
the split second before it paints.

Next, ‘_index.rhtml’ starts the FORM that will submit new text:

@chat =
@chat.user_id =
@chat.panel_id = panel_id

form_remote_for( :chat,
:url=>{ :action=>:utter },
:html=>{ :id => ‘chat_panel_’+panel_id.to_s,
:style => ‘display:inline’ },
) do |f|

The form contains TABLE markup to stretch things out, and it finishes
these important fields:

    <%= f.text_field :utterance, :style=>'width:100%' %>

<%= f.hidden_field :user_id %>
<%= f.hidden_field :panel_id %>

      <input type='hidden'
          id='panel_<%= panel_id %>_last_chat_id'
        value='0' />

      <%= submit_tag 'Send' if @user %>
      <span id="loading" 


<% end %>

I had to explain why we are doing these things, up in the Analysis
so you will recognize the variables. Otherwise they would just look like
excessive markups. (Right?)

Notice we use the remote form’s :loading and :complete parameters to
the text “sending…”. I will eventually upgrade to a system that
twitches the page’s geometry, nor reveals the invisible word
“loading…” if
you drag-select under the Send button.

Another potential improvement: The controller will soon bounce any new
with an empty utterance. (We should not reward the user for banging
after writing nothing, or spaces!) However, I suspect we could bounce at
/this/ layer, if we add more JavaScript into the mix.

When that FORM submits, it triggers the ‘utter’ action:

def utter
return unless request.xhr?

if params[:chat]
  if validate_utterance(params[:chat])
    params[:uttered_at] =
    chat = Chat.create!(params[:chat])
    render(:nothing => true)  #  that's for wasting our time!

last_chat_id = params[:last_chat_id]

tex = Chat.get_chattage(
                  params[:panel_id] || params[:chat][:panel_id],
render(:text => tex)


That (mildly overloaded) action possibly creates a new chat (if the user
really had something to say), and it then returns any pending chats,
possibly including the recently added chat record.

Without the render(:nothing=>true) line, we would reward any user who
insistently. They would cause more server hits than others, and
would fetch chat more often, at the cost of tremendous overhead in their
browser and mild overhead in our server. So we don’t serve new chat to
anyone who did not write an utterance. They can wait for their timer.

(Contrarily, I should have set the limit on our input field. That will
prevent annoying error messages, flooding, etc.)

With only a few more code snippets to go, I will leave out the model
that anyone could write. Get your own model! We are nearing the end.

The faux-Controller method Chat.get_chattage() returns a snip of HTML
containing JavaScript, like this:

Notice I didn’t waste time adding //<![CDATA[ or whatever to mask that
script from naive browsers. If the user gets this far, they must have
JavaScript turned on! (I have also tested my code in Konqueror, Firefox,
other amateur browsers, and I suspect it’s generic…)

AJAX will push that SCRIPT into this SPAN:

<%= Chat.get_chattage(panel_id, 0) %>

The partial ‘_index.rhtml’ also calls get_chattage, so our page loads
with a
complete chat history. (Note that this call will take care of all the
pre-existing , with their correct IDs, in the history DIV, and
will take care of that sneaky ‘last_chat_id’ field! Nice, huh?

get_chattage() returns JavaScript that, one way or another, will call
functions, from ‘application.js’:

function scrollMe(panel_id){ // moves the history DIV to the bottom
var chat_history = ‘panel_’+panel_id+‘_chat_history’;
document.getElementById(chat_history).scrollTop = 0xffff;

var last_user_ids = {};
var last_toggles = {};

function addChat(panel_id, chat_id, fighter_name, utterance)
span_id = ‘panel_’+panel_id+‘chat’+chat_id;
if (document.getElementById(span_id)) return;
var chat_history = ‘panel_’+panel_id+‘_chat_history’;
div = document.getElementById(chat_history);
html = ‘<span id="’+span_id+'" width=“100%” ';
var toggle = last_toggles[panel_id];
toggle = !!toggle;

if (last_user_ids[panel_id] != fighter_name)
  toggle = !toggle;
  last_toggles[panel_id] = toggle;

var bgColor = toggle? 'white': '#eeeeee';
html += 


html += fighter_name;
html += ': ';
html += utterance;
html += ‘
div.innerHTML += html;
last_user_ids[panel_id] = fighter_name;

function setLastChatId(panel_id, id)
var last_chat_id =
last_chat_id.value = id;

If you have been following along, you recognize things like
‘setLastChatId()’. But the ‘bgColor’ and ‘toggle’ stuff is new. Briefly,
assigns the same background color to contiguous utterances by the same
and it alternates the color when alternate characters speak. So if you
2 chats, they both get Gray, for example. Then if your opponent chats,
get White. If someone else chats they get Grey, and if you chat again
now get White. So the system works to join utterances into apparent
paragraphs, and it always contrasts different speakers.

That elusive support function, Chat.get_chattage(), is straightforward
and ActiveRecord code, so I will just touch on its main concepts.
it needs to sanitize user input, and put it into a “string with quotes”.
does this without the help of h(), which I couldn’t figure out how to
(or how to Google for!):

def escapeHTML(str)
gsub(/&/, “&”).
gsub(/"/, “"”).
gsub(/>/, “>”).
gsub(/</, “<”)

def enstring(str) # CONSIDER move this to a helper?
return ‘"’ + escapeHTML(str) + ‘"’

get_chattage also needs to fetch its relevant chat. All the above
architecture has finally pulled all these variables together, right
we need them:

def self.get_chattage(panel_id, last_chat_id)

where = ['panel_id = ? and id > ?',
               panel_id, last_chat_id]

chats = Chat.find(:all, :conditions => where, :order => 


if [] != chats
  tex = '<script>'

  chats.each do |chat|
    tex += chat.to_javascript
    last_chat_id =  if last_chat_id <

  tex += 'setLastChatId('+panel_id.to_s+', ' + last_chat_id.to_s + 

tex += ‘scrollMe(’+panel_id.to_s+‘);’
return tex

return nil


If the system has new chats, we convert each one to a JavaScript
and stuff them all together with .to_javascript(). Finally, we detect
new “high water mark” - the new maximum chat_id, and we pass these to
browser. And we trigger the correct history DIV to scroll to the bottom.

Chat::to_javascript() looks a little bit like this:

def to_javascript
fighter_name = enstring(user.fighter_name)
utterance = enstring(self.utterance)
return “addChat(#{self.panel_id}, #{id}, #{fighter_name},

And that’s the bottom of the cycle. New chats follow this sequence:

  • a user enters them into the big edit field
  • AJAX sends them to the ‘utter’ action
  • ‘utter’ puts them into the database
  • the database sets their id to > last_chat_id
  • our find() detects them
  • we convert them into JavaScript
  • we AJAX them back to the browser
  • the browser evaluates their JavaScript
  • and sticks them into the history

== Testing ==

Every factoid in this post has a matching unit test. Really.

I will conclude the code snippets with an interesting test. Without
introduction, read it and try to see what it does:

require ‘rexml/document’
include REXML

class ChatTest < Test::Unit::TestCase
fixtures :chats, :users

def addChat(*foo)
assert_equal 2, foo[0]
@utterances << foo

def setLastChatId(panel_id, id)
assert_equal 2, panel_id
@last_chat_id = id

def scrollMe(panel_id)
assert_equal 2, panel_id
@scrollMe_was_called = true
end # TODO better name

def test_get_chattage
chattage = Chat.get_chattage(2, 0)

sauce = XPath.first(, '/script').text
@utterances = []
@last_chat_id = nil
@scrollMe_was_called = false

assert eval(sauce)  #  no jury would convict me

expect_utterances = [
  [2,  9, "userA", "chat utterance one"],
  [2, 10, "userB" , "chat utterance two"],
  [2, 11, "userA", "chat utterance three"] ]

assert_equal  expect_utterances, @utterances
assert_equal  11, @last_chat_id
assert @scrollMe_was_called


That test case uses a mock pattern called “self shunt”, where your own
case object works as your mock object.

We are mocking ‘application.js’. Look at our sample SCRIPT again:

The XPath stuff strips out the tags, leaving JavaScript that
suspiciously similar to Ruby source. Rather than wait for all the web
browser venders in the world to get off their butts and add Ruby to
script engines, we simply write only Ruby-style JavaScript.

Then we eval() it. That calls all our mock functions, and they trivially
report they were called correctly.

== TODO ==

Right now, nothing retires old chats. If the user returns to a page left
an open server for a couple months, they will (efficiently) see a couple
months’ worth of chat. Something needs to clean all that up.

The current design is very flexible. (I even flexed it a little as I
this post!:wink: We could, for example, decorate each user’s name in the
Decorate means to add user-specific links, icons, colors, etc. Because
have a working cycle, we can extend it by adding variables to the chat
records and the user records, then we insert them into addChat(), and
them in the DOM.

I suspect one could use “JSON” to render a record as JavaScript, but I
not researched it yet. It might make mocking our JavaScript hard!

I also suspect that there are libraries available to generate JavaScript
from Ruby generators. This implies one could test-first such JavaScript
querying those generators directly.

Lastly, I did not yet re-use this chat into more than one page. This
violates the advice I always give, that you should never add extra code
might work when you re-use. My excessive panel_id variable violated
rule! I might report back what happens when I add more chat panels, and
actually see what will happen when I stress that panel_id!

Phlip ← NOT a blog!!!

Slinky wrote:

was actually trying to do exactly this as part of my learning process.
The question how to copy data from the DOM thru JavaScript into Ajax is
FAQ, and should be in the FAQ that Faisal et al seem to be writing.

The answer is simple - :with => points to a Ruby string containing
JavaScript, and calls it each time it tries to do its thing. So put
the JavaScript anything you want to happen, and if it returns a return
value, the Ajax stuff with append this to the end of the URL it creates.

You must use :method => ‘get’ with :with, so the Ajax stuff will then
actually use the parameters in the URL it sends out.

I could be wrong, and this important system seems to be

Redirecting... ← NOT a blog!!!

This was an interesting email and addressed a problem I was having with
transporting values in a hidden field to the server for processing when
using periodically_call_remote

You seem to use the :with parameter with the function, though, and I
couldn’t seem to find evidence that this parameter is actually
supported by periodically_call_remote. What’s more, if it is
supported, does it function vaguely like the other functions that
include it, as in, running javascript contained within?

I’m having a very hard time getting local javascript browser values to
my ruby code. I’ve seen a lot of suggestions regarding using JSON, but
I can’t seem to find examples of how to actually package and send the
values via an argument in ruby. :frowning: Sorry to hijack your thread, but I
was actually trying to do exactly this as part of my learning process.

Phlip wrote:

actually use the parameters in the URL it sends out.

I could be wrong, and this important system seems to be under-documented.

Redirecting... ← NOT a blog!!!

Thanks so much for your quick response- I’m definitely on the right
track to getting this functional now, but real quick, if I return a
value through :with, what key in the parameter structure will the
returned value be attached to? ‘with’?

Okay, never mind- managed to figure it out. The return value of the
:with parameter gets added direction into the link taht gets sent as
the remote request (which is why it has to use GET). Ruby parses
arguments in GET and POST the same, so you don’t have to worry about it
too much, but you do have to return a correctly-formatted string for
the keys and arguments to be parsed into param hash values correctly.

Basically like this:


You do that and you’ll get a bunch of nice key/value pairs added to
param[], with all the values being strings. From there you can do
whatever! In case you’re wondering, here’s what I did:

In the head of index.rhtml:

	<%= javascript_include_tag "prototype" %>
	<%= javascript_include_tag "scriptaculous" %>
	<%= javascript_tag "function stuff()
					return 'stuff=234';

In the body:

	<%= periodically_call_remote( :url => { :action => 'add_item'},
		:newitem => false , :frequency => 5, :method => :get,
		:with => "stuff()")

I haven’t written the final, functional version, but this does
correctly read in the param values so that param[‘stuff’] = ‘234’.
Anything beyond that is just correctly building the string you need to
use so it has the appropriate values.

Can we add all this to the FAQ?

Slinky wrote:

return ‘stuff=234’;
I haven’t written the final, functional version, but this does
correctly read in the param values so that param[‘stuff’] = ‘234’.
Anything beyond that is just correctly building the string you need to
use so it has the appropriate values.

And figuring out how to work all their nested layers of escapes…

Redirecting... ← NOT a blog!!!