# How to chat

**URL:** <https://www.ruby-forum.com/t/how-to-chat/75990>\
**Category:** Rails\
**Created:** [November 14, 2006, 10:56pm UTC](https://www.ruby-forum.com/t/how-to-chat/75990 "2006-11-14T22:56:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [November 14, 2006, 10:56pm UTC](https://www.ruby-forum.com/t/how-to-chat/75990/1 "2006-11-14T22:56:13Z")

</div>

Railsers:

Seeing as how everyone who works out a Rails technique should blog about  
it,  
(and seeing as how I decline to research how to build a gem, plugin, or  
blog), this post will sketch a system I developed to chat.

If anyone likes this, please put it on your blog, so other blogs can  
point  
to it and increase its Google pagerank, etc.

If anyone /doesn’t/ like it, speak up now, or forever … chat about it  
later!

== Requirements ==

From the user’s viewpoint, a chat panel should work like this:

- hit your page
- all prior chat utterances appear in a scrolling DIV
- user enters text into a long thin field
- the text appears at the bottom of the DIV
- the DIV scrolls down to display it
- anyone else viewing that page sees it, too
- you see their chats

That’s the core of chat. Here’s a naive implementation:

- create a Chat database record and its model
- put a DIV on your page
- add a JavaScript DOM timer to your page
- the timer sends AJAX “refresh” request
- the server returns all the chat as HTML
- the browser pushes the HTML into that DIV
- put a form\_remote\_for :chat on your page
- wire it to call an action, “utter” that adds  
a chat to the database
- when the timer goes off, it “refreshes” this new chat

I hope to improve upon that implementation, but note that my  
implementation  
might be naive too. It will religiously observe the guideline “Premature  
Optimization is the Root of all Hourly Billing Strategies”. I hope it is  
at  
least more accurate and responsive than some of the chat sample code I  
downloaded. For example: It won’t top-post!

== Analysis ==

So let’s guess that sending a long chat history once per timer will be  
slow.  
This implies that each batch of chat should append to the end of the  
history  
DIV.

Let’s also guess we should not set our timer to run faster than every  
couple  
seconds. This implies each chat batch could contain more than one line.

We need a system to delimit a batch of chat, so the server can avoid  
sending  
the same batch of chat over and over again. We must store a “high water  
mark” variable, and only send chat that exceeds this value. And we  
shouldn’t  
store it in a session variable, because they don’t scale. Session  
variables  
are essentially scaffolding; to use in a pinch, before we figure out a  
truly  
stateless solution.

So we add these lines to that algorithm:

- add a hidden last\_chat\_id input field
- the “refresh” timer sends the last\_chat\_id
- the server fetches chat with id \> last\_chat\_id
- the “refresh” response contains the highest  
chat id
- the browser pushes this into the hidden field

That system ass-u-me-s that our database will always increase and never  
roll-over the chat.id variable. We could replace the variable with a  
timestamp; the rest of the algorithm will work the same. Another  
solution  
would be to occassionally “accidentally” erase the entire chat table and  
reset its index ID. (That solution has the added benefit of insulating  
us  
from compliance with court orders requesting proof that someone once  
chatted  
something naughty!)

To install a “high water mark”, we need one variable’s value,  
last\_chat\_id,  
to travel thru these layers:

- a hidden form field
- the ActiveForm helper that builds AJAX
- the JavaScript inside that helper
- the wire
- the server’s param[] bundle of joy
- the ActiveRecord parameters
- the SQL statement
- the SQL’s data store
- the returned recordset (the maximum one in this batch)
- the returned HTML inside the wire
- the AJAX JavaScript handler
- our browser’s page’s DOM elements
- something that pushes last\_chat\_id back into its hidden field!

Gods bless N-tier architectures, huh?

But focus on “something”. If our chat action returned raw HTML (such as  
a  
TABLE full of rows, each containing a chat utterance), then our AJAX  
handler  
would push this into the DIV tag containing our chat history. Not into  
the  
FORM tag containing our hidden field.

This implies our HTML package should return a SCRIPT tag containing  
JavaScript that updates that field to our hidden value.

That sounds like a plan, but here’s another problem.

Suppose our timer goes off, and our browser launches a “refresh”  
message.  
Then the server slows down under load, and the timer gets bored and  
launches  
another one. Each will have the same last\_chat\_id variable. Slowing down  
our  
timer is not an option, because if everyone starts chatting the server  
could  
slow down enough to encounter this problem.

A chat batch should come back to the browser containing enough  
information  
to censor duplicate lines. So this implies the chat batch is not HTML,  
it’s  
all JavaScript. It’s a series of addChat() calls, targetting a function  
in  
our application.js file with this signature:

addChat(panel\_id, chat\_id, fighter\_name, utterance)

That will first check if our browser already contains a SPAN with an ID  
set  
to ‘chat\_id’. If so, this is a duplicate chat, so just bounce out. (In a  
stateless, high-latency system, sending a little more data than we need  
is  
always more efficient than risking sending a little less!)

If the chat utterance is not in the history DIV yet, push it in. (Code  
below  
will show how.)

That JavaScript function has one more variable, and explaining it will  
finish our requirements.

== Productization ==

The ‘panel\_id’ is a number indicating which chat panel we target. Our  
application could have more than one chat - sometimes more than one on  
each  
page! So each DIV, FORM, and SPAN we have discussed will need an ID with  
this number inside it, such as \<div id=‘panel\_\<%= panel\_id  
%\>\_chat\_history’…\>, to make it unique. That’s so all our excessive  
JavaScript can find the correct panel when it goes to harvest data out  
or  
plug data back in.

So henceforth, if I leave out the excessive \<%= panel\_id %\> baggage, and  
only write panel\_1\_, just imagine it’s there, because it’s easy to code.  
Relatively.

== The System ==

So here’s the complete rig:

- a chat model/table with

- the chat-endowed page has

- the chat-endowed page’s controller has (or imports\*)

(\*Note there’s a slight gap between canonical MVC and Rails’s convenient  
implementation. True MVC makes elements in each module pluggable, so  
elements in Views can plug into elements in Controllers, independent of  
each  
other. Rails, by contrast, uses “Controller” to mean “thing that serves  
logic for one coupled View”. [Unless I’m incredibly mistaken.] This is  
still  
a very good pattern - it enforces the most important separation; between  
GUI  
Layer concerns and Representation Layer concerns. Yet if this were true  
MVC,  
then my partial chat View could plug into a truly decoupled Controller.  
Not  
a Controller that couples with every page that contains chat panels. But  
I  
digress…)

Now notice that the same action, ‘utter’, serves both saving chat and  
refreshing the history. This is because we have two AJAX conversations,  
and  
we might as well use both to reflect the latest chat back to the user.  
That  
allows the user to see their own chat, much sooner than our 5 second  
timer  
will allow. This preserves the illusion that other chats arrive  
promptly,  
too.

== Implementation ==

The following code snippets illustrate the stations on that cycle that  
are  
hard to code. The easy ones (like, uh, lots of RHTML, DDL, etc), are  
left as  
an exercise for the reader. The easy tests are left too (and, yes, I  
have  
them!).

Create a chat controller/view with one target action, ‘\_index.rhtml’. Go  
to  
the your page that will host a chat panel, and add this line, wherever  
you  
want chat:

\<%= render :partial =\> ‘chat/index’,  
:locals =\> { :panel\_id =\> 1 }  
%\>

Make the :panel\_id unique, somehow. If you only have a few chats, simply  
hard-code each ones ID. Always remember: Magic Numbers are a perfectly  
safe  
and valid coding practice if they are less than 10.

The first part of ‘\_index.rhtml’ is the DIV that will contain our  
scrolling  
chat history:

That sets the font size a little smaller than normal (a trick I learned  
from

> cough\< Google Chat), and it sets the overflow style to automatic  
> scroll  
> bars.

The

 goes over the wire empty, and your browser will populate it  
during  
the split second before it paints.

Next, ‘\_index.rhtml’ starts the FORM that will submit new text:

\<%  
@chat = Chat.new()  
@chat.user\_id = @user.id  
@chat.panel\_id = panel\_id

form\_remote\_for( :chat,  
:url=\>{ :action=\>:utter },  
:html=\>{ :id =\> ‘chat\_panel\_’+panel\_id.to\_s,  
:style =\> ‘display:inline’ },  
:update=\>‘panel\_’+panel\_id.to\_s+‘\_new\_content\_script’,  
:loading=\>load,  
:complete=\>comp,  
:after=\>“chat\_utterance.value=‘’;”  
) do |f|  
%\>

The form contains TABLE markup to stretch things out, and it finishes  
with  
these important fields:

```
    <%= f.text_field :utterance, :style=>'width:100%' %>

```

…  
\<%= f.hidden\_field :user\_id %\>  
\<%= f.hidden\_field :panel\_id %\>

```
      <input type='hidden'
          id='panel_<%= panel_id %>_last_chat_id'
        name='last_chat_id'
        value='0' />

      <%= submit_tag 'Send' if @user %>
      <span id="loading" 

```

style=“background-color:white;color:white;”\>  
sending…  
  
…  
\<% end %\>

I had to explain why we are doing these things, up in the Analysis  
section,  
so you will recognize the variables. Otherwise they would just look like  
excessive markups. (Right?)

Notice we use the remote form’s :loading and :complete parameters to  
flash  
the text “sending…”. I will eventually upgrade to a system that  
neither  
twitches the page’s geometry, nor reveals the invisible word  
“loading…” if  
you drag-select under the Send button.

Another potential improvement: The controller will soon bounce any new  
chat  
with an empty utterance. (We should not reward the user for banging  
Enter  
after writing nothing, or spaces!) However, I suspect we could bounce at  
/this/ layer, if we add more JavaScript into the mix.

When that FORM submits, it triggers the ‘utter’ action:

def utter  
return unless request.xhr?

```
if params[:chat]
  if validate_utterance(params[:chat])
    params[:uttered_at] = Time.now
    chat = Chat.create!(params[:chat])
  else
    render(:nothing => true) # that's for wasting our time!
    return
  end
end

last_chat_id = params[:last_chat_id]

tex = Chat.get_chattage(
                  params[:panel_id] || params[:chat][:panel_id],
                  last_chat_id.to_i
                  )
render(:text => tex)

```

end

That (mildly overloaded) action possibly creates a new chat (if the user  
really had something to say), and it then returns any pending chats,  
possibly including the recently added chat record.

Without the render(:nothing=\>true) line, we would reward any user who  
banged  
 insistently. They would cause more server hits than others, and  
would fetch chat more often, at the cost of tremendous overhead in their  
browser and mild overhead in our server. So we don’t serve new chat to  
anyone who did not write an utterance. They can wait for their timer.

(Contrarily, I should have set the limit on our input field. That will  
also  
prevent annoying error messages, flooding, etc.)

With only a few more code snippets to go, I will leave out the model  
stuff  
that anyone could write. Get your own model! We are nearing the end.

The faux-Controller method Chat.get\_chattage() returns a snip of HTML  
containing JavaScript, like this:

Notice I didn’t waste time adding //\<![CDATA[ or whatever to mask that  
script from naive browsers. If the user gets this far, they must have  
JavaScript turned on! (I have also tested my code in Konqueror, Firefox,  
and  
other amateur browsers, and I suspect it’s generic…)

AJAX will push that SCRIPT into this SPAN:

 \<%= Chat.get\_chattage(panel\_id, 0) %\>

The partial ‘\_index.rhtml’ also calls get\_chattage, so our page loads  
with a  
complete chat history. (Note that this call will take care of all the  
pre-existing , with their correct IDs, in the history DIV, _and_  
it  
will take care of that sneaky ‘last\_chat\_id’ field! Nice, huh?

get\_chattage() returns JavaScript that, one way or another, will call  
these  
functions, from ‘application.js’:

function scrollMe(panel\_id){ // moves the history DIV to the bottom  
var chat\_history = ‘panel\_’+panel\_id+‘\_chat\_history’;  
document.getElementById(chat\_history).scrollTop = 0xffff;  
}

var last\_user\_ids = {};  
var last\_toggles = {};

function addChat(panel\_id, chat\_id, fighter\_name, utterance)  
{  
span\_id = ‘panel\_’+panel\_id+‘_chat_’+chat\_id;  
if (document.getElementById(span\_id)) return;  
var chat\_history = ‘panel\_’+panel\_id+‘\_chat\_history’;  
div = document.getElementById(chat\_history);  
html = ‘\<span id="’+span\_id+'" width=“100%” ';  
var toggle = last\_toggles[panel\_id];  
toggle = !!toggle;

```
if (last_user_ids[panel_id] != fighter_name)
  {
  toggle = !toggle;
  last_toggles[panel_id] = toggle;
  }

var bgColor = toggle? 'white': '#eeeeee';
html += 

```

‘style="width:100%;color:black;background-color:’+bgColor+';"

> ';  
> html += fighter\_name;  
> html += ': ';  
> html += utterance;  
> html += ‘  
> ’;  
> div.innerHTML += html;  
> last\_user\_ids[panel\_id] = fighter\_name;  
> }

function setLastChatId(panel\_id, id)  
{  
var last\_chat\_id =  
document.getElementById(‘panel\_’+panel\_id+‘\_last\_chat\_id’);  
last\_chat\_id.value = id;  
}

If you have been following along, you recognize things like  
‘setLastChatId()’. But the ‘bgColor’ and ‘toggle’ stuff is new. Briefly,  
it  
assigns the same background color to contiguous utterances by the same  
user,  
and it alternates the color when alternate characters speak. So if you  
send  
2 chats, they both get Gray, for example. Then if your opponent chats,  
they  
get White. If someone else chats they get Grey, and if you chat again  
you  
now get White. So the system works to join utterances into apparent  
paragraphs, and it always contrasts different speakers.

That elusive support function, Chat.get\_chattage(), is straightforward  
Rails  
and ActiveRecord code, so I will just touch on its main concepts.  
Firstly,  
it needs to sanitize user input, and put it into a “string with quotes”.  
It  
does this without the help of h(), which I couldn’t figure out how to  
import  
(or how to Google for!):

def escapeHTML(str)  
return(str.  
gsub(/&/, “&”).  
gsub(/"/, “"”).  
gsub(/\>/, “\>”).  
gsub(/\</, “\<”)  
)  
end

def enstring(str) # CONSIDER move this to a helper?  
return ‘"’ + escapeHTML(str) + ‘"’  
end

get\_chattage also needs to fetch its relevant chat. All the above  
architecture has _finally_ pulled all these variables together, right  
where  
we need them:

def self.get\_chattage(panel\_id, last\_chat\_id)

```
where = ['panel_id = ? and id > ?',
               panel_id, last_chat_id]

chats = Chat.find(:all, :conditions => where, :order => 

```

‘uttered\_at’)

```
if [] != chats
  tex = '<script>'

  chats.each do |chat|
    tex += chat.to_javascript
    last_chat_id = chat.id if last_chat_id < chat.id
  end

  tex += 'setLastChatId('+panel_id.to_s+', ' + last_chat_id.to_s + 

```

‘);’  
tex += ‘scrollMe(’+panel\_id.to\_s+‘);’  
return tex  
end

```
return nil

```

end

If the system has new chats, we convert each one to a JavaScript  
statement,  
and stuff them all together with .to\_javascript(). Finally, we detect  
the  
new “high water mark” - the new maximum chat\_id, and we pass these to  
the  
browser. And we trigger the correct history DIV to scroll to the bottom.

Chat::to\_javascript() looks a little bit like this:

def to\_javascript  
fighter\_name = enstring(user.fighter\_name)  
utterance = enstring(self.utterance)  
return “addChat(#{self.panel\_id}, #{id}, #{fighter\_name},  
#{utterance});”  
end

And that’s the bottom of the cycle. New chats follow this sequence:

- a user enters them into the big edit field
- AJAX sends them to the ‘utter’ action
- ‘utter’ puts them into the database
- the database sets their id to \> last\_chat\_id
- our find() detects them
- we convert them into JavaScript
- we AJAX them back to the browser
- the browser evaluates their JavaScript
- and sticks them into the history

== Testing ==

Every factoid in this post has a matching unit test. Really.

I will conclude the code snippets with an interesting test. Without  
introduction, read it and try to see what it does:

require ‘rexml/document’  
include REXML

class ChatTest \< Test::Unit::TestCase  
fixtures :chats, :users

def addChat(\*foo)  
assert\_equal 2, foo[0]  
@utterances \<\< foo  
end

def setLastChatId(panel\_id, id)  
assert\_equal 2, panel\_id  
@last\_chat\_id = id  
end

def scrollMe(panel\_id)  
assert\_equal 2, panel\_id  
@scrollMe\_was\_called = true  
end # TODO better name

def test\_get\_chattage  
chattage = Chat.get\_chattage(2, 0)

```
sauce = XPath.first(Document.new(chattage), '/script').text
@utterances = []
@last_chat_id = nil
@scrollMe_was_called = false

assert eval(sauce) # no jury would convict me

expect_utterances = [
  [2, 9, "userA", "chat utterance one"],
  [2, 10, "userB" , "chat utterance two"],
  [2, 11, "userA", "chat utterance three"] ]

assert_equal expect_utterances, @utterances
assert_equal 11, @last_chat_id
assert @scrollMe_was_called

```

end  
end

That test case uses a mock pattern called “self shunt”, where your own  
test  
case object works as your mock object.

We are mocking ‘application.js’. Look at our sample SCRIPT again:

The XPath stuff strips out the tags, leaving JavaScript that  
looks  
suspiciously similar to Ruby source. Rather than wait for all the web  
browser venders in the world to get off their butts and add Ruby to  
their  
script engines, we simply write only Ruby-style JavaScript.

Then we eval() it. That calls all our mock functions, and they trivially  
report they were called correctly.

== TODO ==

Right now, nothing retires old chats. If the user returns to a page left  
on  
an open server for a couple months, they will (efficiently) see a couple  
months’ worth of chat. Something needs to clean all that up.

The current design is very flexible. (I even flexed it a little as I  
wrote  
this post!😉 We could, for example, decorate each user’s name in the  
list.  
Decorate means to add user-specific links, icons, colors, etc. Because  
we  
have a working cycle, we can extend it by adding variables to the chat  
records and the user records, then we insert them into addChat(), and  
render  
them in the DOM.

I suspect one could use “JSON” to render a record as JavaScript, but I  
have  
not researched it yet. It might make mocking our JavaScript hard!

I also suspect that there are libraries available to generate JavaScript  
from Ruby generators. This implies one could test-first such JavaScript  
by  
querying those generators directly.

Lastly, I did not yet re-use this chat into more than one page. This  
violates the advice I always give, that you should never add extra code  
that  
_might_ work when you re-use. My excessive panel\_id variable violated  
that  
rule! I might report back what happens when I add more chat panels, and  
I  
actually see what will happen when I stress that panel\_id!

–  
Phlip  
[http://www.greencheese.us/ZeekLand](http://www.greencheese.us/ZeekLand) ← NOT a blog!!!

---

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [December 12, 2006, 2:16am UTC](https://www.ruby-forum.com/t/how-to-chat/75990/2 "2006-12-12T02:16:57Z")

</div>

Slinky wrote:

> was actually trying to do exactly this as part of my learning process.  
> The question how to copy data from the DOM thru JavaScript into Ajax is  
> a  
> FAQ, and should be in the FAQ that Faisal et al seem to be writing.

The answer is simple - :with =\> points to a Ruby string containing  
JavaScript, and calls it each time it tries to do its thing. So put  
inside  
the JavaScript anything you want to happen, and if it returns a return  
value, the Ajax stuff with append this to the end of the URL it creates.

You must use :method =\> ‘get’ with :with, so the Ajax stuff will then  
actually use the parameters in the URL it sends out.

I could be wrong, and this important system seems to be  
under-documented.

–  
Phlip  
[Redirecting...](http://www.greencheese.us/ZeekLand) ← NOT a blog!!!

---

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [December 12, 2006, 1:34am UTC](https://www.ruby-forum.com/t/how-to-chat/75990/3 "2006-12-12T01:34:50Z")

</div>

This was an interesting email and addressed a problem I was having with  
transporting values in a hidden field to the server for processing when  
using periodically\_call\_remote

You seem to use the :with parameter with the function, though, and I  
couldn’t seem to find evidence that this parameter is actually  
supported by periodically\_call\_remote. What’s more, if it is  
supported, does it function vaguely like the other functions that  
include it, as in, running javascript contained within?

I’m having a very hard time getting local javascript browser values to  
my ruby code. I’ve seen a lot of suggestions regarding using JSON, but  
I can’t seem to find examples of how to actually package and send the  
values via an argument in ruby. ☹ Sorry to hijack your thread, but I  
was actually trying to do exactly this as part of my learning process.

---

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [December 12, 2006, 4:21am UTC](https://www.ruby-forum.com/t/how-to-chat/75990/4 "2006-12-12T04:21:08Z")

</div>

Phlip wrote:

> actually use the parameters in the URL it sends out.
> 
> I could be wrong, and this important system seems to be under-documented.
> 
> –  
> Phlip  
> [Redirecting...](http://www.greencheese.us/ZeekLand) ← NOT a blog!!!

Thanks so much for your quick response- I’m definitely on the right  
track to getting this functional now, but real quick, if I return a  
value through :with, what key in the parameter structure will the  
returned value be attached to? ‘with’?

---

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [December 12, 2006, 6:09am UTC](https://www.ruby-forum.com/t/how-to-chat/75990/5 "2006-12-12T06:09:18Z")

</div>

Okay, never mind- managed to figure it out. The return value of the  
:with parameter gets added direction into the link taht gets sent as  
the remote request (which is why it has to use GET). Ruby parses  
arguments in GET and POST the same, so you don’t have to worry about it  
too much, but you do have to return a correctly-formatted string for  
the keys and arguments to be parsed into param hash values correctly.

Basically like this:

key1=value1&key2=value2

You do that and you’ll get a bunch of nice key/value pairs added to  
param[], with all the values being strings. From there you can do  
whatever! In case you’re wondering, here’s what I did:

In the head of index.rhtml:

```
	<%= javascript_include_tag "prototype" %>
	<%= javascript_include_tag "scriptaculous" %>
	<%= javascript_tag "function stuff()
				{
					return 'stuff=234';
				}"
	%>

```

In the body:

```
	<%= periodically_call_remote( :url => { :action => 'add_item'},
		:newitem => false , :frequency => 5, :method => :get,
		:with => "stuff()")
	%>

```

I haven’t written the final, functional version, but this does  
correctly read in the param values so that param[‘stuff’] = ‘234’.  
Anything beyond that is just correctly building the string you need to  
use so it has the appropriate values.

---

<div class="post-metadata">

**Author:** ![Phlip](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/ed8c4c/32.png) [@Phlip](https://www.ruby-forum.com/u/Phlip)\
**Post date:** [December 12, 2006, 6:53am UTC](https://www.ruby-forum.com/t/how-to-chat/75990/6 "2006-12-12T06:53:25Z")

</div>

Can we add all this to the FAQ?

Slinky wrote:

> return ‘stuff=234’;  
> I haven’t written the final, functional version, but this does  
> correctly read in the param values so that param[‘stuff’] = ‘234’.  
> Anything beyond that is just correctly building the string you need to  
> use so it has the appropriate values.

And figuring out how to work all their nested layers of escapes…

–  
Phlip  
[Redirecting...](http://www.greencheese.us/ZeekLand) ← NOT a blog!!!