Fast proxy

Today i have a application that send mails and this applications need to
know when the user has opened the mail, its kind easy do this sending a
id in some image file requested from the user
mail(src=“www.sample.com/image&id=1212”).

But this need to be veryyyy fast.

Actually testing on my notebook, apache could serve some image 7000
times a second, using em-proxy poiting to the same apache server, i
could only get 2000 times a second, and i did nothing on em-proxy, just
get from one side and send to another.

Have any way to this faster?

Diego B. wrote:

Today i have a application that send mails and this applications need to
know when the user has opened the mail, its kind easy do this sending a
id in some image file requested from the user
mail(src=“www.sample.com/image&id=1212”).

But this need to be veryyyy fast.

Actually testing on my notebook, apache could serve some image 7000
times a second, using em-proxy poiting to the same apache server, i
could only get 2000 times a second, and i did nothing on em-proxy, just
get from one side and send to another.

Have any way to this faster?

forgot to say, and taking that id and inserting into redis reduce the
performance to 1500 req/s

On Thu, Aug 12, 2010 at 8:26 AM, Diego B.
[email protected] wrote:

Actually testing on my notebook, apache could serve some image 7000
times a second, using em-proxy poiting to the same apache server, i
could only get 2000 times a second, and i did nothing on em-proxy, just
get from one side and send to another.

Have any way to this faster?

forgot to say, and taking that id and inserting into redis reduce the
performance to 1500 req/s

One can proxy very, very fast with Ruby and EM. Fast enough that
messing with redis will, by a long shot, be your bottleneck.

I suggest not doing it that way if you need really high speeds.

Were it me, I’d write a simple server to itself return the image, and
I’d throw records of images served into a log. That lets you decouple
serving the image with recording it in redis, and it makes it trivial
to break that task up into multiple pieces, or take advantage of
off-peak times to catch up on data loading if your peak times actually
serves images faster than the data can be stuffed into redis.

Kirk H.

Kirk H. wrote:

Were it me, I’d write a simple server to itself return the image, and
I’d throw records of images served into a log.

Just parse the Apache logs.

Or, if you want it more real-time, you can use the pipe log capability
of Apache, so you can have a persistent Ruby process which receives the
logs on stdin. But then again, you might as well just receive the HTTP
in Ruby.

Note: you should be aware that most modern E-mail clients don’t open
remote images in E-mails, for exactly the reason that it gives away
information about them being read (*)

Also: if you send out one million E-mails, and all of them are opened
within a 24 hour period, and 20% of the users have old clients which
do open remote images, then you’d only have to handle an average of 2.3
hits per second. Hardly need the world’s fastest webserver for that.

Regards, Brian.

(*) And hence HTML E-mails with images usually now embed the images
within the message, e.g. RFC2557, RFC2392

Thanks all for the help.

Thanks for the tip Kirk, im gonna parse the log, its easier and faster,
the applications isnt real time, i have time to process the log. The log
files gonna rotate every hour, gonna process one file per hour.

I also find something usefull, x-sendfile, i didint know that, this was
the reason i was getting so “slow” req/s using ruby, i think using this
with sinatra i could get near req/s as apache.

Brian,

About that, yes, its true, most mail block external images. But is the
only way i can know if the user has read or not the mail.
And the list is send to people who wants the mail, they trust in the
mail content.
Have any other way to track the mails?

Again, thanks.

Diego B.

Diego B. wrote:

About that, yes, its true, most mail block external images. But is the
only way i can know if the user has read or not the mail.
And the list is send to people who wants the mail, they trust in the
mail content.
Have any other way to track the mails?

Not really. Some mailers will honour requests for return receipts, but
they will pop up a dialog first asking the user if they’re happy to send
it back.

The best I can think of is that you could put the content they’re
looking for on a web page, then just E-mail them a short message with
the link. e.g. “Click here to read this month’s newsletter”. Include a
unique cookie in the link, e.g. as a query parameter. Then you can tell
if they clicked through or not.

The real question is, what problem are you trying to solve by
determining whether the user opened the mail or not?

If the problem is a commercial one (e.g. your customer wants to pay you
based on the number of E-mails opened, not the number of E-mails
delivered) then you could use surveys or statistical samples to estimate
what proportion of their E-mails is opened. In other words, a bit like
TV viewer figures.

B.

Brian C. wrote:

Diego B. wrote:

About that, yes, its true, most mail block external images. But is the
only way i can know if the user has read or not the mail.
And the list is send to people who wants the mail, they trust in the
mail content.
Have any other way to track the mails?

Not really. Some mailers will honour requests for return receipts, but
they will pop up a dialog first asking the user if they’re happy to send
it back.

The best I can think of is that you could put the content they’re
looking for on a web page, then just E-mail them a short message with
the link. e.g. “Click here to read this month’s newsletter”. Include a
unique cookie in the link, e.g. as a query parameter. Then you can tell
if they clicked through or not.

The real question is, what problem are you trying to solve by
determining whether the user opened the mail or not?

If the problem is a commercial one (e.g. your customer wants to pay you
based on the number of E-mails opened, not the number of E-mails
delivered) then you could use surveys or statistical samples to estimate
what proportion of their E-mails is opened. In other words, a bit like
TV viewer figures.

B.

The issue is, this system deal with marketing mail, know if the user has
open mail is useful to my client for a lot of reasons, like, later this
numbers turn into reports so they can discuss a new strategy and so.

And yea its a good idea to direct the user to a website using some query
string to track.

Diego.