Re: sending personalized emails efficiently

Rails
1

Hi Tom,

I noticed this message from you saying that you can use ActionMailer
to efficiently do high volume mailing systems. I need to create
something that mails out once a day to all my users status updates.
But I must admit I don’t see how you can make ActionMailer do all
these efficiences you state since it’s API is so limited.
Specifically I don’t see how to NOT connect/disconnect from the SMTP
server for each message, parallel delivery scheme, etc.

Your help would be appreciated.

Thanks,
Frank

Original Message

On Oct 20, 2005, at 1:46 AM, Gerret A. wrote:

I’m working on a project where the non-techy site administrator wants
the ability to send template-based emails to all his registered users
(on the order of thousands), meaning that each of the emails generated
is unique and needs its own SMTP envelope.

If I were going to do something like this I would avoid actionmailer as it seems to not like to send that many emails very fast. I would probably fork a ruby script that uses Tmail or Rhizmail to build the mime messages and the use threads in that process to send the mails with net/smtp. You could also look at drb to run a ruby dbr server that would just wait for a request from your rails app and then have that take care of the mail build and send process. It would probably work even better if you had a mail server on the same machine and you could push the emails into the mailer daemon's queue. The queue of the smtp daemon is really built to queue up many emails like that so the more you can push down into the mail daemon the better off you will be.

ActionMailer is a wrapper around Tmail, which is included in the source.

Why would you expect it to be slow?

I’ve done some high volume mailing systems before (not spam, but event
notification!) and the solution to sending a lot of emails is:

  1. Avoid serial latencies, which will kill you. Do NOT
    connect/disconnect
    from the SMTP server for each message.
  2. Avoid serial latencies, which will kill you. :slight_smile: Devise a scheme to
    have
    parallel delivery.
  3. Avoid serial latencies, which will kill you. :slight_smile: :slight_smile: Do not deliver
    directly
    to recipients mail servers, which makes 1 & 2 impossible.
  4. Avoid serial latencies, which will kill you. :slight_smile: :slight_smile: :slight_smile: Do implement
    a local
    mail server (separate machine is ideal if you have very high traffic)
    and
    use the fastest mail server available (which may not be sendmail).

It should be noted that if you configure your mail server in a way that
it
is only accessible internally, then you can pretty much shut off all
spam
measures (RBL, etc) which introduce nasty serial latencies. This means
that you should likely NOT use an existing mail server accessible to the
outside world.

–
– Tom M.