Tracking the start and complete of a download?

How could I configure nginx so that when a download is started and
when it completes, it makes a HTTP GET request to another web server?

e.g.

  1. User requests http://nginx.com/download/file.zip?data=123456 (the
    filename and some additional getargs)
  2. nginx calls
    http://myotherserver.com/downloadStarted?url=/download/file.zip&data=123456
    (passes the original url with getargs)
  3. nginx serves byte-range requests of file.zip to user
  4. nginx determines that all byte-range requests have been served and
    calls
    http://myotherserver.com/downloadCompleted?url=/download/file.zip&data=123456
    (passes the original url with getargs)


Robert La Ferla
VP Engineering
OMS SafeHarbor

This message (and any attachments) contains confidential information and
is protected by law. If you are not the intended recipient, you should
delete this message and are hereby notified that any disclosure,
copying, distribution, or the taking of any action based on this
message, is strictly prohibited.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Il 15/10/2010 17:21, Robert La Ferla ha scritto:

How could I configure nginx so that when a download is started and when
it completes, it makes a HTTP GET request to another web server?

This is not possible.
You need to write a custom Nginx module.

Writing the module is not hard, the complex thing is the HTTP GET, since
it should be done so that Nginx main process is not blocked during
request.

However there are alternatives, as an example you can have Nginx write
to a FIFO in asynchonous mode, and have an external process read the
FIFO and do the work.

If FIFO is busy (external process is slow to read data), however, Nginx
will have to skip sending notification.

[…]

Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAky4v3sACgkQscQJ24LbaUS9ggCgmJacyMY8pqqRXlAgSay4SoP4
/PkAnirUh8+LuOW+kVG2EmxoEp2r1EOW
=gUxU
-----END PGP SIGNATURE-----

Hi,

On 15/10/2010 21:54, Manlio P. wrote:

How could I configure nginx so that when a download is started and when
it completes, it makes a HTTP GET request to another web server?
This is not possible.
You need to write a custom Nginx module.

I think you should be able to use the ngx_http_echo_module to achieve
this. Just use an echo_subrequest directive to send a request to the
URL that you want. This request will only be made once the upload has
completed. Any other module that calls a subrequest after the files
have been uploaded should work too.

One thing : if you don’t know the server that you’ll connect to at the
time the config file is read, you’ll need to set up a resolver for DNS.

Good luck!

Marcus.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Il 15/10/2010 23:12, Eugaia ha scritto:

this. Just use an echo_subrequest directive to send a request to the
URL that you want. This request will only be made once the upload has
completed. Any other module that calls a subrequest after the files
have been uploaded should work too.

One thing : if you don’t know the server that you’ll connect to at the
time the config file is read, you’ll need to set up a resolver for DNS.

Another problem is that the external server should be able to process
the same number of requests per second of the Nginx server.

This is the reason why, as an example, Nginx does not support syslog.

Regards Manlio
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAky5amkACgkQscQJ24LbaUSEcQCgh8mQf194T4Z8DxxZHFTNb4ER
od4AoInROdJTgitplHVab9jPYb7i/T00
=2oNX
-----END PGP SIGNATURE-----

Hi,

On 16/10/2010 10:03, Manlio P. wrote:

Another problem is that the external server should be able to process
the same number of requests per second of the Nginx server.

This is the reason why, as an example, Nginx does not support syslog.
Presumably if an upload tracker is required, the number of requests
relative to the volume of data transfer will be low (at least for this
type of operation on that server), so even if the other server isn’t as
perfomant as Nginx, I don’t think this should be a problem.

Robert could of course install Nginx on the other server as well, or
perhaps even just make a call to a FastCGI/WSGI process directly from
Nginx. :slight_smile:

Cheers,

Marcus.

On 10/16/2010 5:03 AM, Manlio P. wrote:

Another problem is that the external server should be able to process
the same number of requests per second of the Nginx server.

This is the reason why, as an example, Nginx does not support syslog.

Not sure why it would need to because the other server would handle a
mere fraction of the requests. In particular, nginx would be handling
thousands of byte-range requests per file downloaded. The other server
would just get 2 requests per file downloaded and in the case of an
incomplete download, only 1 request.


Robert La Ferla
VP Engineering
OMS SafeHarbor

This message (and any attachments) contains confidential information and
is protected by law. If you are not the intended recipient, you should
delete this message and are hereby notified that any disclosure,
copying, distribution, or the taking of any action based on this
message, is strictly prohibited.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs