Streaming to/from MySQL


When I upload a 10Mb file (from my browser) to my Rails app, a Tempfile
object gets created on the server. When the contents gets written to the
database, it’s a simple INSERT, so basically the data is read to memory
and then inserted into MySQL.

How come the data is not streamed to the DB so we avoid the memory
issue? Is this a short coming of ActiveRecord or the MySQL/Ruby driver
or something else?

I guess that it’s not possible to stream from the database either? (At
least the API doesn’t appear to support this).

Any insight much appreciated.


Simple solution: Don’t store files in a db. Streaming files to the db
is not part of the SQL standard so we would have to support adapter
specific extensions to get this done.
Storing files in the db is never a good idea however so there
shouldn’t be a problem that this corner case isn’t well served by the

Your file system is already a database for files. Its several lorders
of magnitude more efficient then a database can ever hope to be.

On 4/29/06, Morten [email protected] wrote:

or something else?
[email protected]

Tobi - modern e-commerce software - Open source weblog engine - Technical weblog

Thanks for the clarification.

I disagree that adhering to the SQL standard should be a show stopper
for supporting streaming, and suspect that it would be in place if it
was relatively easy to support.

I store files in the DB for backup simplicity and explicit referential
integrity, but I do serve cached versions to the user off the FS.



Tobias Lütke wrote:

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs