Streaming to/from MySQL

Hi.

When I upload a 10Mb file (from my browser) to my Rails app, a Tempfile
object gets created on the server. When the contents gets written to the
database, it’s a simple INSERT, so basically the data is read to memory
and then inserted into MySQL.

How come the data is not streamed to the DB so we avoid the memory
issue? Is this a short coming of ActiveRecord or the MySQL/Ruby driver
or something else?

I guess that it’s not possible to stream from the database either? (At
least the API doesn’t appear to support this).

Any insight much appreciated.

Morten

Simple solution: Don’t store files in a db. Streaming files to the db
is not part of the SQL standard so we would have to support adapter
specific extensions to get this done.
Storing files in the db is never a good idea however so there
shouldn’t be a problem that this corner case isn’t well served by the
adapter.

Your file system is already a database for files. Its several lorders
of magnitude more efficient then a database can ever hope to be.

On 4/29/06, Morten [email protected] wrote:

or something else?
[email protected]
http://lists.rubyonrails.org/mailman/listinfo/rails


Tobi
http://shopify.com - modern e-commerce software
http://typo.leetsoft.com - Open source weblog engine
http://blog.leetsoft.com - Technical weblog

Thanks for the clarification.

I disagree that adhering to the SQL standard should be a show stopper
for supporting streaming, and suspect that it would be in place if it
was relatively easy to support.

I store files in the DB for backup simplicity and explicit referential
integrity, but I do serve cached versions to the user off the FS.

Br,

Morten

Tobias Lütke wrote: