Forum: Ruby on Rails Storing images in Ruby

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Ef31136ad29b9ce06c9f718109a584cc?d=identicon&s=25 Pete (Guest)
on 2006-03-24 11:04
Hi,

I'd like to allow users to upload images and store those images in a
Postgres database. I've looked through the example in "aguile
development with rails" which suggests using the MySQL blob field.

When I wrote a similar app in Java, we Base64 encoded the file and then
stored it as a text field. Is this easy to do in RoR too? Any advice you
can give me would be greatly appreciated :)
Bf66e10c8fc4abefebde0425e7f6f15a?d=identicon&s=25 Norman Timmler (Guest)
on 2006-03-24 12:18
(Received via mailing list)
Am Freitag, den 24.03.2006, 11:04 +0100 schrieb Pete:
> Hi,
>
> I'd like to allow users to upload images and store those images in a
> Postgres database. I've looked through the example in "aguile
> development with rails" which suggests using the MySQL blob field.
>
> When I wrote a similar app in Java, we Base64 encoded the file and then
> stored it as a text field. Is this easy to do in RoR too? Any advice you
> can give me would be greatly appreciated :)

Why not store a file in the filesystem. That's easy. If you want, store
it in the database as an base64 encoded text. That would be only a few
lines of code. I think the procedure is the same as in java, only the
language differs. Personally i never caught the sense of storing a file
in a database.

--
Norman Timmler

http://blog.inlet-media.de
6ef8cb7cd7cd58077f0b57e4fa49a969?d=identicon&s=25 Brian Hogan (Guest)
on 2006-03-24 17:08
(Received via mailing list)
file_column plugin is awesome for storing files on the file system. In
my
opinion, that's the best route to go if you're going to be serving these
images via a web server. That way, the web server can deal with them
instead
of having to invoke your Rails application just to pull a file out of
the
database.

The only reason I can see a need to store the images in the database is,
like sessions, you need to scale across many servers. If that's the
case, I
believe there are some wonderful entries on storing binary files into
the
database on the wiki.
75ceef25624b3e44943a3b7319ba9612?d=identicon&s=25 Chris Schumann (torgo)
on 2006-03-24 17:10
(Received via mailing list)
Another reason for storing in the database is that you get a backup of
your whole system just by backing up the database. No need to include a
list of other directories.
14b7344777a56ac9a1fad849f5d6da2a?d=identicon&s=25 sayoyo Sayoyo (sayoyo)
on 2006-03-24 17:12
(Received via mailing list)
I have a similar need, how can we store a file on the
file system with rails?

Saiho

--- Brian Hogan <bphogan@gmail.com> wrote:

> The only reason I can see a need to store the images
> >
> > >
> > lines of code. I think the procedure is the same
> > _______________________________________________
> > Rails mailing list
> > Rails@lists.rubyonrails.org
> >
> http://lists.rubyonrails.org/mailman/listinfo/rails
> >
> > _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>


The mind is its own place, and in itself.
Can make a Heaven of Hell, a Hell of Heaven.

http://www.geocities.com/sayoyo/

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
3e9b9641aa790eac2fcb94f8068d95da?d=identicon&s=25 Nis Sarup (Guest)
on 2006-03-24 17:16
(Received via mailing list)
The first hit when you search for upload in the RubyonRails Wiki is
this:
http://wiki.rubyonrails.com/rails/pages/HowtoUploadFiles

I have been using the info from this article for several apps.

--Nis.
B8d11b4c89cfc6e34d6c9fb121c9acac?d=identicon&s=25 Claus Guttesen (Guest)
on 2006-03-24 17:44
(Received via mailing list)
> > file_column plugin is awesome for storing files on the file system. In my
> > opinion, that's the best route to go if you're going to be serving these
> > images via a web server. That way, the web server can deal with them instead
> > of having to invoke your Rails application just to pull a file out of the
> > database.
> >
> > The only reason I can see a need to store the images in the database is,
> > like sessions, you need to scale across many servers. If that's the case, I
> > believe there are some wonderful entries on storing binary files into the
> > database on the wiki.

Share the filesystem via nfs or some other means. Storing images in
your db doesn't scale. Make a reference in your db to your file
instead.

regards
Claus
970efdae7d4dd77370743065457b17fe?d=identicon&s=25 Steven Mohapi-Banks (Guest)
on 2006-03-24 17:49
(Received via mailing list)
I think storing images on the filesystem is definitely the way to go.
And if you need to scale at a later date use something like MogileFS
(http://www.danga.com/mogilefs/) to handle replication across machines.

Steve
Bf66e10c8fc4abefebde0425e7f6f15a?d=identicon&s=25 Norman Timmler (Guest)
on 2006-03-24 17:51
(Received via mailing list)
Am Freitag, den 24.03.2006, 17:44 +0100 schrieb Claus Guttesen:
>
> Share the filesystem via nfs or some other means. Storing images in
> your db doesn't scale. Make a reference in your db to your file
> instead.

+1. Would you store 100,000 1MB files in a DB? A filesystem is made for
handling files. It's fast and reliable. DB's are made for organizing and
finding data, not for storing large binaries.

--
Norman Timmler

http://blog.inlet-media.de
6ef8cb7cd7cd58077f0b57e4fa49a969?d=identicon&s=25 Brian Hogan (Guest)
on 2006-03-24 18:01
(Received via mailing list)
Oh trust me, I totally agree. I've written quite a few apps that do this
kind of thing and have *never* stored files in my database. I don't
think
"it's easier to backup" is really that big of an issue.... that sounds
like
lazy dba talk to me :)

(I'm an Oracle dba, btw)

Like I said... the "only reason" I can think of is the scaling issue...
and
even that's not a good one.
B8d11b4c89cfc6e34d6c9fb121c9acac?d=identicon&s=25 Claus Guttesen (Guest)
on 2006-03-24 18:08
(Received via mailing list)
> > Share the filesystem via nfs or some other means. Storing images in
> > your db doesn't scale. Make a reference in your db to your file
> > instead.
>
> +1. Would you store 100,000 1MB files in a DB?

No I wouldn't do that.

> A filesystem is made for
> handling files. It's fast and reliable. DB's are made for organizing and
> finding data, not for storing large binaries.

When I wrote "share the filesystem" I meant (maby not so obvious :-)
that one keeps the images in files, which is then shared by nfs or
some other means. *Only* keep a pointer in the db which then
references the file (located on a filesystem).

Storing images in the db does *not* scale, so if the site does have
ambitions several hundreds GB's reaching into the TB's in the db, is
*not* the way to do it.

regards
Claus
Bf66e10c8fc4abefebde0425e7f6f15a?d=identicon&s=25 Norman Timmler (Guest)
on 2006-03-24 18:22
(Received via mailing list)
Am Freitag, den 24.03.2006, 18:07 +0100 schrieb Claus Guttesen:
> > finding data, not for storing large binaries.
>
> When I wrote "share the filesystem" I meant (maby not so obvious :-)
> that one keeps the images in files, which is then shared by nfs or
> some other means. *Only* keep a pointer in the db which then
> references the file (located on a filesystem).
>
> Storing images in the db does *not* scale, so if the site does have
> ambitions several hundreds GB's reaching into the TB's in the db, is
> *not* the way to do it.

Sorry, you misunderstood me! I totally agree with you! The question was
directed to Chris Schumann. NFS is also my prefered way.

--
Norman Timmler

http://blog.inlet-media.de
59de94a56fd2c198f33d9515d1c05961?d=identicon&s=25 Tom Mornini (Guest)
on 2006-03-24 18:25
(Received via mailing list)
On Mar 24, 2006, at 8:51 AM, Norman Timmler wrote:

>>>>
>> instead.
>
> +1. Would you store 100,000 1MB files in a DB? A filesystem is made
> for
> handling files. It's fast and reliable. DB's are made for
> organizing and
> finding data, not for storing large binaries.

What precludes large binaries from being data?

And, as often as it's said, people should wonder why every database
has a
BLOB type column!

I guess someone should alert the database coders and vendors that they
should discontinue BLOB support in the future, since that's not what
databases should be used for.

:-)

I don't entirely disagree with the position, but I don't entirely agree
with it either.

--
-- Tom Mornini
75ceef25624b3e44943a3b7319ba9612?d=identicon&s=25 Chris Schumann (torgo)
on 2006-03-24 18:49
(Received via mailing list)
Norman Timmler wrote:
>> When I wrote "share the filesystem" I meant (maby not so obvious :-)
>
What question? Nothing quoted above was from me.

I wrote that one could use a single backup point as a reason to put
images in a database. Someone else wrote it could scale. I have no idea,
and wouldn't do it either. I just wrote it as a response to someone who
wrote that he couldn't think of another reason to do it.

Chris Schumann
58479f76374a3ba3c69b9804163f39f4?d=identicon&s=25 Eric Hodel (Guest)
on 2006-03-24 19:16
(Received via mailing list)
On Mar 24, 2006, at 8:45 AM, Steven Mohapi-Banks wrote:

>> can give me would be greatly appreciated :)
>
> I think storing images on the filesystem is definitely the way to
> go. And if you need to scale at a later date use something like
> MogileFS (http://www.danga.com/mogilefs/) to handle replication
> across machines.

This is what we did for 43 Things.

Stored images on an NFS-mounted directory until the disk got nearly
full, then switched to MogileFS.

So now there's a Ruby MogileFS client:

http://blog.segment7.net/articles/2006/03/22/mogil...

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
Bf66e10c8fc4abefebde0425e7f6f15a?d=identicon&s=25 Norman Timmler (Guest)
on 2006-03-24 19:52
(Received via mailing list)
Am Freitag, den 24.03.2006, 11:47 -0600 schrieb Chris Schumann:
> What question? Nothing quoted above was from me.
>
> I wrote that one could use a single backup point as a reason to put
> images in a database. Someone else wrote it could scale. I have no
> idea,
> and wouldn't do it either. I just wrote it as a response to someone
> who
> wrote that he couldn't think of another reason to do it.

Sorry for the confusion. I missed to quote you.


"Another reason for storing in the database is that you get a backup of
your whole system just by backing up the database. No need to include a
list of other directories."


I thought you would suggest this way as an answer to the initial
question of Pete, what was:


"I'd like to allow users to upload images and store those images in a
Postgres database. I've looked through the example in "aguile
development with rails" which suggests using the MySQL blob field.

When I wrote a similar app in Java, we Base64 encoded the file and then
stored it as a text field. Is this easy to do in RoR too? Any advice
you
can give me would be greatly appreciated :)"


I did not realize the diferent context.
--
Norman Timmler

http://blog.inlet-media.de
F68e4842b0a5487678e26046fdb2fdf3?d=identicon&s=25 Frank (Guest)
on 2006-03-24 21:02
(Received via mailing list)
You may also want to check out this very interesting discussion we had
at Sheeri's blog about the same topic, i.e. storing  images (36
million) in a database.

  http://sheeri.com/archives/39

  Includes comments from James Day of Wikipedia.

  I second Brian in that you shouldn't do this just because "it's easier
to backup".

  Frank

  Brian Hogan <bphogan@gmail.com> wrote:Oh  trust me, I totally agree.
I've written quite a few apps that do this  kind of thing and have
*never* stored files in my database. I don't  think "it's easier to
backup" is really that big of an issue.... that  sounds like lazy dba
talk to me :)

(I'm an Oracle dba, btw)

Like I said... the "only reason" I can think of is the scaling issue...
and even that's not a good one.

On 3/24/06,   Norman Timmler <lists@inlet-media.de> wrote:




Want to read my Pro Rails book or Pro Server Management book? Become a
beta reader now. Write me to learn more.

Rails Blog: http://railsruby.blogspot.com
MySQL Blog: http://mysqldatabaseadministration.blogspot.com
Linux / Security Blog: http://frankmash.blogspot.com
Programming One Liners: http://programming-oneliners.blogspot.com
B8d11b4c89cfc6e34d6c9fb121c9acac?d=identicon&s=25 Claus Guttesen (Guest)
on 2006-03-24 22:57
(Received via mailing list)
> You may also want to check out this very interesting discussion we had at
> Sheeri's blog about the same topic, i.e. storing  images (36 million) in a
> database.
>
>  http://sheeri.com/archives/39

Quoting from the blog:

"My company's site has over 2 million images to store/retrieve, and
our current setup is at its limit and doesn't scale well ? 2 NFS
servers. We'd rather avoid adding another NFS server, ..."

You don't add another nfs-server, you implement a solution where you
upgrade your current nfs-server with more disk-storage and add that
storage to some sort of storage-pool.

Storing and retrieving 2 mill. images is also about planning your
layout, either by date or incrementing a counter or by other means
dividing your images into sub-dirs. You may have two nfs-server, but
that's usually to provide some sort of fail-over.

But this is becoming off-topic and ought to be continued on a db- or
serverperformance-related list instead.

regards
Claus
F68e4842b0a5487678e26046fdb2fdf3?d=identicon&s=25 Frank (Guest)
on 2006-03-25 02:14
(Received via mailing list)
What I don't get is how quickly discussions get labeled as off-topic.
Using Rails does involves server / db issues. Why not let discussions
run their course to accumulate the specific practices being used by the
Rails community.

  Just my two cents.

  Frank

Claus Guttesen <kometen@gmail.com> wrote:

But this is becoming off-topic and ought to be continued on a db- or
serverperformance-related list instead.

regards
Claus



Want to read my Pro Rails book or Pro Server Management book? Become a
beta reader now. Write me to learn more.

Rails Blog: http://railsruby.blogspot.com
MySQL Blog: http://mysqldatabaseadministration.blogspot.com
Linux / Security Blog: http://frankmash.blogspot.com
Programming One Liners: http://programming-oneliners.blogspot.com
366472c969c9754ae2b8079e42fed1ca?d=identicon&s=25 Jon Maddox (Guest)
on 2006-03-29 09:26
(Received via mailing list)
I just solved this problem myself.

I used the file_column plugin and let it upload to its default /
RAILS_ROOT/public/upload directory.

I think the folder was named after my model. While in development, i
had subversion ignore this directory so that when i deployed, it
wouldn't be there. I then created a symlink to a directory under my
"shared" directory outside of my app.
This way when i redeploy, users aren't going to lose their uploads.

/current -> /releases/200629030912
/releases
/releases/200629030912
/releases/200629030912/public/upload -> /shared/upload
/shared
/shared/upload


I hope that wasn't too confusing.
This topic is locked and can not be replied to.