Forum: Ruby on Rails StringIO to Tempfile?

A0f6b57661bc57b6520010a2a333faba?d=identicon&s=25 R. Elliott Mason (eleo)
on 2007-05-01 02:15
I recently discovered that uploaded files under ~20KB are used to create
StringIO objects instead of Tempfile objects.  This has a negative
effect in my models:

class Image < Upload
  def get_file_info
    self[:width], self[:height] = `identify -format "%w %h" #{file.path
+ "[0]"}`.split
  end
end

This doesn't work as StringIO objects have no path, whereas Tempfile
objects do.  So using ImageMagick I am unable to manipulate the uploaded
image as a StringIO due to the fact that it isn't on the filesystem and
ImageMagick takes a filename.  ImageMagick (as far as I know) cannot
operate on a blob, so whenever a user uploads a small image everything
falls apart.

What I -could- do is write the file to the filesystem first and then get
its width/height but this would be a sloppier solution.  Since I use
height and width values for validation then I would have to: save the
file, then validate it, and if it fails validation manually delete that
file.

With larger files the upload is stored temporarily as a Tempfile, so it
is (if I'm not mistaken) automatically deleted if the validation fails
and the object is not saved.  In this case I am afforded the convenience
of writing to the filesystem permanently only after_create.

So I'm wondering is there a way to create a Tempfile from a StringIO?  I
have looked at the docs and been unable to find out how to create a
Tempfile from a blob.
Ce01db9bec66e5796cad9fe202acf8e1?d=identicon&s=25 Chris Martin (Guest)
on 2007-05-01 03:57
(Received via mailing list)
On 4/30/07, R. Elliott Mason <rails-mailing-list@andreas-s.net> wrote:
> end
> height and width values for validation then I would have to: save the
> Tempfile from a blob.
Personally, I'm not sure that's the best way to go about it.
I'd just use the rmagick interface.
http://rmagick.rubyforge.org/

With that you can get the width/height/type much easier. See these
(slightly verbose) examples.
http://textsnippets.com/posts/show/543
http://www.rubyonrailsblog.com/articles/2006/09/08...

In particular, you could use something like

file.rewind
img = Magick::Image.from_blob(file.read)[0]

Then width & height are available in img.columns and img.rows

--

Chris Martin
Web Developer
Open Source & Web Standards Advocate
http://www.chriscodes.com/
821395fe70906c8290df7f18ac4ac6cf?d=identicon&s=25 Rick Olson (Guest)
on 2007-05-01 03:58
(Received via mailing list)
> So I'm wondering is there a way to create a Tempfile from a StringIO?  I
> have looked at the docs and been unable to find out how to create a
> Tempfile from a blob.

It works like File.

TempFile.new do |f|
  f.write file.to_s
end

--
Rick Olson
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com
A0f6b57661bc57b6520010a2a333faba?d=identicon&s=25 R. Elliott Mason (eleo)
on 2007-05-01 06:09
Doesn't seem to be working?  Gives me a File object that is 0 bytes.  I
can open the file on my and it will be blank.

def get_file_info
  @file = Tempfile.new(@file.original_filename) {|f|
f.write(@file.read)}
  self[:width], self[:height] = `identify -format "%w %h" #{@file.path +
"[0]"}`.split
end

Width/height end up as nil.  :(

Doesn't seem to work from the console either.  I'm guessing I could
write an arbitrary string to a Tempfile like:

file = Tempfile.new('test') {|f| f.write('123456789')}

But in this case file.read gives me "" and file.length gives me 0.

I'd use RMagick, but it's overkill.  I don't need any complex
processing, just the width/height of the image.  In my experience
RMagick adds a lot of overhead.
Ef3aa7f7e577ea8cd620462724ddf73b?d=identicon&s=25 Rob Biedenharn (Guest)
on 2007-05-01 18:32
(Received via mailing list)
On May 1, 2007, at 12:09 AM, R. Elliott Mason wrote:

> I'd use RMagick, but it's overkill.  I don't need any complex
> processing, just the width/height of the image.  In my experience
> RMagick adds a lot of overhead.

Well, I intend this to make it onto RubyForge one day, but since you
have a need ;-)  Drop this into your RAILS_ROOT/lib and Enjoy!  If
you want the test/unit/image_size_test.rb, send me an offline email.

-Rob


# image_size.rb
#
# Copyright (c) 2007 Rob Biedenharn
#   Rob [at] AgileConsultingLLC.com
#   Rob_Biedenharn [at] alum.mit.edu
#
# Permission is hereby granted, free of charge, to any person
obtaining a copy
# of this software and associated documentation files (the
"Software"), to
# deal in the Software without restriction, including without
limitation the
# rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or
# sell copies of the Software, and to permit persons to whom the
Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE
# SOFTWARE.

require 'stringio'              # strictly for the ::of_blob

# Given a duck-typed +IO+ object (which could be from +StringIO+,
+File+,
# +open-uri+, etc.) containing image data, attempt to grab the width and
# height.  As a convenience, the size is defined as "#{width}x#{height}"
#
# The description of JPEG and GIF files was taken from the
# <tt>/usr/share/file/magic</tt> file from Mac OS X.  The PNG
description was
# validated against the spec at
# http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html
class ImageSize

   # Prepare a new IO source.  Should support IO#pos=, IO#read, and
IO#gets.
   #
   # Supported formats:
   # * JPEG
   # * PNG
   # * GIF
   def initialize source
     @image = source
     @header = nil
     @width = @height = nil
   end

   # A convenience method to pull size information from a file
   def self.of_file path
     obj = nil
     File.open(path) do |f|
       obj = new(f)
       obj.size
     end
     obj
   end

   # A convenience method to pull size information from a blob (i.e.,
a +String+)
   def self.of_blob blob
     obj = nil
     sio = ::StringIO.new(blob)
     obj = self.new(sio)
     obj.size
     obj
   end

   def size
     [self.width,self.height] * 'x'
   end

   [ :width, :height ].each do |property|
     meth = "def #{property}; @#{property} ||= case\n"
     [ :jpeg, :png, :gif ].each do |format|
       meth << " when #{format}? : #{format}_#{property}; "
     end
     meth << " else -1; end; end"
     class_eval meth
   end

   protected
   # Generic header takes bytes from the front.  Once a type is known, a
   # format-specific header is retrieved from the data.
   def header
     @header unless @header.nil?
     @image.pos=0
     @image.read(100)
   end

   # ==================================================
   # # GIF
   # 0  string    GIF8    GIF image data
   # >4  string    7a    \b, version 8%s,
   # >4  string    9a    \b, version 8%s,
   def gif?
     header[0...6] =~ /GIF8[79]a/
   end
   # >6  leshort    >0    %hd x
   # >8  leshort    >0    %hd
   def gif_width
     gif_header[6...8].unpack('v').first
   end
   def gif_height
     gif_header[8...10].unpack('v').first
   end

   def gif_header
     @header ||= read_gif_header
   end

   def read_gif_header
     @image.pos=0
     @image.read(10)
   end

   # ==================================================
   # PNG
   # # 137 P N G \r \n ^Z \n [4-byte length] H E A D [HEAD data]
[HEAD crc] ...
   # #
   # 0  string    \x89PNG    PNG image data,
   # >4  belong    !0x0d0a1a0a  CORRUPTED,
   # >4  belong    0x0d0a1a0a
   def png?
     header[0...8] == "\x89PNG\r\n\x1a\n"
   end
   # >>16  belong    x    %ld x
   # >>20  belong    x    %ld,
   #
   # and from the spec: http://www.libpng.org/pub/png/spec/1.2/PNG-
Chunks.html
   #
   # 4.1.1. IHDR Image header
   #
   # The IHDR chunk must appear FIRST. It contains:
   #
   #    Width:              4 bytes
   #    Height:             4 bytes
   #    Bit depth:          1 byte
   #    Color type:         1 byte
   #    Compression method: 1 byte
   #    Filter method:      1 byte
   #    Interlace method:   1 byte
   def png_width
     png_header[16...20].unpack('N').first
   end
   def png_height
     png_header[20...24].unpack('N').first
   end

   def png_header
     @header ||= read_png_header
   end

   def read_png_header
     @image.pos=0
     @image.read(37)             # through the end of the IHDR chunk
   end

   # ==================================================
   # JPEG
   def jpeg?
     # 0  beshort    0xffd8    JPEG image data
     header[0...2] == "\xff\xd8" # ffd8 is the SOI - Start of Image
marker
   end

   def jpeg_header
     @header ||= read_jpeg_header
   end

   # The JFIF header is nasty in the sense that the size marker isn't
in a
   # fixed place.  Build the header up in pieces by reading the
markers and
   # data sizes until an "interesting" segment is found.
   JPEG_SEG = /\xff[\xc0\xc1\xc2]/nm
   def read_jpeg_header
     @image.pos=0
     buffer = ''

     buffer << @image.read(4)    # ffd8 ff__
     found_segment = false       # the first one has to be ffe_ which
is an
                                 # APPx segment and not one that
would match
                                 # our interesting JPEG_SEG pattern
     while buffer[-2,2] != "\xff\xd9" # ffd9 is the EOI - End of
Image marker
       buffer << @image.read(2)
       datasize = buffer[-2,2].unpack('n').first
       # datasize includes the 2 bytes for itself so this gets the data
       # (datasize-2 bytes) and the next marker (2 bytes), too
       buffer << @image.read(datasize)
       break if found_segment
       found_segment = buffer[-2,2] =~ JPEG_SEG
     end
     buffer
   end

   # Or, we can show the encoding type (I've included only the three
most common)
   # and image dimensions if we are lucky and the SOFn (image
segment) is here:
   # >(4.S+5)  byte    0xC0    \b, baseline
   # >>(4.S+6)  byte    x    \b, precision %d
   # >>(4.S+7)  beshort    x    \b, %dx
   # >>(4.S+9)  beshort    x    \b%d
   # >(4.S+5)  byte    0xC1    \b, extended sequential
   # >>(4.S+6)  byte    x    \b, precision %d
   # >>(4.S+7)  beshort    x    \b, %dx
   # >>(4.S+9)  beshort    x    \b%d
   # >(4.S+5)  byte    0xC2    \b, progressive
   # >>(4.S+6)  byte    x    \b, precision %d
   # >>(4.S+7)  beshort    x    \b, %dx
   # >>(4.S+9)  beshort    x    \b%d

   DIM_RE = Regexp.new(%{#{JPEG_SEG}...(..)(..)}, Regexp::MULTILINE,
'n')

   def jpeg_width
     begin
       jpeg_header[DIM_RE, 2].unpack('n').first if jpeg?
     rescue => e
       raise ArgumentError, "Looking for #{DIM_RE} in JPEG data:\n#
{jpeg_header.unpack('H*').first.scan(/.{2,64}/).join("\n")}\n"
     end
   end
   def jpeg_height
     begin
       jpeg_header[DIM_RE, 1].unpack('n').first if jpeg?
     rescue => e
       raise ArgumentError, "Looking for #{DIM_RE} in JPEG data:\n#
{jpeg_header.unpack('H*').first.scan(/.{2,64}/).join("\n")}\n"
     end
   end

end

__END__

Rob Biedenharn    http://agileconsultingllc.com
Rob@AgileConsultingLLC.com
A0f6b57661bc57b6520010a2a333faba?d=identicon&s=25 R. Elliott Mason (eleo)
on 2007-05-02 04:00
Code got kinda mangled by this forum's forced linebreaks.  Any chance
you could http://pastie.caboo.se this or something along those lines?
Looks great, it's just that comments got cut off unto newlines.  Thanks.
Ef3aa7f7e577ea8cd620462724ddf73b?d=identicon&s=25 Rob Biedenharn (Guest)
on 2007-05-02 16:17
(Received via mailing list)
On May 1, 2007, at 10:00 PM, R. Elliott Mason wrote:

>
> Code got kinda mangled by this forum's forced linebreaks.  Any chance
> you could http://pastie.caboo.se this or something along those lines?
> Looks great, it's just that comments got cut off unto newlines.
> Thanks.
>

http://pastie.caboo.se/58251

One of these days I need to put everything like this into an
anonymously accessible repository, but this will do for now.

-Rob

Rob Biedenharn    http://agileconsultingllc.com
Rob@AgileConsultingLLC.com
A0f6b57661bc57b6520010a2a333faba?d=identicon&s=25 R. Elliott Mason (eleo)
on 2007-05-03 05:39
Thanks.  Perhaps you don't want to be technical support, but I'm having
trouble getting this to work with jpg files:

can't convert nil into String

#{RAILS_ROOT}/lib/image_size.rb:182:in `<<'
#{RAILS_ROOT}/lib/image_size.rb:182:in `read_jpeg_header'
#{RAILS_ROOT}/lib/image_size.rb:166:in `jpeg_header'
#{RAILS_ROOT}/lib/image_size.rb:214:in `jpeg_width'
(eval):2:in `width'
#{RAILS_ROOT}/lib/image_size.rb:69:in `size'
#{RAILS_ROOT}/lib/image_size.rb:54:in `of_file'
#{RAILS_ROOT}/lib/image_size.rb:52:in `open'
#{RAILS_ROOT}/lib/image_size.rb:52:in `of_file'
#{RAILS_ROOT}/app/models/image.rb:25:in `get_file_info'
#{RAILS_ROOT}/app/controllers/imageboard_controller.rb:70:in `new_post'

It seems to do fine with jpg blobs, but not actual files.

I haven't experienced this with gifs or pngs.  It's not an immediate
problem because for File objects I can call ImageMagick and use your
class for StringIO's.  Just though I'd let you know.
Ef3aa7f7e577ea8cd620462724ddf73b?d=identicon&s=25 Rob Biedenharn (Guest)
on 2007-05-03 14:56
(Received via mailing list)
On May 2, 2007, at 11:39 PM, R. Elliott Mason wrote:
> (eval):2:in `width'
> I haven't experienced this with gifs or pngs.  It's not an immediate
> problem because for File objects I can call ImageMagick and use your
> class for StringIO's.  Just though I'd let you know.

Can you send me an offline email (i.e., not through the mailing list
or forum) that has both the JPEG file and the actual code you're
using?  I suspect that there's just a quirk in your JPEG file (I ran
into a few oddities myself) that needs a new unit test to uncover.
It looks like the end-of-file is being reached without finding the
segment that contains size information.

-Rob

Rob Biedenharn    http://agileconsultingllc.com
Rob@AgileConsultingLLC.com
Skype:  rob.biedenharn
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.