Trying to download first 10k

Ruby
1

Hi, I’m trying to download via http only the first, say, 10k of a file.
I’m using open-uri but even when I try to break out of the core loop
where it’s reading the buffer, I can hear my hard-drive grinding away
downloading the entire 3MB. When I look at the file written to the disk
it shows only 10k, but judging by the length of time it takes to execute
the script and the grinding sound of my hard-drive, I think it’s
downloading the entire file.

I read the other posts about a progress bar, but that’s not really what
I’m trying to do… or is it?

Is there a way to interrupt the download? As you can see I tried to
break out of the code blocks every chance I got.

Here is my code:

def download_first_n_kilobytes n, file_name
n = 1024 * n
uri = file_name
count = 0
open(uri) do |fin|
open(convert_to_file_name(uri), ‘w’) do |fout|
while(buf = fin.read(n))
count += 1
fout.write buf
if count > 0
break
end # if
end # while
break
end # open
break
end # open
end # def

def convert_to_file_name uri
“mp3/” + File.basename(uri).to_s
end

uri = “http://www.sapht.com/music/tmp/sapht_-_alvaros_messenger.mp3”
download_first_n_kilobytes 10, uri

Thanks a lot for looking at this!

2

Oh yeah, at the top there should be:

require ‘open-uri’

3

On Dec 2, 2007 10:22 AM, David B. [email protected] wrote:

 open(convert_to_file_name(uri), 'w') do |fout|

end # open
end # def

def convert_to_file_name uri
“mp3/” + File.basename(uri).to_s
end

uri = “http://www.sapht.com/music/tmp/sapht_-_alvaros_messenger.mp3”
download_first_n_kilobytes 10, uri

Looking at the documentation for open-uri at
http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/, it seems to be
that it is default open-uri behavior to download the whole file and
just then give you control. You might succeed if you give a
:progress_proc that read()s and close()s if size>n, but I doubt it
would work.

Otherwise, use a more powerful way of downloading files, like
Net::HTTP, with something like
s = 0
http.request_get(‘/index.html’) {|res|
res.read_body do |segment|
s += segment.size
print segment
res.finish if s > n
end
}

See the docs at
http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/index.html

Aur

4

Quoth David B.:

 open(convert_to_file_name(uri), 'w') do |fout|

end # open
Thanks a lot for looking at this!
You’ll probably not want to be using open-uri but instead a custom HTTP
query
with a “Range: bytes=0-10239” header.

HTH,