Continue an interrupted download

Ruby
#1

Hi,

a while back I have pushed a gem “get_mp3” which can be used to download
selected radio-broadcasts from two french radio-stations to the local
file-system.

My use of the HTTP-connection I have learned from this thread on
stackoverflow:
https://stackoverflow.com/questions/2263540/how-do-i-download-a-binary-file-over-http/6964173#6964173

Currently, I try to add a download-history to the program which shall
permit a user to finish a discontinued download or to queue downloads
for later execution. For this, I already persist asynchronously the
current position of the file-cursor, file-name and download-url.

What I am missing, is a way to skip the bytes, which are already stored
in the local file and to begin downloading at an advanced
cursor-position, so that eventually the file is completed without having
to overwrite the existing chunks…

You can see all the relevant code in the current get_mp3 gem (on
rubygems.org) or in this pastebin, which mirrors the current
development-state of the main routines:

http://pastebin.com/RCEXFnvA
The answer that I seek should lead to a modified method http_to_file and
a pre-setting of the variable last_pos.

I hope, that this is enough background for you and that I have explained
clearly what I am after.

Thanks in advance,

Michael

#2

I respond myself.

To continue an interrupted download, the http-request must be endowed
with a ‘Range’ header and a byte-range indicating the portion of the
file which must still be transferred. This requires that the original
procedure (see pastebin) be modified and the request be prepared
before a call to Net::HTTP.request(request). The example reads 1M of
data, starting right after the first mega-byte.

req = Net::HTTP::Get.new(uri.path)
req[‘Range’] = ‘bytes=1000000-2000000’;

The line from the original code
http.request_get(uri.path) do |res|
must be replaced by
http.get(req) do |res|

Two additional manipulations are needed to ensure
1.) that the http-server supports resuming file-transfers (by means of a
HEAD request with Range)
2.) that the remote file had not been modified or replaced in the
meantime, e.g. by means of a HEAD request with If-Modified-Since and a
check on the response.

Thank you. Bye.