Writing a script to download videos

Ruby
1

I’m trying to write a script to download my edX course videos for me.
This
is what I’ve written but it doesn’t seem to be working. can someone help
me
out please?

require ‘open-uri’
require ‘rubygems’

File.open(“video.mp4”, “w+”) do |file|

open("
Authn | edX")
do |read_file|
file.write(read_file.read)
end
end

–
Himel S.,
himelsarkarforever.blogspot.com
http://himelsarkarforever.blogspot.com

  • Here’s to the crazy ones. The misfits. The rebels. The troublemakers.
    The
    round pegs in the square holes. The ones who see things differently.
    They’re not fond of rules. And they have no respect for the status quo.
    You
    can quote them, disagree with them, glorify or vilify them. About the
    only
    thing you can’t do is ignore them. Because they change things. They push
    the human race forward. And while some may see them as the crazy ones,
    we
    see genius. Because the people who are crazy enough to think they can
    change the world are the ones who do.*
2

Is this legal?

3

Subject: Writing a script to download videos
Date: Wed 12 Feb 14 01:54:47PM +0530

Quoting Himel S. ([email protected]):

I’m trying to write a script to download my edX course videos for me. This
is what I’ve written but it doesn’t seem to be working. can someone help me
out please?

When I try to download whatever points to your URL with ‘wget’, I do
not get any sort of MP4 video stream - just a HTML warning page. If
you are sure about the URL, this probably means that successful
communication to the website requires the setting of some sort of
cookies, and/or other sorts of out-of-band data exchange. You need to
make sense of the complete conversation that takes place between your
browser and the site, and then mimic the conversation.

This is not an easy task, and requires you to learn a lot of
things. Both for sniffing and for mimicing, Ruby can be an excellent
help, but the only way I know requires learning the ins and outs of
how HTTP conversation takes place. If you type on your terminal

ri Net::HTTP

do you understand what you read? If you don’t, then maybe the task is
too large a step for you.

Maybe someone else on the list can propose a ready-made saolution…

Carlo

4

On Wed, Feb 12, 2014 at 9:24 AM, Himel S. [email protected]
wrote:

open(“Authn | edX”)

do |read_file|
file.write(read_file.read)
end
end

It seems that in order to access that video, you first have to login.
If you check the file you generated, it’s an html file with a form to
login.

In order to web scrape like you need, I recommend using the mechanize
gem, with which you can direct the navigation, fill out forms, submit,
follow links, etc programmatically. If the site doesn’t require
javascript to work it works fine. I use it a lot to automate tasks on
websites. Here an example to post to an internal wiki we use at work:

    agent = Mechanize.new
    agent.user_agent_alias='Linux Mozilla'
    #get login page
    page = agent.get(URL)

    # do login
    form = page.form("login_form")
    form.username = configuration["WikiUser"]
    form.password = configuration["WikiPassword"]
    page = agent.submit(form, form.buttons.first)

    # go to B!Wiki
    page = page.link_with(:text => "B!Wiki").click
    # Submit a form to login in the wiki
    page = page.form("userlogin").submit
    page = page.link_with(:text => "Welcome").click

    # find the page and edit it
    calendar_page = configuration["WikiPage"] % year
    page = agent.get(calendar_page)
    page = page.link_with(:href =>

/action=edit&section=#{month+1}/).click

    # insert the new value for the table
    form = page.form("editform")
    form.wpTextbox1 = content
    # submit
    agent.submit(form, form.buttons.first)

Hope this helps,

Jesus.

5

Subject: Re: Writing a script to download videos
Date: Wed 12 Feb 14 09:46:51AM +0100

Quoting Joel P. ([email protected]):

Is this legal?

If the download requires some sort of login, then he’d have to read
the fine print of what he subscribed to when getting his
login. Otherwise, as far as I know, nothing would bind him to Edx. But
IANAL.

Carlo