Forum: Ruby url-monitoring script question

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-11-21 10:40
(Received via mailing list)
Hi @all,

i made a script, that monitors some web-sites of our company using
net::http
and net::smtp.

it pings multiple sites and it's based on this script:
http://habtm.com/articles/2005/09/29/website-monit...

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is
down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no 'history' information to
recognize,
how many notifications have been sent already.

any suggestion how that can be done?

(i hope i made it clear -- sorry for my bad english --)

Best Regards,
Torsten
6b4566518f6675477dab9b8ba813cf3c?d=identicon&s=25 ruby.brian (Guest)
on 2005-11-21 11:00
(Received via mailing list)
On 21/11/05, Torsten Schmidt <torstello@gmail.com> wrote:
>
> any suggestion how that can be done?
>
> (i hope i made it clear -- sorry for my bad english --)
>
> Best Regards,
> Torsten
>
>

You correctly identified the problem. You need to collect history
information somewhere. I would store the error state of each possible
error seperately in a file or db. If you store the state
(online/offline) for each address, you can mail only when the state
changes.

cheers,

Brian

--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/
9fa8c9b0ef45b8d19ce65f726166e520?d=identicon&s=25 ihspm (Guest)
on 2005-11-21 11:04
(Received via mailing list)
If the number of sites is known, you could keep a file for all sites
and the related status (e.g. one line per site)

mysite.mydomain.org: ok
      could mean, last run the site could be accessed
mysite.mydomain.org: 2
      could mean, site not accesible for two consecutive tries

Therefore, compare your findings against the file, send an email if 1<=
errcount<=3, and update the file afterwards (incrementing or resetting
the error count)
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-11-21 11:45
(Received via mailing list)
Thanks for the input so far.

another possible solution come to my (ruby-beginner) mind instead of
writing
the status-code to a seperate file and parse it:

some kind of deamon-process:
what about to put something like 'sleep 300' in the script to make an
indefinetly loop.
so it's easier to store the exit-code of the http-requests and compare
them.

with this it would be possible to send 5 error-emails (25 min) and after
that, send another email when the url 'recovers'.

What are your opinions for that solution?

Torsten

2005/11/21, Heiko Leberer <ihspm@haleb.de>:
D8914caf9cc67403f1f153e0a2104293?d=identicon&s=25 dave.baldwin (Guest)
on 2005-11-21 11:57
(Received via mailing list)
On 21 Nov 2005, at 10:44, Torsten Schmidt wrote:

> compare them.
>>
>> errcount<=3, and update the file afterwards (incrementing or
>> resetting
>> the error count)
>>

I had a need recently to monitor my internet connection at home.
This is the ruby program I wrote to do it.  It just keeps track of
the on line and off line time in a terminal window.

require 'open-uri.rb'

onAt = Time.now
offAt = Time.now
online = false
working = true

puts "Log starting at #{Time.now}"

def printTime (text, start)
	dur = Time.now - start
	hours = (dur / 3600).to_i
	minutes = (dur / 60).to_i - hours * 60
	print "#{text} #{start}, for #{hours} hours and #{minutes} minutes"
	$stdout.flush
end

while true
	working = false
	3.times do
		begin
			ans = open("http://www.google.com/").read
			working = true
			break
		rescue Timeout::Error, StandardError
		end
	end
	if !online && working
		onAt = Time.now
		online = true
		printTime("\nOn line at", onAt)
	elsif !online && !working
		printTime("\rOff line at", offAt)
	elsif online && working
		printTime("\rOn line at", onAt)
	else online && !working
		offAt = Time.now
		online = false
		printTime("\nOff line at", offAt)
	end
	sleep 10
end
Cfdeff3ac35010e4de8f85d954f24f4a?d=identicon&s=25 damphyr (Guest)
on 2005-11-21 12:21
(Received via mailing list)
Torsten Schmidt wrote:
> after that, send another email when the url 'recovers'.
>
> What are your opinions for that solution?
What you can also do is hold the state in a simple class:
class SiteState
	def initialize
		@errors=Array.new
		@notified=false
		...
	end
	...
end
and just use Marshal to dump the class in a file between calls to the
script.
It's the easiest way to persist information and you don't need to come
up with a file structure and parse code (even if it is trivial)
Cheers,
V.-
--
http://www.braveworld.net/riva
Fa4c23dfa1fe9e5671bd89c8539f1522?d=identicon&s=25 mike (Guest)
on 2005-11-21 12:54
(Received via mailing list)
On Mon, Nov 21, 2005 at 06:40:21PM +0900, Torsten Schmidt wrote:
> any suggestion how that can be done?

http://tildeslash.com/monit/
5befe95e6648daec3dd5728cd36602d0?d=identicon&s=25 bob.news (Guest)
on 2005-11-21 13:46
(Received via mailing list)
Damphyr wrote:
>> with this it would be possible to send 5 error-emails (25 min) and
> ...
> end
> and just use Marshal to dump the class in a file between calls to the
> script.
> It's the easiest way to persist information and you don't need to come
> up with a file structure and parse code (even if it is trivial)
> Cheers,

Right.  But it doesn't necessarily need to be inside a single special
class.  A hash might do as well.

Kind regards

    robert
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 ara.t.howard (Guest)
on 2005-11-21 17:00
(Received via mailing list)
On Mon, 21 Nov 2005, Torsten Schmidt wrote:

>
> any suggestion how that can be done?
here's mine - it mails only 3 times.  it's ugly, but functional:

   harp:~ > cat bin/uriup.rb
   #! /home/ahoward/bin/ruby
   #
   # simple script to monitor uris
   #
   # sample cron line
   #
   #   */5 * * * * /usr/local/ruby-1.8.0/bin/ruby
/full/path/to/this/script > /dev/null 2>&1
   #

     require "net/http"
     require "net/smtp"
     require "yaml/store"
     require "socket"
   #
   # array of urls to ping
   #
     uris = %w(
       www.codeforpeople.com
       sciruby.codeforpeople.com
       www.zstone.net
       www.ithmezipper.net
     )
   #
   # array of people to notify if urls are down
   #
     recipients = %w(
       ara.t.howard@noaa.gov
     )
   #
   # message format string
   #
     msg_fmt = %Q(
       URI: %s

       TIME: %s

       EXCEPTION: %s\n%s
     )
   #
   # user to send messages as
   #
     user = ENV["USER"] || "ahoward"
   #
   # host to send messages from
   #
     host = ENV["HOSTNAME"] || ENV["HOST"] || Socket::gethostname
   #
   # maximum number of messages to send
   #
     msg_max = 3
   #
   # db class to store codes/notifications
   #
     class DB
       attr "path"
       attr "db"
       def initialize path = File::join(File::expand_path("~"),
".uri.db")
         @path = path
         @db = ::YAML::Store::new @path
       end
       def reset uri
         @db.transaction{ @db[uri] = {"success" => true, "msgs" => 0} }
       end
       def [] uri
         @db.transaction{ @db[uri] } || reset(uri)
       end
       def []= uri, record
         @db.transaction{ @db[uri] = record }
       end
     end
   #
   # umbrella error class
   #
     class SiteDownError < StandardError; end
   #
   # ping each url, mail messages if failure for any reason...
   #
     db = DB::new

     uris.each do |uri|
       begin
         raise SiteDownError unless
           Net::HTTPOK === Net::HTTP::new(uri, 80).get("/")
         y uri => "up"
         db.reset uri

       rescue Exception => e
         y uri => "down"
         record = db[uri]

         if record["msgs"] < msg_max
           now = Time::now
           msg = msg_fmt % [uri, now, e,
e.backtrace.join("\n").gsub(%r/^/,"\t")]
           from = "%s@%s" % [user, host]

           Net::SMTP::start("localhost") do |smtp|
             recipients.each do |recipient|
               email = "From: #{ from }\r\n" <<
                       "To: #{ recipient }\r\n" <<
                       "Subject: #{ uri } DOWN @ #{ now }\r\n" <<
                       "\r\n#{ msg }"
               smtp.send_message email, from, recipient
             end
           end

           record["success"] = false
           record["msgs"] += 1
           db[uri] = record
         end
       end
     end


the database files looks like this:

   harp:~ > cat ~/.uri.db
   ---
   www.codeforpeople.com:
     success: true
     msgs: 0
   www.ithmezipper.net:
     msgs: 0
     success: true
   sciruby.codeforpeople.com:
     msgs: 0
     success: true
   www.zstone.net:
     success: true
     msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-11-21 19:29
(Received via mailing list)
Thanks a lot to all for your imput.

different ways to work with are always good for learning ruby the
practical
way.

Best wishes,

Torsten

---------- Forwarded message ----------
From: Ara.T.Howard <ara.t.howard@noaa.gov>
Date: 21.11.2005 16:57
Subject: Re: url-monitoring script question
To: ruby-talk ML <ruby-talk@ruby-lang.org>

On Mon, 21 Nov 2005, Torsten Schmidt wrote:

> Hi @all,
>
> i made a script, that monitors some web-sites of our company using
net::http
>
> i would like to prevent my mailbox being filled up, when one site is down
> for the whole night or so
> as it can be done with nagios.
>
> one problem is, that the script has no 'history' information to recognize,
> how many notifications have been sent already.
>
> any suggestion how that can be done?

here's mine - it mails only 3 times. it's ugly, but functional:

harp:~ > cat bin/uriup.rb
#! /home/ahoward/bin/ruby
#
# simple script to monitor uris
#
# sample cron line
#
# */5 * * * * /usr/local/ruby-1.8.0/bin/ruby /full/path/to/this/script >
/dev/null 2>&1
#

require "net/http"
require "net/smtp"
require "yaml/store"
require "socket"
#
# array of urls to ping
#
uris = %w(
www.codeforpeople.com <http://www.codeforpeople.com>
sciruby.codeforpeople.com <http://sciruby.codeforpeople.com>
www.zstone.net <http://www.zstone.net>
www.ithmezipper.net <http://www.ithmezipper.net>
)
#
# array of people to notify if urls are down
#
recipients = %w(
ara.t.howard@noaa.gov
)
#
# message format string
#
msg_fmt = %Q(
URI: %s

TIME: %s

EXCEPTION: %s\n%s
)
#
# user to send messages as
#
user = ENV["USER"] || "ahoward"
#
# host to send messages from
#
host = ENV["HOSTNAME"] || ENV["HOST"] || Socket::gethostname
#
# maximum number of messages to send
#
msg_max = 3
#
# db class to store codes/notifications
#
class DB
attr "path"
attr "db"
def initialize path = File::join(File::expand_path("~"), ".uri.db")
@path = path
@db = ::YAML::Store::new @path
end
def reset uri
@db.transaction{ @db[uri] = {"success" => true, "msgs" => 0} }
end
def [] uri
@db.transaction{ @db[uri] } || reset(uri)
end
def []= uri, record
@db.transaction{ @db[uri] = record }
end
end
#
# umbrella error class
#
class SiteDownError < StandardError; end
#
# ping each url, mail messages if failure for any reason...
#
db = DB::new

uris.each do |uri|
begin
raise SiteDownError unless
Net::HTTPOK === Net::HTTP::new(uri, 80).get("/")
y uri => "up"
db.reset uri

rescue Exception => e
y uri => "down"
record = db[uri]

if record["msgs"] < msg_max
now = Time::now
msg = msg_fmt % [uri, now, e, e.backtrace.join("\n").gsub(%r/^/,"\t")]
from = "%s@%s" % [user, host]

Net::SMTP::start("localhost") do |smtp|
recipients.each do |recipient|
email = "From: #{ from }\r\n" <<
"To: #{ recipient }\r\n" <<
"Subject: #{ uri } DOWN @ #{ now }\r\n" <<
"\r\n#{ msg }"
smtp.send_message email, from, recipient
end
end

record["success"] = false
record["msgs"] += 1
db[uri] = record
end
end
end


the database files looks like this:

harp:~ > cat ~/.uri.db
---
www.codeforpeople.com <http://www.codeforpeople.com>:
success: true
msgs: 0
www.ithmezipper.net <http://www.ithmezipper.net>:
msgs: 0
success: true
sciruby.codeforpeople.com <http://sciruby.codeforpeople.com>:
msgs: 0
success: true
www.zstone.net <http://www.zstone.net>:
success: true
msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a
--
===============================================================================
| ara [dot] t [dot] howard [at] gmail [dot] com
| all happiness comes from the desire for others to be happy. all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
===============================================================================
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-12-02 13:48
(Received via mailing list)
Hi @all,

i'll be back with an issue of my web-monitoring-script.

yesterday i got the following email from my cron-daemon:

/usr/lib/ruby/1.8/net/protocol.rb:83:in `initialize': Connection refused
-
connect(2) (Errno::ECONNREFUSED)
    from /usr/lib/ruby/1.8/net/protocol.rb:83:in `new'
    from /usr/lib/ruby/1.8/net/protocol.rb:83:in `connect'
    from /usr/lib/ruby/1.8/net/protocol.rb:82:in `timeout'
    from /usr/lib/ruby/1.8/timeout.rb:55:in `timeout'
    from /usr/lib/ruby/1.8/net/protocol.rb:82:in `connect'
    from /usr/lib/ruby/1.8/net/protocol.rb:64:in `initialize'
    from /usr/lib/ruby/1.8/net/http.rb:430:in `open'
    from /usr/lib/ruby/1.8/net/http.rb:430:in `do_start'
    from /usr/lib/ruby/1.8/net/http.rb:419:in `start'
    from /usr/lib/ruby/1.8/net/http.rb:296:in `get_by_uri'
    from /usr/lib/ruby/1.8/net/http.rb:282:in `get_response'
    from /usr/local/bin/webcattest/WebCatMonitoring.rb:34
    from /usr/local/bin/webcattest/WebCatMonitoring.rb:32:in `each'
    from /usr/local/bin/webcattest/WebCatMonitoring.rb:32


i was able to handle a timeout-error, but the error above was not
handled.

Unfortunately i don't know how to reproduce the connection-refused error
where i can test my new rescue method:
see rescue Exception => err

-------------------------------------------------------------------
require 'net/smtp'
require 'net/http'

apps = [
  { :name => 'url-test',         :url => 'test/url' },
#...
]
for app in apps
  app[:url] = URI.parse("http://#{app[:url]}")
end

errors = []

for app in apps.sort { rand }
  begin
    res = Net::HTTP.get_response(app[:url])

    error = nil
    if %w[ 301 302 ].include?res.response.code
      res = Net::HTTP.get_response( URI.parse(res.header["location"] ))
    end
    if res.response.code.to_i != 200
      error = ["<h2>#{app[:name]}</h2><p> error code:
#{res.response.code
}</p>"]
#...
      errors << error
    end
  rescue Timeout::Error => err
    errors << error = ["Timeout Error: #{app[:name]} not reachable!"]
  rescue Exception => exception
    errors << "<p></p><strong>#{app[:name]}</strong> with #{app[:url]}
</strong>not reachable!<br/><pre>#{exception.message}</pre>"
  end
 end
# handling Emails...
-------------------------------------------------------------------

Any ideas how to reproduce a connection-refused error or suggestions if
my
rescue Exception can handle this?

(sorry for my bad english)

Regards,
Torsten
6b4566518f6675477dab9b8ba813cf3c?d=identicon&s=25 ruby.brian (Guest)
on 2005-12-02 14:05
(Received via mailing list)
>
Try to connect to a not existing address. Something like
xyz.example.com.

cheers,

brian


--
http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/
59c57451c9fe66b55b0f163806b659ba?d=identicon&s=25 nakohl (Guest)
on 2005-12-02 14:05
(Received via mailing list)
You should be able to get a connection refused by trying to connect to
an
unused port. For example, if your "real" url is
http://testing.com/test/url/then try
http://testing.com:81/test/url/. As long as there's no web server
running on
port 81, you should get a connection refused error.

--
Neil Kohl
nakohl@gmail.com
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-12-02 14:17
(Received via mailing list)
i've already tried this, but i only get an timout-error which is handled
in
my first rescue statement.


2005/12/2, Neil Kohl <nakohl@gmail.com>:
82e62c756d89bc6fa0a0a2d7f2b1e617?d=identicon&s=25 rosco (Guest)
on 2005-12-02 14:50
(Received via mailing list)
On Fri, 02 Dec 2005 13:13:22 -0000, Torsten Schmidt
<torstello@gmail.com>
wrote:
>>
> i've already tried this, but i only get an timout-error which is handled
> in
> my first rescue statement.
>

Unused ports will timeout - for active refusal, try a firewalled port.
84.92.85.107, any port, should refuse you if you don't have anything
handy.
Bd5c8bbb64259e72bde214f2e50a3169?d=identicon&s=25 torstello (Guest)
on 2005-12-02 15:18
(Received via mailing list)
thanks Ross, that was it.

the rescue works.

2005/12/2, Ross Bamford <rosco@roscopeco.remove.co.uk>:
This topic is locked and can not be replied to.