Url-monitoring script question


#1

Hi @all,

i made a script, that monitors some web-sites of our company using
net::http
and net::smtp.

it pings multiple sites and it’s based on this script:
http://habtm.com/articles/2005/09/29/website-monitoring-script

every time an error occures, an email ist sent.
i want to do this every 5 min as a cron-job.

Is there a way to limit the email-notification, that for example only 3
emails with the same error are delivered?

i would like to prevent my mailbox being filled up, when one site is
down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no ‘history’ information to
recognize,
how many notifications have been sent already.

any suggestion how that can be done?

(i hope i made it clear – sorry for my bad english --)

Best Regards,
Torsten


#2

If the number of sites is known, you could keep a file for all sites
and the related status (e.g. one line per site)

mysite.mydomain.org: ok
could mean, last run the site could be accessed
mysite.mydomain.org: 2
could mean, site not accesible for two consecutive tries

Therefore, compare your findings against the file, send an email if 1<=
errcount<=3, and update the file afterwards (incrementing or resetting
the error count)


#3

On 21/11/05, Torsten S. removed_email_address@domain.invalid wrote:

any suggestion how that can be done?

(i hope i made it clear – sorry for my bad english --)

Best Regards,
Torsten

You correctly identified the problem. You need to collect history
information somewhere. I would store the error state of each possible
error seperately in a file or db. If you store the state
(online/offline) for each address, you can mail only when the state
changes.

cheers,

Brian


http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/


#4

Thanks for the input so far.

another possible solution come to my (ruby-beginner) mind instead of
writing
the status-code to a seperate file and parse it:

some kind of deamon-process:
what about to put something like ‘sleep 300’ in the script to make an
indefinetly loop.
so it’s easier to store the exit-code of the http-requests and compare
them.

with this it would be possible to send 5 error-emails (25 min) and after
that, send another email when the url ‘recovers’.

What are your opinions for that solution?

Torsten

2005/11/21, Heiko L. removed_email_address@domain.invalid:


#5

Torsten S. wrote:

after that, send another email when the url ‘recovers’.

What are your opinions for that solution?
What you can also do is hold the state in a simple class:
class SiteState
def initialize
@errors=Array.new
@notified=false

end

end
and just use Marshal to dump the class in a file between calls to the
script.
It’s the easiest way to persist information and you don’t need to come
up with a file structure and parse code (even if it is trivial)
Cheers,
V.-

http://www.braveworld.net/riva


#6

On Mon, Nov 21, 2005 at 06:40:21PM +0900, Torsten S. wrote:

any suggestion how that can be done?

http://tildeslash.com/monit/


#7

Damphyr wrote:

with this it would be possible to send 5 error-emails (25 min) and

end
and just use Marshal to dump the class in a file between calls to the
script.
It’s the easiest way to persist information and you don’t need to come
up with a file structure and parse code (even if it is trivial)
Cheers,

Right. But it doesn’t necessarily need to be inside a single special
class. A hash might do as well.

Kind regards

robert

#8

On 21 Nov 2005, at 10:44, Torsten S. wrote:

compare them.

errcount<=3, and update the file afterwards (incrementing or
resetting
the error count)

I had a need recently to monitor my internet connection at home.
This is the ruby program I wrote to do it. It just keeps track of
the on line and off line time in a terminal window.

require ‘open-uri.rb’

onAt = Time.now
offAt = Time.now
online = false
working = true

puts “Log starting at #{Time.now}”

def printTime (text, start)
dur = Time.now - start
hours = (dur / 3600).to_i
minutes = (dur / 60).to_i - hours * 60
print “#{text} #{start}, for #{hours} hours and #{minutes} minutes”
$stdout.flush
end

while true
working = false
3.times do
begin
ans = open(“http://www.google.com/”).read
working = true
break
rescue Timeout::Error, StandardError
end
end
if !online && working
onAt = Time.now
online = true
printTime("\nOn line at", onAt)
elsif !online && !working
printTime("\rOff line at", offAt)
elsif online && working
printTime("\rOn line at", onAt)
else online && !working
offAt = Time.now
online = false
printTime("\nOff line at", offAt)
end
sleep 10
end


#9

On Mon, 21 Nov 2005, Torsten S. wrote:

any suggestion how that can be done?
here’s mine - it mails only 3 times. it’s ugly, but functional:

harp:~ > cat bin/uriup.rb
#! /home/ahoward/bin/ruby

simple script to monitor uris

sample cron line

*/5 * * * * /usr/local/ruby-1.8.0/bin/ruby

/full/path/to/this/script > /dev/null 2>&1

 require "net/http"
 require "net/smtp"
 require "yaml/store"
 require "socket"

array of urls to ping

 uris = %w(
   www.codeforpeople.com
   sciruby.codeforpeople.com
   www.zstone.net
   www.ithmezipper.net
 )

array of people to notify if urls are down

 recipients = %w(
   removed_email_address@domain.invalid
 )

message format string

 msg_fmt = %Q(
   URI: %s

   TIME: %s

   EXCEPTION: %s\n%s
 )

user to send messages as

 user = ENV["USER"] || "ahoward"

host to send messages from

 host = ENV["HOSTNAME"] || ENV["HOST"] || Socket::gethostname

maximum number of messages to send

 msg_max = 3

db class to store codes/notifications

 class DB
   attr "path"
   attr "db"
   def initialize path = File::join(File::expand_path("~"), 

“.uri.db”)
@path = path
@db = ::YAML::Store::new @path
end
def reset uri
@db.transaction{ @db[uri] = {“success” => true, “msgs” => 0} }
end
def [] uri
@db.transaction{ @db[uri] } || reset(uri)
end
def []= uri, record
@db.transaction{ @db[uri] = record }
end
end

umbrella error class

 class SiteDownError < StandardError; end

ping each url, mail messages if failure for any reason…

 db = DB::new

 uris.each do |uri|
   begin
     raise SiteDownError unless
       Net::HTTPOK === Net::HTTP::new(uri, 80).get("/")
     y uri => "up"
     db.reset uri

   rescue Exception => e
     y uri => "down"
     record = db[uri]

     if record["msgs"] < msg_max
       now = Time::now
       msg = msg_fmt % [uri, now, e, 

e.backtrace.join("\n").gsub(%r/^/,"\t")]
from = “%s@%s” % [user, host]

       Net::SMTP::start("localhost") do |smtp|
         recipients.each do |recipient|
           email = "From: #{ from }\r\n" <<
                   "To: #{ recipient }\r\n" <<
                   "Subject: #{ uri } DOWN @ #{ now }\r\n" <<
                   "\r\n#{ msg }"
           smtp.send_message email, from, recipient
         end
       end

       record["success"] = false
       record["msgs"] += 1
       db[uri] = record
     end
   end
 end

the database files looks like this:

harp:~ > cat ~/.uri.db

www.codeforpeople.com:
success: true
msgs: 0
www.ithmezipper.net:
msgs: 0
success: true
sciruby.codeforpeople.com:
msgs: 0
success: true
www.zstone.net:
success: true
msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a


#10

Thanks a lot to all for your imput.

different ways to work with are always good for learning ruby the
practical
way.

Best wishes,

Torsten

---------- Forwarded message ----------
From: Ara.T.Howard removed_email_address@domain.invalid
Date: 21.11.2005 16:57
Subject: Re: url-monitoring script question
To: ruby-talk ML removed_email_address@domain.invalid

On Mon, 21 Nov 2005, Torsten S. wrote:

Hi @all,

i made a script, that monitors some web-sites of our company using
net::http

i would like to prevent my mailbox being filled up, when one site is down
for the whole night or so
as it can be done with nagios.

one problem is, that the script has no ‘history’ information to recognize,
how many notifications have been sent already.

any suggestion how that can be done?

here’s mine - it mails only 3 times. it’s ugly, but functional:

harp:~ > cat bin/uriup.rb
#! /home/ahoward/bin/ruby

simple script to monitor uris

sample cron line

*/5 * * * * /usr/local/ruby-1.8.0/bin/ruby /full/path/to/this/script >

/dev/null 2>&1

require “net/http”
require “net/smtp”
require “yaml/store”
require “socket”

array of urls to ping

uris = %w(
www.codeforpeople.com http://www.codeforpeople.com
sciruby.codeforpeople.com http://sciruby.codeforpeople.com
www.zstone.net http://www.zstone.net
www.ithmezipper.net http://www.ithmezipper.net
)

array of people to notify if urls are down

recipients = %w(
removed_email_address@domain.invalid
)

message format string

msg_fmt = %Q(
URI: %s

TIME: %s

EXCEPTION: %s\n%s
)

user to send messages as

user = ENV[“USER”] || “ahoward”

host to send messages from

host = ENV[“HOSTNAME”] || ENV[“HOST”] || Socket::gethostname

maximum number of messages to send

msg_max = 3

db class to store codes/notifications

class DB
attr “path”
attr “db”
def initialize path = File::join(File::expand_path("~"), “.uri.db”)
@path = path
@db = ::YAML::Store::new @path
end
def reset uri
@db.transaction{ @db[uri] = {“success” => true, “msgs” => 0} }
end
def [] uri
@db.transaction{ @db[uri] } || reset(uri)
end
def []= uri, record
@db.transaction{ @db[uri] = record }
end
end

umbrella error class

class SiteDownError < StandardError; end

ping each url, mail messages if failure for any reason…

db = DB::new

uris.each do |uri|
begin
raise SiteDownError unless
Net::HTTPOK === Net::HTTP::new(uri, 80).get("/")
y uri => “up”
db.reset uri

rescue Exception => e
y uri => “down”
record = db[uri]

if record[“msgs”] < msg_max
now = Time::now
msg = msg_fmt % [uri, now, e, e.backtrace.join("\n").gsub(%r/^/,"\t")]
from = “%s@%s” % [user, host]

Net::SMTP::start(“localhost”) do |smtp|
recipients.each do |recipient|
email = “From: #{ from }\r\n” <<
“To: #{ recipient }\r\n” <<
“Subject: #{ uri } DOWN @ #{ now }\r\n” <<
“\r\n#{ msg }”
smtp.send_message email, from, recipient
end
end

record[“success”] = false
record[“msgs”] += 1
db[uri] = record
end
end
end

the database files looks like this:

harp:~ > cat ~/.uri.db

www.codeforpeople.com http://www.codeforpeople.com:
success: true
msgs: 0
www.ithmezipper.net http://www.ithmezipper.net:
msgs: 0
success: true
sciruby.codeforpeople.com http://sciruby.codeforpeople.com:
msgs: 0
success: true
www.zstone.net http://www.zstone.net:
success: true
msgs: 0

using YAML::Store eliminates the need to roll-your-own.

regards.

-a

===============================================================================
| ara [dot] t [dot] howard [at] gmail [dot] com
| all happiness comes from the desire for others to be happy. all misery
| comes from the desire for oneself to be happy.
| – bodhicaryavatara


#11

Try to connect to a not existing address. Something like
xyz.example.com.

cheers,

brian


http://ruby.brian-schroeder.de/

Stringed instrument chords: http://chordlist.brian-schroeder.de/


#12

You should be able to get a connection refused by trying to connect to
an
unused port. For example, if your “real” url is
http://testing.com/test/url/then try
http://testing.com:81/test/url/. As long as there’s no web server
running on
port 81, you should get a connection refused error.


Neil K.
removed_email_address@domain.invalid


#13

Hi @all,

i’ll be back with an issue of my web-monitoring-script.

yesterday i got the following email from my cron-daemon:

/usr/lib/ruby/1.8/net/protocol.rb:83:in `initialize’: Connection refused

connect(2) (Errno::ECONNREFUSED)
from /usr/lib/ruby/1.8/net/protocol.rb:83:in new' from /usr/lib/ruby/1.8/net/protocol.rb:83:inconnect’
from /usr/lib/ruby/1.8/net/protocol.rb:82:in timeout' from /usr/lib/ruby/1.8/timeout.rb:55:intimeout’
from /usr/lib/ruby/1.8/net/protocol.rb:82:in connect' from /usr/lib/ruby/1.8/net/protocol.rb:64:ininitialize’
from /usr/lib/ruby/1.8/net/http.rb:430:in open' from /usr/lib/ruby/1.8/net/http.rb:430:indo_start’
from /usr/lib/ruby/1.8/net/http.rb:419:in start' from /usr/lib/ruby/1.8/net/http.rb:296:inget_by_uri’
from /usr/lib/ruby/1.8/net/http.rb:282:in get_response' from /usr/local/bin/webcattest/WebCatMonitoring.rb:34 from /usr/local/bin/webcattest/WebCatMonitoring.rb:32:ineach’
from /usr/local/bin/webcattest/WebCatMonitoring.rb:32

i was able to handle a timeout-error, but the error above was not
handled.

Unfortunately i don’t know how to reproduce the connection-refused error
where i can test my new rescue method:
see rescue Exception => err


require ‘net/smtp’
require ‘net/http’

apps = [
{ :name => ‘url-test’, :url => ‘test/url’ },
#…
]
for app in apps
app[:url] = URI.parse(“http://#{app[:url]}”)
end

errors = []

for app in apps.sort { rand }
begin
res = Net::HTTP.get_response(app[:url])

error = nil
if %w[ 301 302 ].include?res.response.code
  res = Net::HTTP.get_response( URI.parse(res.header["location"] ))
end
if res.response.code.to_i != 200
  error = ["<h2>#{app[:name]}</h2><p> error code: 

#{res.response.code
}

"]
#…
errors << error
end
rescue Timeout::Error => err
errors << error = [“Timeout Error: #{app[:name]} not reachable!”]
rescue Exception => exception
errors << “

#{app[:name]} with #{app[:url]}
not reachable!
#{exception.message}

end
end

handling Emails…


Any ideas how to reproduce a connection-refused error or suggestions if
my
rescue Exception can handle this?

(sorry for my bad english)

Regards,
Torsten


#14

On Fri, 02 Dec 2005 13:13:22 -0000, Torsten S.
removed_email_address@domain.invalid
wrote:

i’ve already tried this, but i only get an timout-error which is handled
in
my first rescue statement.

Unused ports will timeout - for active refusal, try a firewalled port.
84.92.85.107, any port, should refuse you if you don’t have anything
handy.


#15

i’ve already tried this, but i only get an timout-error which is handled
in
my first rescue statement.

2005/12/2, Neil K. removed_email_address@domain.invalid:


#16

thanks Ross, that was it.

the rescue works.

2005/12/2, Ross B. removed_email_address@domain.invalid: