Help parsing simple structured data

Ruby
#1

I’m rather new to Ruby and am trying to figure out how to parse some
structured data… This is pretty much the most common problem I run into
and it’d be really helpful to see an example to wrap my head around
blocks
& hashes

The data I’m working with is the output of df -h which looks like this
:

Filesystem Size Used Avail Use% Mounted on
/dev/md0 9.9G 1.6G 7.8G 17% /
/dev/md1 60G 154M 57G 1% /data
/dev/sda1 99M 12M 83M 13% /boot

In the end what I’d like to end up with is to be able to say print
hash[md0(size)] and have it
return 9.9G or hash[sda1(Mount)] and have it show the mountpoint of
/boot

If anyone could help I’d really appreciate it.

Craig

#2

Nuby R. wrote:

/dev/md1 60G 154M 57G 1% /data
/dev/sda1 99M 12M 83M 13% /boot

In the end what I’d like to end up with is to be able to say print
hash[md0(size)] and have it
return 9.9G or hash[sda1(Mount)] and have it show the mountpoint of /boot

If anyone could help I’d really appreciate it.

Craig

Try this:

rows = df -h.split("\n")[1…-1]
hash = Hash.new {|hash, key| hash[key] = {}}
rows.each do |row|
data = row.split /\s+/
device = data[0].sub(%r|^/dev/|, “”)
%w{size used avail use mount}.each_with_index do |field, column|
hash[device][field] = data[column+1]
end
end

After running it, you should be able to find your data in
hash[“md0”][“size”] or hash[“sda1”][“mount”].

Does this work for you?

Dan

#3

Dan,

That works perfectly, thank you a lot! This type of data I end up
dealing
with day-in and day-out, and this will help a lot.

Craig

#4

Nuby R. wrote:

structured data… This is pretty much the most common problem I run into
/dev/sda1 99M 12M 83M 13% /boot

After running it, you should be able to find your data in
hash[“md0”][“size”] or hash[“sda1”][“mount”].

Does this work for you?

Dan

Craig,

By the way, I just realized that my script assumes that the mount point
has no spaces. Nothing horrible will happen if it does, but
hash[device][“mount”] will only contain the first word, like “/d/My” of
“/d/My Documents”. If you want to fix that, you could replace a few of
the lines in the above script with these:

%w{size used avail use}.each_with_index do |field, column|
hash[device][field] = data[column+1]
end
hash[device][mount] = data[5…-1].join(" ")

Have fun,
Dan

#5

In my situation it’ll always be safe to assume there are no mount
points,
but I’ll grab your code anyways in case I ever run it on a windows or
mac
box.

BTW, here’s the code I’m working on, thanks for the tidbit, this little
bit
of processing is the bit of code I use most in my daily life, and the
hardest for me to wrap my head around when I play with new langauges.

#!/usr/bin/env ruby

require ‘rubygems’
require ‘getoptlong’
require ‘net/ssh’

opts = GetoptLong.new(
[ “–host”, “-h”, GetoptLong::REQUIRED_ARGUMENT]
)

hostname = nil
opts.each do |opt, arg|
case opt
when ‘–host’
hostname = arg
end
end

if hostname == nil
puts “Missing --host argument”
puts “usage: `ssh-test.rb --host hostname”
exit 0
end

Net::SSH.start( hostname,
:password=>‘somepassword’,
:port=>22,
:username=>‘rubyuser’
) do |session|
shell = session.shell.open
shell.df “-h”
sleep 0.5

              disks = shell.stdout.split("\n")[1..-1] while

shell.stdout
?

              hash = Hash.new {|hash, key| hash[key] = {}}
              disks.each do |disk|
                data = disk.split /\s+/
                device = data[0].sub(%r|^/dev/|, "")
                %w{size  used  avail  use  mount}.each_with_index do

|field, column|
hash[device][field] = data[column+1]
end
end

              print hash["md0"]["mount"]
              puts "\n"

end

#6

From: Nuby R. [mailto:[email protected]]

of processing is the bit of code I use most in my daily life, and the

hardest for me to wrap my head around when I play with new langauges.

you sound like an admin or you do admin tasks daily.

just want to tell you that there’s a gem called ruport used for
reporting task. …just in case it may help you.

this is just a simple example. i snipped the initial echos fr irb. the
last outputs will give you an idea…

so,

require ‘rubygems’
require ‘ruport’

data =df -h.split “\n”
header=data.shift.split " ",6
data = data.map{|line| line.split}
report = Ruport::Data::Table.new :data => data, :column_names => header

puts report
±------------------------------------------------------------------------------+
| Filesystem | Size | Used | Avail | Use% | Mounted on |
±------------------------------------------------------------------------------+
| /dev/hda1 | 3.8G | 3.1G | 566M | 85% | / |
| varrun | 125M | 68K | 125M | 1% | /var/run |
| varlock | 125M | 4.0K | 125M | 1% | /var/lock |
| udev | 125M | 76K | 125M | 1% | /dev |
| devshm | 125M | 0 | 125M | 0% | /dev/shm |
| lrm | 125M | 18M | 107M | 15% | /lib/modules/2.6.15-28-686/volatile |
| /dev/hdb1 | 19G | 6.9G | 13G | 36% | /disk2 |
| /dev/hdd1 | 2.0G | 731M | 1.3G | 37% | /disk3-1 |
| /dev/hdd2 | 2.1G | 65M | 1.9G | 4% | /disk3-2 |
±------------------------------------------------------------------------------+
=> nil

puts report.column(“Filesystem”)
/dev/hda1
varrun
varlock
udev
devshm
lrm
/dev/hdb1
/dev/hdd1
/dev/hdd2
=> nil

puts report.column(“Size”)
3.8G
125M
125M
125M
125M
125M
19G
2.0G
2.1G
=> nil

there are a lot of fxns in ruport that allow you to sum on columns, swap
them, remove/add columns, drill down on records, etc…

many thanks to GBrown and MMilner, et al for ruport.
kind regards -botp

#7

On Aug 29, 7:49 am, Thibaut Barrère [email protected] wrote:

just want to tell you that there’s a gem calledruportused for reporting task. …just in case it may help you.

+1 forruport, it makes these things very easy to achieve, with less
code.

FWIW, you’ll get extra features at no cost, like:

File.open(‘report.pdf’,‘w’) do |file|
file << table.to_pdf
end

Or as of today:

table.to_pdf(:file => "report.pdf)

#8

just want to tell you that there’s a gem called ruport used for reporting task. …just in case it may help you.

+1 for ruport, it makes these things very easy to achieve, with less
code.

FWIW, you’ll get extra features at no cost, like:

File.open(‘report.pdf’,‘w’) do |file|
file << table.to_pdf
end

cheers

Thibaut

#9

Oh man this is awesome. I love ruby, I spent the last hour playing with
ruport, and am in love with it’s simplicity… The reason I’m learning
Ruby
is to make it easier for me to get off my butt and write reports.