Optimization help - reading out of /proc on Solaris

Hi all,

Ruby 1.8.6
Solaris 10

I recently converted a C extension to get process table information on
Solaris into a pure Ruby. I knew it would be slower, I just didn’t
realize how much slower it would be. I was expecting the pure Ruby
version to be about 1/10th as fast. Instead, it’s about 1/70th as
fast. Anticipating the, “Is it fast enough?” question, my answer is,
“I’m not sure”. Besides, tuning can be fun. :slight_smile:

Anyway, below is the code. I ran it through the profiler, but the top
two most costly ops were Dir.foreach, which I don’t see any way to
optimize*, and the loop that gathers environment information, which I
again see no way to optimize.

sunos.rb

A pure Ruby version of sys-proctable for SunOS 5.8 or later

#–

Directories under /proc on Solaris 2.8+

The Sys module serves as a namespace only.

module Sys

The ProcTable class encapsulates process table information.

class ProcTable

  # The version of the sys-proctable library
  VERSION = '0.8.0'

  private

  PRNODEV = -1 # non-existent device

  FIELDS = [
     :flag,      # process flags (deprecated)
     :nlwp,      # number of active lwp's in the process
     :pid,       # unique process id
     :ppid,      # process id of parent
     :pgid,      # pid of session leader
     :sid,       # session id
     :uid,       # real user id
     :euid,      # effective user id
     :gid,       # real group id
     :egid,      # effective group id
     :addr,      # address of the process
     :size,      # size of process in kbytes
     :rssize,    # resident set size in kbytes
     :ttydev,    # tty device (or PRNODEV)
     :pctcpu,    # % of recent cpu used by all lwp's
     :pctmem,    # % of system memory used by process
     :start,     # absolute process start time
     :time,      # usr + sys cpu time for this process
     :ctime,     # usr + sys cpu time for reaped children
     :fname,     # name of the exec'd file
     :psargs,    # initial characters argument list
     :wstat,     # if a zombie, the wait status
     :argc,      # initial argument count
     :argv,      # address of initial argument vector
     :envp,      # address of initial environment vector
     :dmodel,    # data model of the process
     :taskid,    # task id
     :projid,    # project id
     :nzomb,     # number of zombie lwp's in the process
     :poolid,    # pool id
     :zoneid,    # zone id
     :contract,  # process contract
     :lwpid,     # lwp id
     :wchan,     # wait address for sleeping lwp
     :stype,     # synchronization event type
     :state,     # numeric lwp state
     :sname,     # printable character for state
     :nice,      # nice for cpu usage
     :syscall,   # system call number (if in syscall)
     :pri,       # priority
     :clname,    # scheduling class name
     :name,      # name of system lwp
     :onpro,     # processor which last ran thsi lwp
     :bindpro,   # processor to which lwp is bound
     :bindpset,  # processor set to which lwp is bound
     :count,     # number of contributing lwp's
     :tstamp,    # current time stamp
     :create,    # process/lwp creation time stamp
     :term,      # process/lwp termination time stamp
     :rtime,     # total lwp real (elapsed) time
     :utime,     # user level cpu time
     :stime,     # system call cpu time
     :ttime,     # other system trap cpu time
     :tftime,    # text page fault sleep time
     :dftime,    # text page fault sleep time
     :kftime,    # kernel page fault sleep time
     :ltime,     # user lock wait sleep time
     :slptime,   # all other sleep time
     :wtime,     # wait-cpu (latency) time
     :stoptime,  # stopped time
     :minf,      # minor page faults
     :majf,      # major page faults
     :nswap,     # swaps
     :inblk,     # input blocks
     :oublk,     # output blocks
     :msnd,      # messages sent
     :mrcv,      # messages received
     :sigs,      # signals received
     :vctx,      # voluntary context switches
     :ictx,      # involuntary context switches
     :sysc,      # system calls
     :ioch,      # chars read and written
     :path,      # array of symbolic link paths from /proc/<pid>/

pid
:contracts, # array symbolic link paths from /proc//
contracts
:fd, # array of used file descriptors
:cmd_args, # array of command line arguments
:environ # hash of environment associated with the process
]

  public

  ProcTableStruct = Struct.new("ProcTableStruct", *FIELDS)

  # In block form, yields a ProcTableStruct for each process entry

that you
# have rights to. This method returns an array of
ProcTableStruct’s in
# non-block form.
#
# If a +pid+ is provided, then only a single ProcTableStruct is
yielded or
# returned, or nil if no process information is found for that
+pid+.
#
# Example:
#
# # Iterate over all processes
# ProcTable.ps do |proc_info|
# p proc_info
# end
#
# # Print process table information for only pid 1001
# p ProcTable.ps(1001)
#
def self.ps(pid = nil)
array = block_given? ? nil : []

     Dir.foreach("/proc") do |file|
        next if file =~ /\D/ # Skip non-numeric entries under /

proc

        # Only return information for a given pid, if provided
        if pid
           next unless file.to_i == pid
        end

        # Skip over any entries we don't have permissions to read
        begin
           psinfo = IO.read("/proc/#{file}/psinfo")
        rescue StandardError, Errno::EACCES
           next
        end

        struct = ProcTableStruct.new

        struct.flag   = psinfo[0,4].unpack("i")[0]  # pr_flag
        struct.nlwp   = psinfo[4,4].unpack("i")[0]  # pr_nlwp
        struct.pid    = psinfo[8,4].unpack("i")[0]  # pr_pid
        struct.ppid   = psinfo[12,4].unpack("i")[0] # pr_ppid
        struct.pgid   = psinfo[16,4].unpack("i")[0] # pr_pgid
        struct.sid    = psinfo[20,4].unpack("i")[0] # pr_sid
        struct.uid    = psinfo[24,4].unpack("i")[0] # pr_uid
        struct.euid   = psinfo[28,4].unpack("i")[0] # pr_euid
        struct.gid    = psinfo[32,4].unpack("i")[0] # pr_gid
        struct.egid   = psinfo[36,4].unpack("i")[0] # pr_egid
        struct.addr   = psinfo[40,4].unpack("L")[0] # pr_addr

        struct.size   = psinfo[44,4].unpack("L")[0] * 1024 #

pr_size
struct.rssize = psinfo[48,4].unpack(“L”)[0] * 1024 #
pr_rssize

        # skip pr_pad1

        # TODO: Convert this to a human readable string somehow
        struct.ttydev = psinfo[56,4].unpack("i")[0] # pr_ttydev

        # pr_pctcpu
        struct.pctcpu = (psinfo[60,2].unpack("S")[0] * 100).to_f /

0x8000

        # pr_pctmem
        struct.pctmem = (psinfo[62,2].unpack("S")[0] * 100).to_f /

0x8000

        struct.start = Time.at(psinfo[64,8].unpack("L")[0]) #

pr_start
struct.time = psinfo[72,8].unpack(“L”)[0] #
pr_time
struct.ctime = psinfo[80,8].unpack(“L”)[0] #
pr_ctime

        struct.fname  = psinfo[88,16].strip          # pr_fname
        struct.psargs = psinfo[104,80].strip         # pr_psargs
        struct.wstat  = psinfo[184,4].unpack("i")[0] # pr_wstat
        struct.argc   = psinfo[188,4].unpack("i")[0] # pr_argc
        struct.argv   = psinfo[192,4].unpack("L")[0] # pr_argv
        struct.envp   = psinfo[196,4].unpack("L")[0] # pr_envp
        struct.dmodel = psinfo[200,1].unpack("C")[0] # pr_dmodel

        # skip pr_pad2

        struct.taskid   = psinfo[204,4].unpack("i")[0] # pr_taskid
        struct.projid   = psinfo[208,4].unpack("i")[0] #

pr_projectid
struct.nzomb = psinfo[212,4].unpack(“i”)[0] # pr_nzomb
struct.poolid = psinfo[216,4].unpack(“i”)[0] # pr_poolid
struct.zoneid = psinfo[220,4].unpack(“i”)[0] # pr_zoneid
struct.contract = psinfo[224,4].unpack(“i”)[0] #
pr_contract

        # skip pr_filler

        ### lwpsinfo struct info

        # skip pr_flag

        struct.lwpid = psinfo[236,4].unpack("i")[0] # pr_lwpid

        # skip pr_addr

        struct.wchan   = psinfo[244,4].unpack("L")[0] # pr_wchan
        struct.stype   = psinfo[248,1].unpack("C")[0] # pr_stype
        struct.state   = psinfo[249,1].unpack("C")[0] # pr_state
        struct.sname   = psinfo[250,1]                # pr_sname
        struct.nice    = psinfo[251,1].unpack("C")[0] # pr_nice
        struct.syscall = psinfo[252,2].unpack("S")[0] # pr_syscall

        # skip pr_oldpri
        # skip pr_cpu

        struct.pri = psinfo[256,4].unpack("i")[0] # pr_syscall

        # skip pr_pctcpu
        # skip pr_pad
        # skip pr_start
        # skip pr_time

        struct.clname   = psinfo[280,8].strip          # pr_clname
        struct.name     = psinfo[288,16].strip         # pr_name
        struct.onpro    = psinfo[304,4].unpack("i")[0] # pr_onpro
        struct.bindpro  = psinfo[308,4].unpack("i")[0] #

pr_bindpro
struct.bindpset = psinfo[308,4].unpack(“i”)[0] #
pr_bindpset

        # Get the full command line out of /proc/<pid>/as.
        begin
           fd = File.open("/proc/#{file}/as")

           fd.sysseek(struct.argv, IO::SEEK_SET)
           address = fd.sysread(struct.argc * 4).unpack("L")[0]

           struct.cmd_args = []

           0.upto(struct.argc - 1){ |i|
              fd.sysseek(address, IO::SEEK_SET)
              data = fd.sysread(128)[/^[^\0]*/] # Null strip
              struct.cmd_args << data
              address += data.length + 1 # Add 1 for the space
           }

           # Get the environment hash associated with the process.
           struct.environ = {}

           fd.sysseek(struct.envp, IO::SEEK_SET)

           env_address = fd.sysread(128).unpack("L")[0]

           loop do
              fd.sysseek(env_address, IO::SEEK_SET)
              data = fd.sysread(1024)[/^[^\0]*/] # Null strip
              break if data.empty?
              key, value = data.split('=')
              struct.environ[key] = value
              env_address += data.length + 1 # Add 1 for the space
           end
        rescue Errno::EACCES, Errno::EOVERFLOW, EOFError
           # Skip this if we don't have proper permissions, if

there’s
# no associated environment, or if there’s a largefile
issue.
ensure
fd.close if fd
end

        ### struct prusage

        begin
           prusage = 0.chr * 512
           prusage = IO.read("/proc/#{file}/usage")

           # skip pr_lwpid
           struct.count    = prusage[4,4].unpack("i")[0]  #

pr_count
struct.tstamp = prusage[8,8].unpack(“L”)[0] #
pr_tstamp
struct.create = prusage[16,8].unpack(“L”)[0] #
pr_create
struct.term = prusage[24,8].unpack(“L”)[0] #
pr_term
struct.rtime = prusage[32,8].unpack(“L”)[0] #
pr_rtime
struct.utime = prusage[40,8].unpack(“L”)[0] #
pr_utime
struct.stime = prusage[48,8].unpack(“L”)[0] #
pr_stime
struct.ttime = prusage[56,8].unpack(“L”)[0] #
pr_ttime
struct.tftime = prusage[64,8].unpack(“L”)[0] #
pr_tftime
struct.dftime = prusage[72,8].unpack(“L”)[0] #
pr_dftime
struct.kftime = prusage[80,8].unpack(“L”)[0] #
pr_kftime
struct.ltime = prusage[88,8].unpack(“L”)[0] #
pr_ltime
struct.slptime = prusage[96,8].unpack(“L”)[0] #
pr_slptime
struct.wtime = prusage[104,8].unpack(“L”)[0] #
pr_wtime
struct.stoptime = prusage[112,8].unpack(“L”)[0] #
pr_stoptime
struct.minf = prusage[120,4].unpack(“L”)[0] #
pr_minf
struct.majf = prusage[124,4].unpack(“L”)[0] #
pr_majf
struct.nswap = prusage[128,4].unpack(“L”)[0] #
pr_nswap
struct.inblk = prusage[128,4].unpack(“L”)[0] #
pr_inblk
struct.oublk = prusage[128,4].unpack(“L”)[0] #
pr_oublk
struct.msnd = prusage[128,4].unpack(“L”)[0] #
pr_msnd
struct.mrcv = prusage[128,4].unpack(“L”)[0] #
pr_mrcv
struct.sigs = prusage[128,4].unpack(“L”)[0] #
pr_sigs
struct.vctx = prusage[128,4].unpack(“L”)[0] #
pr_vctx
struct.ictx = prusage[128,4].unpack(“L”)[0] #
pr_ictx
struct.sysc = prusage[128,4].unpack(“L”)[0] #
pr_sysc
struct.ioch = prusage[128,4].unpack(“L”)[0] #
pr_ioch
rescue Errno::EACCES
# Do nothing if we lack permissions. Just move on.
end

        # Information from /proc/<pid>/path. This is represented

as a hash,
# with the symbolic link name as the key, and the file it
links to
# as the value, or nil if it cannot be found.
#–
# Note that cwd information can be gathered from here,
too.
struct.path = {}

        Dir["/proc/#{file}/path/*"].each{ |entry|
           link = File.readlink(entry) rescue nil
           struct.path[File.basename(entry)] = link
        }

        # Information from /proc/<pid>/contracts. This is

represented as
# a hash, with the symbolic link name as the key, and the
file
# it links to as the value.
struct.contracts = {}

        Dir["/proc/#{file}/contracts/*"].each{ |entry|
           link = File.readlink(entry) rescue nil
           struct.contracts[File.basename(entry)] = link
        }

        # Information from /proc/<pid>/fd. This returns an array

of
# numeric file descriptors used by the process.
struct.fd = Dir["/proc/#{file}/fd/*"].map{ |f|
File.basename(f).to_i }

        if block_given?
           yield struct
        else
           array << struct
        end
     end

     pid ? array[0] : array
  end

end
end

Thanks,

Dan

  • I tried tossing threads at it in a one-thread-per-directory
    approach, but they didn’t get along with IO.read in MRI, and seemed to
    provide no real speed benefit with JRuby.

Anyway, below is the code. I ran it through the profiler, but the top
two most costly ops were Dir.foreach, which I don’t see any way to
optimize*, and the loop that gathers environment information, which I
again see no way to optimize.

Could you post your profiling? If you run using “time”, how much user
CPU versus system CPU are you using?

Have you tried using Dir.open.each instead of Dir["/foo/*"]? Maybe
globbing is expensive.

Your environment loop does a fresh sysread(1024) for each var=val pair,
even if you’ve only consumed (say) 7 bytes from the previous call. You
would make many fewer system calls if you read a big chunk and chopped
it up afterwards. This may also avoid non-byte-aligned reads.

I would also be tempted to write one long unpack instead of lots of
string slicing and unpacking. The overhead here may be negligible, but
the code may end up being smaller and simpler. e.g.

struct = ProcTableStruct.new(*psinfo.unpack(<<PATTERN))
i i i i
i i i i
i i L L
L x4i ss
…etc
PATTERN

Perhaps you could combine it with your struct building, e.g.

  FIELDS = [
     [:flag,"i"],      # process flags (deprecated)
     [:nlwp,"i"],      # number of active lwp's in the process
     ...
     [:size,"s"],      # size of process in kbytes
     [:rssize,"s"],    # resident set size in kbytes
     [nil,"X4"],       # skip pr_pad1
     ... etc

HTH,

Brian.

2008/9/16 Daniel B. [email protected]:

again see no way to optimize.
You can optimize the loop body. Profiler output takes a while to get
used to.

Few things that caught my attention:

You can combine the first two “next” in one. Also matching with the
RX on the left side is faster AFAIK.

Try to use as few psinfo.unpack as possible, i.e. ideally only 1.

Use the block form of File.open.

Do not use sysread/syswrite/sysseek unless you have to (most of the
time you don’t).

You can replace all but the first checks for block_given? with
“array”. Might be faster.

Have fun!

robert

Brian C. [email protected] wrote:

Your environment loop does a fresh sysread(1024) for each var=val pair,
even if you’ve only consumed (say) 7 bytes from the previous call. You
would make many fewer system calls if you read a big chunk and chopped
it up afterwards. This may also avoid non-byte-aligned reads.

I would also be tempted to write one long unpack instead of lots of
string slicing and unpacking. The overhead here may be negligible, but
the code may end up being smaller and simpler. e.g.

Method calls are expensive. Right now he’s got 3 per field (array
indexing, unpack, and assignment). He could get that down to 1 per
field (assignment only) pretty easily by following your suggestion.

Brian C. [email protected] wrote:

i i i i
[:nlwp,“i”], # number of active lwp’s in the process

[:size,“s”], # size of process in kbytes
[:rssize,“s”], # resident set size in kbytes
[nil,“X4”], # skip pr_pad1
… etc

HTH,

Brian.

Here’s a concept for metaprogramming that that I was able to generate
mostly by running regexes to transform the code. I’ve only tackled
/proc/#{file}/psinfo, but it should be fairly simple to extend to
the other files as well

The Sys module serves as a namespace only.

module Sys

The ProcTable class encapsulates process table information.

class ProcTable

  # The version of the sys-proctable library
  VERSION = '0.8.0'

  private

  PRNODEV = -1 # non-existent device


  #Dissecting the format of this, we have a symbol mapping to an 

unpack format string segment
#the @ sign followed by a number indicates the offset in the
string, and the text following that number is the format
#of the data to unpack
FIELDS=[
[:flag , “@0 i”],
[:nlwp , “@4 i”],
[:pid , “@8 i”],
[:ppid , “@12 i”],
[:pgid , “@16 i”],
[:sid , “@20 i”],
[:uid , “@24 i”],
[:euid , “@28 i”],
[:gid , “@32 i”],
[:egid , “@36 i”],
[:addr , “@40 L”],
[:size , “@44 L”],
[:rssize , “@48 L”],
[:ttydev , “@56 i”],
[:pctcpu , “@60 S”],
[:pctmem , “@62 S”],
[:start , “@64 L”],
[:time , “@72 L”],
[:ctime , “@80 L”],
#note that the A format specifier automatically does what the #strip
method does
#so I don’t have to call .strip in the ps method
[:fname , “@88 A16”],
[:psargs , “@104 A80”],
[:wstat , “@184 i”],
[:argc , “@188 i”],
[:argv , “@192 L”],
[:envp , “@196 L”],
[:dmodel , “@200 C”],
[:taskid , “@204 i”],
[:projid , “@208 i”],
[:nzomb , “@212 i”],
[:poolid , “@216 i”],
[:zoneid , “@220 i”],
[:contract , “@224 i”],
[:lwpid , “@236 i”],
[:wchan , “@244 L”],
[:stype , “@248 C”],
[:state , “@249 C”],
[:sname , “@250 a1”],
[:nice , “@251 C”],
[:syscall , “@252 S”],
[:pri , “@256 i”],
[:clname , “@280 A8”],
[:name , “@288 A16”],
[:onpro , “@304 i”],
[:bindpro , “@308 i”],
[:bindpset , “@308 i”]
]

  field_names,format_strings=FIELDS.transpose

  eval <<-"end;"
    def first_pass_fill string
      struct=ProcTableStruct.new

      #{ field_names.collect{|x| "struct.#{x}"}.join(", ") } = 

string.unpack “#{format_strings.join ’ '}”
end
end;

  #repeat the above with a new array instead of FIELDS and a new 

method name
#for any other file you want to unpack this way

=begin
This eval will define a function with the following code. The arrays and
metaprogramming are just an easier way to manage the format string and
fieldnames that you can understand them when maintenence time comes
around.

    def first_pass_fill string
      struct=ProcTableStruct.new

      struct.flag, struct.nlwp, struct.pid, struct.ppid, 

struct.pgid,
struct.sid, struct.uid, struct.euid, struct.gid, struct.egid,
struct.addr,
struct.size, struct.rssize, struct.ttydev, struct.pctcpu,
struct.pctmem,
struct.start, struct.time, struct.ctime, struct.fname,
struct.psargs,
struct.wstat, struct.argc, struct.argv, struct.envp,
struct.dmodel,
struct.taskid, struct.projid, struct.nzomb, struct.poolid,
struct.zoneid,
struct.contract, struct.lwpid, struct.wchan, struct.stype,
struct.state,
struct.sname, struct.nice, struct.syscall, struct.pri,
struct.clname,
struct.name, struct.onpro, struct.bindpro, struct.bindpset =
string.unpack “@0
i @4 i @8 i @12 i @16 i @20 i @24 i @28 i @32 i @36 i @40 L
@44 L @48 L @56 i
@60 S @62 S @64 L @72 L @80 L @88 A16 @104 A80 @184 i @188 i
@192 L @196 L @200
C @204 i @208 i @212 i @216 i @220 i @224 i @236 i @244 L @248
C @249 C @250 a1
@251 C @252 S @256 i @280 A8 @288 A16 @304 i @308 i @308 i”
end
=end

  public

  ProcTableStruct = Struct.new("ProcTableStruct", *field_names)

  #if you have multiple files you're reading from with their field 

names
#in multiple different variables, you’ll want to replace
field_names
#with some array concatentation

  # In block form, yields a ProcTableStruct for each process entry 

that you
# have rights to. This method returns an array of
ProcTableStruct’s in
# non-block form.
#
# If a +pid+ is provided, then only a single ProcTableStruct is
yielded or
# returned, or nil if no process information is found for that
+pid+.
#
# Example:
#
# # Iterate over all processes
# ProcTable.ps do |proc_info|
# p proc_info
# end
#
# # Print process table information for only pid 1001
# p ProcTable.ps(1001)
#
def self.ps(pid = nil)
array = block_given? ? nil : []
Dir.foreach(“/proc”) do |file|
next if file =~ \D # Skip non-numeric entries under / proc

        # Only return information for a given pid, if provided
        if pid
           next unless file.to_i == pid
        end

        # Skip over any entries we don't have permissions to read
        begin
           psinfo = IO.read("/proc/#{file}/psinfo")
        rescue StandardError, Errno::EACCES
           next
        end


        #the first pass fill just gets the raw data and unpacks it
        struct = first_pass_fill psinfo
        #now we do the transformations we need on the few fields 

that need it
struct.pctcpu= (struct.pctcpu100).to_f / 0x8000
struct.pctmem= (struct.pctmem
100).to_f / 0x8000
struct.start=Time.at(struct.start)
#the fields that needed stripping were handled by unpack

        #repeat the above for other files that we need to deal with

        if block_given?
           yield struct
        else
           array << struct
        end
     end

     pid ? array[0] : array
  end

end
end