Hi! I would like to extend nfdump in Ruby, which will be probably
challenge for me, because I have never write any extension.
What is core thing is, that nfdump reads binary file with huge amount of
data. This reading and all other job I want to do nfdump which is in C
and dont have library files, so I cant use ffi or something like that.
Nfdump reads file header, makes a structure represents a netflow file
and than it reads n flow records which is represented by struct in which
is another struct and variables.
I need to get this structure to Ruby.
My first question is, if its possible to get struct in which is other
struct etc. to Ruby.
Second is about approach. I have some problems with embedding Ruby
Interpreter in C- example from Programming in Ruby fails on SegFault. So
do you thing it is possible to write this as Ruby extension?
In the case of Ruby extension I didnt see in any example how to get to
Ruby struct which is inicialized in C. I cant inicialize it in Ruby
because its a lot of data and I dont know how many flows are in file. I
cant give structs to array because of memory problem and also I need to
work with record as soon as its read from the file by C.
Your suggestion are highly welcomed. Thanks in advance.
Hi Matěj P,
I might be able to offer a few starting points. I’ll just pre-warn you
that I’m not in any way an expert on this stuff!
On 19/01/12 08:53, Matěj P. wrote:
Hi! I would like to extend nfdump in Ruby, which will be probably
challenge for me, because I have never write any extension.
What is core thing is, that nfdump reads binary file with huge amount
of
data. This reading and all other job I want to do nfdump which is in
C
and dont have library files, so I cant use ffi or something like
that.
Nfdump reads file header, makes a structure represents a netflow file
and than it reads n flow records which is represented by struct in
which
is another struct and variables.
I need to get this structure to Ruby.
My first question is, if its possible to get struct in which is other
struct etc. to Ruby.
You can wrap up opaque data (eg. C structures) using Data_Wrap_Struct,
and it back using Data_Get_Struct. You can then define a number of calls
(using rb_define_method) to extract relevant values from your
structures.
Second is about approach. I have some problems with embedding Ruby
Interpreter in C- example from Programming in Ruby fails on SegFault.
I’ve personally found “Programming Ruby - The Pragmatic Programmers’
Guide”, in the chapter “Extending Ruby” to be the best source of
information on embedding Ruby. It’s not easy- in fact, I’ve embedded Lua
and Python and they were far easier to get working, but once I’d got it
going, I found Ruby to be more rewarding (since I vastly prefer it over
Lua and Python).
As for the seg fault, you might need to give a bit more info for someone
to help you. 
So
do you thing it is possible to write this as Ruby extension?
In the case of Ruby extension I didnt see in any example how to get
to
Ruby struct which is inicialized in C. I cant inicialize it in Ruby
because its a lot of data and I dont know how many flows are in file.
I
cant give structs to array because of memory problem and also I need
to
work with record as soon as its read from the file by C.
Your suggestion are highly welcomed. Thanks in advance.
Data_Wrap_Struct will probably solve your problem. You can create your
object in C, and then wrap it up in a Ruby object.
I hope this helps.
Garth
Thank you Garth for your answer.
I figured out the problem with segfault. It was caused by shared
parametr in make file.
I will try to embeded Ruby in C.
So your opinion is that Data_Wrap_Struct should work also for nested
structs? Do you have any experience with it?
I quite understand, what you wrote. When I wrap structure I have to
specify a Ruby class. If I understand well this Ruby class will wrap my
C struct so it should have same signature like my struct? How is it
inicialized?
I made a gist of my example: Embedding a Ruby Interpreter- working with struct · GitHub
What I want to do is only to make a struct representing address and in
Ruby code just print parameters of that struct…
Should I use:
API: Defining Structures
VALUE rb_struct_define( char *name, char *attribute…, NULL )
VALUE rb_struct_new( VALUE sClass, VALUE args…, NULL )
When Im embedding Ruby?
Hi Matěj P,
On 19/01/12 19:55, Matěj P. wrote:
Thank you Garth for your answer.
I figured out the problem with segfault. It was caused by shared
parametr in make file.
I will try to embeded Ruby in C.
So your opinion is that Data_Wrap_Struct should work also for nested
structs? Do you have any experience with it?
Data_Wrap_Struct essentially bundles up any C pointer into a Ruby
object. Thus anything that you can create a reference or pointer to, you
can use it for. Ruby on its own can’t do anything with it, but you can
supply methods that act upon it on the C side. Like any pointer though,
if you need to clean up the resource it points to, you have to make sure
that something is performing the required cleanup. So yes, it’s
perfectly fine for nested structures, if I understand you correctly.
For my own project, I have Data_Wrap_Struct in use under an abstraction
layer that I’ve written. The call that triggers Data_Wrap_Struct in my
code is used extensively, so in answer to your question, yes, I have
direct experience with it. I use it to manage a huge number of different
C++ objects, including appropriate cleanup, etc. Some of these objects
are highly complex- from the point of view of the Ruby code though, it’s
just a pointer to an opaque object. I have layered my own type
management system on top of that, but you may not need to do this in
your case. Once I’d gotten everything working correctly, it has been
quite stable.
Garth
Hi Matěj P,
The Ruby class you specify (as a Ruby “VALUE”) defines the class of the
object that is created. This object then carries around the pointer you
supplied it, which can be extracted at a later time. For me, the Ruby
class comes from the alloc call for the object, but you can create a
class with “rb_define_class” as well.
At this point I would definitely recommend writing a small, detached
program, and experimenting with it there, because honestly, some of this
stuff is really hard, and you can learn a lot from this
experimentation (this is what I did initially, reluctantly, and it
helped enormously). The book I mentioned before is useful, and has lots
of good examples, but failing that, Google searches with terms such as
“Ruby embed” or “Ruby extend” might give you something close. As a last
resort, the two ruby.h headers are good- search for the text of what you
are after and sometimes you turn up useful functions.
Garth
As you can see from my link on gist, I have that program for
experiments. And I own book you have mentioned.
The good point which I didnt realize is that probably I can wrap my
struct to Ruby class which I already have and I dont have to make
another one.
I tried to use API to make a struct and its working like a charm… But
its not a way I can (want to) you in the real, where I have predefined
structures in C, so I would like to wrap them.