Forum: Ruby-core Solution for debugging files more than 8192 lines, building my own debugger.

Hi!

I’m trying to find a solution for a problem that origin in the fact that
the line number constant that can be fetched in Ruby not is the true
line number but the line number modulus 8192. (I’m running Ruby 1.8.7
and I am unfortunately stuck with that.)

I have made a simple ruby debugger for a customer that handles
breakpoints, step over, step in, step out, variable watch and
pausing/stopping running code.

Unfortunately I cannot share the code of the debugger since it belongs
to the customer and I also do not have access to the code any more since
my work there is done. (Actually since my assignment has ended, I’m just
hoping to fix this “8192-error” during my spare time out of pride, a
payment as good as any! :-) )

I will however be glad to share design ideas and also to answer any
questions if anyone else are facing problems of building a Ruby
debugger. I will write a section about the debugger further down that
may be interesting for anyone doing the same.


___Problem with large files___
For files larger than 8192 bytes, an incorrect line number is reported
in the set_trace_func proc. This makes it impossible to debug large
files, code stepping etc totally fails. (This has been verified as a
limitation by matz here:
http://www.velocityreviews.com/forums/t817552-erro...)

The customer that ordered the debugger gets huge files with settings in
Ruby code from an external provider. When a setup is incorrect (in
value, timing of order) it would be nice to be able to debug also these
huge files. I can think of 3 ways to solve this, but maybe there are
other ways as well.

1) Split the large files to smaller files. This must be done in runtime
and I assume that I could override the “require” or “include” keywords.
(All files are in source control so I cannot change the content of a
file and store it.) Drawback: This is so complex that it opens up for
bugs/errors.

2) Use a regular expression to compare the id and the classname trying
to determine what line among all the lines that are candidates. (If line
x is provided, all lines x+(n*8192) are candidates.) Drawback: It’s not
100% safe, wrong line could be suggested. Especially since the huge
files contains a lot of settings that uses the same Ruby command.

3) Go into the Ruby core code and see if I can make a clone that handles
bigger files. I brief description is posted in the forum thread
referenced above. Drawback: Requires update at all users since Ruby is
installed locally at the customer.

Any ideas about unforeseen drawbacks with the bullets above? Any ideas
of other solutions? Anyone better that the others? All input is
appreciated!



___About the debugger___
This is basically how I did the debugger. Please contact me if you are
building your own debugger and need some input!

The set_trace_func is used it interpret all code. Only line events are
analysed. A debug TCP-server is started (in a separate thread) this is
the glue layer between the debuggee and the GUI. Also the GUI is started
(in a separate thread).

Once set_trace_func  is invoked, the file, line and caller.length is
sent to a function that takes an early debug decision. First a blacklist
of files included in the debugger is checked. Then the current depth is
compared to the desired call stack depth. This is used for step over and
step out. Finally a compare to a whitelist is done. The whitelist
contains all files that contain breakpoints. Finally, the file and line
is compared to a list of breakpoints.

Once this is done, if the file must be debugged but no breakpoint is hit
the debuggee connects to the debug server and asks if it should continue
or not. If the reply is Yes, it continues, else it pauses. The pause is
created by a TCP server that is started by the debuggee (inside the
set_trace_func), called “stalled code server”. The stalled code server
gets requests from the debug server. It may get a stop request, and the
code is then halted. It may get a request of the local and global
variables and it may get a request of the call stack. Finally it may get
a request to continue.

If the debug server gets a request from the debuggee with a question
about whether  it should continue or not, and debug server answers NO,
it knows that a stalled code server is initiated. It then connects to
this server. It also sends a message to the GUI TCP server that the code
is now halted at FILE and LINE.

GUI presents this to the user and may send some different commands to
the debug server. Continue, Step, Step out, Break, Get variables, Get
call stack etc. If for example continue is requested by the user, GUI
sends “continue” to the debug server. Debug server sends “continue to
the debuggee. The debuggee closes the stalled code server and continues
execution.

That’s basically it! :-)

___Profiling___
Unfortunately the TCP server is quite slow in Ruby 1.8.7, especially to
start up and close down servers. (Stalled code server starts and closes
for every line since it cannot keep any data.) When I did my prototyping
and profiling of my prototype I used Ruby 1.9.3 that is much faster! In
ruby 1.9.3, the “Early Debug Decision” code that executes in the context
of the debuggee is not really needed and was added to minimize the TCP
communication between the debuggee and the debug server.

In Ruby 1.9.3, the time to step one line of code is about 15 ms, and to
execute one line was about 10 ms.

To “step one line of code” includes taking an early decision that the
code must be debugged and the answer is YES. It sends a request to the
debug server asking if it should continue, the answer is NO. Stalled
Code Server is started and code paused. The debug server sends a request
to GUI. GUI immediately answers “Step” to debug server. Debug server
connects to the stalled code server and sends a command to continue.
Stalled code server is closed and code continues.

To “execute one line” of code includes taking an early decision that the
code must be debugged and the answer is YES. It sends a request to the
debug server asking if it should continue, the answer is YES. Code
continues.

Best Regards,
Andreas Lundgren
firstname.lastname.x@gmail.com