Looking for thoughts and opinions on Ruport, and reporting i

[naething]

Hi Folks,

I announced Ruport 1.0 RC1 a few days ago, and a few folks here on
RubyTalk were really helpful about pointing out some issues.

Since we’ve only got a few weeks left before our scheduled ‘real’
release of 1.0, I’d like to get a bit of the RubyTalk effect on a few
questions. I’ll try to keep my pandering down to release
announcements and this thread, but I really value the input from this
list, which is why I’m asking.

Feel free to answer whatever questions you want from the below, I’m
also more than happy to just start a general discussion here about the
topic of writing reporting software in Ruby, as I think it’s still a
hard domain to tackle…

questions follow:

1. If you’ve used Ruport before, what version(s), and what was your
   experience with the software (good or bad).

2. If you’ve not used Ruport before, but know about it and have a
   reason for not using it, what is that reason?

3. If you didn’t know about it before, but have reporting needs, what
   are those needs?
   (I can help tell you if Ruport is a good fit)

4. If you’re a regular user, what have been the biggest pain points
   for you working with our software? What about the best stuff?

5. If you’re not a user yet, what can we do to get you interested?
   (Besides turn it into a video game or pay you in large bags of money.
   )

6. Anything on your wishlist for reporting related Ruby stuff?

7. Last year we did something called RuportDay, which is like
   RailsDay but with smaller prizes and less fun. It was actually a
   pretty interesting and productive day where devs and contributors and
   users could get together and hack / chat. We also did offer some
   small cash prizes. If we were able to organize this again this year,
   sometime shortly before or after 1.0, would you be interested in
   participating?

8. From a community-involvement perspective, any criticisms or praises
   regarding Ruport?

[naething]

On 4/27/07, Gregory B. [email protected] wrote:

5. If you’re not a user yet, what can we do to get you interested?

OK, I’ll bite.

Ruport is one of those projects that I’ve had on my list to look at
one of these days. I don’t think I have an immediate need for it, but
it sounded like the kind of thing that I might need some day.

So I just went to what I assume is the Ruport home page
(http://ruport.rubyforge.org/, which redirects to
http://stonecode.svnrepository.com/ruport/trac.cgi). Since I don’t
have any experience with Ruport, or with reporting toolsets in
general, I immediately started looking for a tutorial or something to
give me an idea of what this really is – a sort of “Hello, World!”
for Ruport.

I clicked through the “Tutorials, Examples and Articles” link, and see
tutorials for “ActsAsReportable” (which sounds Rails-oriented), a
“Rope Cheatsheet” (and I don’t yet know what Rope is) and a “Recipe
Book”. None of those sounded like what I’m looking for. I started
going through the presentation slides found elsewhere on the page, and
finally in the middle of the second presentation, I saw my first (and
only) screenshots of Ruport-generated reports (yay!). But I guess what
I would love to see on that page (or elsewhere) is a really basic,
“Ruport in 10 minutes” kind of tutorial, with even a trivial data set,
that walks you through the process of generating a report.

[naething]

On 4/27/07, Lyle J. [email protected] wrote:

On 4/27/07, Gregory B. [email protected] wrote:

“Rope Cheatsheet” (and I don’t yet know what Rope is) and a “Recipe
Book”. None of those sounded like what I’m looking for. I started
going through the presentation slides found elsewhere on the page, and
finally in the middle of the second presentation, I saw my first (and
only) screenshots of Ruport-generated reports (yay!). But I guess what
I would love to see on that page (or elsewhere) is a really basic,
“Ruport in 10 minutes” kind of tutorial, with even a trivial data set,
that walks you through the process of generating a report.

I have an example from yesterday I could probably expand a bit and add
commentary to, but you’re absolutely right, we need a quick tutorial.

http://pastie.caboo.se/56959

The trouble was the API was forever changing. That’s stabilized now.
Lets see if I can get us a Ruport in 10 minutes tutorial before RC2.

Thanks Lyle!

[naething]

On 4/27/07, Jamey C. [email protected] wrote:

I’ve been meaning to check out Ruport for ages, so, prompted by
Gregory’s request, I pulled up the webpage, and ran into the same issue
as Lyle did.

The webpage is worse-off-then-ever. We deleted everything that isn’t
directly relevant to the latest released code over last weekend.
We’re in the progress of putting this all back together, just as soon
as the API is redocumented, which is what I’ll be spending my day on
today.

One area where I have thought of using Ruport is in a Rails app for my
company that tracks all of our server info. I have a “Reports” page in
that app that allows the users to run various canned and ad-hoc reports
against the database. What I have done up to now is to provide the
query result set as a csv file that they can open in Excel.

I keep waiting for someone to tell me they need their report to be
formatted in a certain way, but, so far, everyone has been happy with
csv files.

This does often seem to be the case. Usually when that fails, I
change one word in my Ruport code to switch to HTML. If that doesn’t
work, I switch to PDF but then I don’t have CSS so I need to roll up
my sleeves a bit. (Getting Much Better though!)

If there was an in-depth tutorial showing how to use Ruport in Rails
from start to finish (i.e. query to output) that would be awesome.

I have a bunch of acts_as_reportable code starting to accumulate. The
trouble here is that I don’t really do any Rails at all, just a fair
bit of camping and standalone ActiveRecord. Mike Milner, our other
lead dev, is a Rails programmer but hasn’t quite dug deep into
integrating Ruport into his app yet.

Would camping examples be helpful to Rails users? I imagine they
wouldn’t be terribly hard to port over, as the Ruport code and the
ActiveRecord code and anything hinging on ActiveSupport would be the
same. If that’s the case, I’ll try to get some stuff out there.

I in no way want to imply anything negative about Ruport. I appreciate
Gregory’s hard work on this project, not to mention his endurance.
Also, I respect the fact that every post I have ever read of Gregory’s
has been very polite and respectful of our community. I have also
noticed that he goes out of his way to answer many newbie Ruby/Rails
questions that have nothing to do with Ruport.

Thank you for the kind words. We’ve had a lot of help, especially
from the other devs on the project, and recently very much from Mike.
However, it is a long, hard, project, so I really appreciate the
compliments.

As far as endurance goes… we see a light at the end of the tunnel,
just hoping it’s not a train.

warm regards,
-greg

[naething]

On Apr 27, 2007, at 10:59 PM, Gregory B. wrote:

This does often seem to be the case. Usually when that fails, I
change one word in my Ruport code to switch to HTML. If that doesn’t
work, I switch to PDF but then I don’t have CSS so I need to roll up
my sleeves a bit. (Getting Much Better though!)

What do you mean by “… I don’t have CSS…”??

[naething]

Lyle J. wrote:

So I just went to what I assume is the Ruport home page
Book". None of those sounded like what I’m looking for. I started
going through the presentation slides found elsewhere on the page, and
finally in the middle of the second presentation, I saw my first (and
only) screenshots of Ruport-generated reports (yay!). But I guess what
I would love to see on that page (or elsewhere) is a really basic,
“Ruport in 10 minutes” kind of tutorial, with even a trivial data set,
that walks you through the process of generating a report.

Damn! I take off my tinfoil hat for one minute, and Lyle ends up
reading my mind again!

I was going to post something along the same lines, but Lyle put it much
more elegantly than I would have.

I’ve been meaning to check out Ruport for ages, so, prompted by
Gregory’s request, I pulled up the webpage, and ran into the same issue
as Lyle did.

One area where I have thought of using Ruport is in a Rails app for my
company that tracks all of our server info. I have a “Reports” page in
that app that allows the users to run various canned and ad-hoc reports
against the database. What I have done up to now is to provide the
query result set as a csv file that they can open in Excel.

I keep waiting for someone to tell me they need their report to be
formatted in a certain way, but, so far, everyone has been happy with
csv files.

If there was an in-depth tutorial showing how to use Ruport in Rails
from start to finish (i.e. query to output) that would be awesome.

I in no way want to imply anything negative about Ruport. I appreciate
Gregory’s hard work on this project, not to mention his endurance.
Also, I respect the fact that every post I have ever read of Gregory’s
has been very polite and respectful of our community. I have also
noticed that he goes out of his way to answer many newbie Ruby/Rails
questions that have nothing to do with Ruport.

HTH,

Jamey

[naething]

On Apr 27, 2007, at 11:30 PM, Gregory B. wrote:

What do you mean by “… I don’t have CSS…”??
So we have methods like these:
axis

• @cursor@ - get the current location of the cursor on the y-axis
• @draw_text@ - places text at a specified position on the page

And it’s up to us to make it look pretty. Compare these two
formatters, to see what I mean:
Parked at Loopia

I got it. You mean you don’t have CSS available to you for PDF files?
Interesting. On OS X it is a built in facility to print to PDF.
CSS does itself present some formatting options for printing but
they’re not well implemented everywhere. The scaling options should
be sufficient to generate nicely printable documents generally. I’m
sure you know that though. but … When a boss asks for a pdf they
want a pdf.

[naething]

On 4/27/07, Gregory B. [email protected] wrote:

I have an example from yesterday I could probably expand a bit and add
commentary to, but you’re absolutely right, we need a quick tutorial.

Yes, that example looks like just about the right level of detail for
a quickstart tutorial. One thing I’d add (if it’s straightforward to
do so) is a demonstration of how to produce a nice HTML or PDF report
in addition to the plain-text report.

With regards to Jamey’s comments: I hope my previous message didn’t
come across as hostile or overly negative. I know you and the other
Ruport developers have put a lot of work into this, and now that I
know a little more about what Ruport can do, I’m already thinking
about ways we might be able to use it here at work. I think that by
adding just a little more newbie-oriented information on the home page
– whether it’s tutorials, images of sample reports, etc. – you’re
going to be able to hook a lot more people like me who were in the
dark about Ruport.

[naething]

On 4/27/07, John J. [email protected] wrote:

On Apr 27, 2007, at 10:59 PM, Gregory B. wrote:

This does often seem to be the case. Usually when that fails, I
change one word in my Ruport code to switch to HTML. If that doesn’t
work, I switch to PDF but then I don’t have CSS so I need to roll up
my sleeves a bit. (Getting Much Better though!)

What do you mean by “… I don’t have CSS…”??

Well the easiest way for me to make a formatted report from Ruport if
I get a ‘But can I make that blue’ comment is output vanilla HTML and
then tweak the CSS to do the formatting.

However, if we need printable reports, I need to move to PDF stuff.
Ruport’s support has gotten a lot better, we have a ton of helper
functions that provide in some cases simplified interfaces and in
other cases just delegators to PDF::Writer.

So we have methods like these:

• @add_text@ - adds text to your output
• @center_image_in_box@ - takes a path to an image file and centers it
  within boundaries you specify
• @rounded_text_box@ - draw text surrounded by a rounded-corner box
• @watermark@ - places a centered watermark on each page of your report
• @move_cursor@ - moves cursor specified number of units along the
  y-axis
• @move_cursor_to@ - moves cursor to a specified location on the y-axis
• @pad@ - adds a specified amount of space above and below some output
• @pad_top@ - adds a specified amount of space above some output
• @pad_bottom@ - adds a specified amount of space below some output
• @draw_table@ - uses PDF::SimpleTable to draw a table to output
• @horizontal_line@ - draw a horizontal line
• @vertical_line@ - draw a vertical line
• @left_boundary@ - get the left boundary of the page
• @right_boundary@ - get the right boundary of the page
• @top_boundary@ - get the top boundary of the page
• @bottom_boundary@ - get the bottom boundary of the page
• @cursor@ - get the current location of the cursor on the y-axis
• @draw_text@ - places text at a specified position on the page

And it’s up to us to make it look pretty. Compare these two
formatters, to see what I mean:
http://pastie.caboo.se/57062

[naething]

On 4/27/07, Gregory B. [email protected] wrote:

I have an example from yesterday I could probably expand a bit and add
commentary to, but you’re absolutely right, we need a quick tutorial.

Parked at Loopia

I actually just found a way to make that a whole lot shorter. (With
the help of JEG2)

He wrote the FCSV equivalent:
http://pastie.textmate.org/57068

And then I used Ruport’s slip-cover over FCSV to get the float
converters, and also used the lambda hash trick he did to get this:
http://pastie.caboo.se/57075

It’s going to depend on the job though. For straight CSV stuff, I
still use FCSV for its speed and simplicity… for more advanced
stuff, I use Ruport, often the more complicated stuff even.

I wrote a very weird post on the Ruport blog about this a while ago.
MVC with FCSV.
It’s here if anyone is interested. (Happy to get feedback, too)

http://blog.rubyreports.org/archives/41

[naething]

On 4/27/07, John J. [email protected] wrote:

I got it. You mean you don’t have CSS available to you for PDF files?

Correct. As far as I know there is no formattting facility for PDF
that reeks of cool, much less something standardized

Interesting. On OS X it is a built in facility to print to PDF.
CSS does itself present some formatting options for printing but
they’re not well implemented everywhere. The scaling options should
be sufficient to generate nicely printable documents generally. I’m
sure you know that though. but … When a boss asks for a pdf they
want a pdf.

Right, we need to generate PDFs and email them as part of reports
that run in the middle of the night…

Doing html → pdf is sometimes, but not always, a good solution.

[naething]

Right, we need to generate PDFs and email them as part of reports
that run in the middle of the night…

Doing html -> pdf is sometimes, but not always, a good solution.

Oh I hear you, especially for things to be printed overly pretty-
like, it sucks up a lot of overhead real fast.
Just spitting out a huge organization’s pay slips formatted for a dot-
matrix printer is a lot. Give 'em dots!!
Can we have a faux dot matrix printer filter?!

[naething]

On 4/27/07, Lyle J. [email protected] wrote:

On 4/27/07, Gregory B. [email protected] wrote:

I have an example from yesterday I could probably expand a bit and add
commentary to, but you’re absolutely right, we need a quick tutorial.

Yes, that example looks like just about the right level of detail for
a quickstart tutorial. One thing I’d add (if it’s straightforward to
do so) is a demonstration of how to produce a nice HTML or PDF report
in addition to the plain-text report.

a.to_html
a.to_pdf
a.to_csv

I will probably show how to make custom output for those though, as
you’re just going to get tables for each of them, which isn’t so
exciting.

With regards to Jamey’s comments: I hope my previous message didn’t
come across as hostile or overly negative.

I didn’t take it that way. Your points were spot on. It’s a common
concern, but by reminding us, it’ll force us to think about getting it
done.

I know you and the other
Ruport developers have put a lot of work into this, and now that I
know a little more about what Ruport can do, I’m already thinking
about ways we might be able to use it here at work.

That’s awesome! We tend to be isolationist (well not really), so if
you don’t hear much from us on RubyTalk, definitely drop by the list
(http://list.rubyreports.org) or #ruport on Freenode if you start
using it.

I think that by
adding just a little more newbie-oriented information on the home page
– whether it’s tutorials, images of sample reports, etc. – you’re
going to be able to hook a lot more people like me who were in the
dark about Ruport.

We were very afraid to do this when everything was changing so fast.
Now that things are getting a bit more polished, I think we’ll be able
to fill in that gap and hopefully help some folks.

[naething]

On 4/27/07, Mariusz Pêkala [email protected] wrote:

Maybe generating TeX and then PDF from it may be good solution?
TeX can do ‘styling’.

However, I don’t know anything about Ruport (yet) so forgive me if what I
said is stupid.

It’s not stupid. I just don’t think TeX styling is going to be any
higher level than our PDF::Writer wrappers, and it introduces a huge
toolchain. We at one point were doing TeX output for tables but
dropped it.

We’re pretty convinced that for now, the solution we have is the most
pragmatic one for simple support. We’ll leave it up to our users to
create custom formatters that wrap other solutions.
(I’ve wrapped htmldoc before)

I should have an example out soon enough that shows how to wrap rfpdf
with Ruport, but I have no idea how well that will go, or if it will
be worth it. The main appeal there is that it has better i18n
support, but I don’t see it replacing PDF::Writer in Ruport any time
soon. :-/

[naething]

want a pdf.

Right, we need to generate PDFs and email them as part of reports
that run in the middle of the night…

Doing html -> pdf is sometimes, but not always, a good solution.

Maybe generating TeX and then PDF from it may be good solution?
TeX can do ‘styling’.

However, I don’t know anything about Ruport (yet) so forgive me if what
I
said is stupid.

[naething]

On 4/27/07, John J. [email protected] wrote:

Naturally, it would be ideal if there were general intermediary data
format or document format.
The best bet is often XML since we’re talking about fundamentally
generated styled documents from a structured document, but of course
it all depends where that data is coming from and where it might go
and how much overhead is really practical for functionality needs
today and future functionality needs. Ideally if it is a valid xhtml
document, that should be more translatable to pdf.

I had very seriously considered integrating XML/Fo as I figured that’d
let people use heavy weight tools like FOP and possibly other stuff to
do complex rendering.

I scratched it off as ‘too much work’ to do between now and May 15th,
and the project I was planning on doing that depended on FOP got
pushed backwards. It’s possible that you might see this support in
Ruport eventually, or as a gem_plugin, or in ruport-util.

I’d likely accept a well written patch which generates XML/Fo for our
four standard renderers
(Row,Table,Group,Grouping) and put it directly in Ruport. So if
anyone out there is looking for a project, I’d be happy to help you
work on that…

[naething]

On Apr 28, 2007, at 12:14 AM, Gregory B. wrote:

be sufficient to generate nicely printable documents generally. I’m
sure you know that though. but … When a boss asks for a pdf they
want a pdf.

Right, we need to generate PDFs and email them as part of reports
that run in the middle of the night…

Doing html → pdf is sometimes, but not always, a good solution.

Naturally, it would be ideal if there were general intermediary data
format or document format.
The best bet is often XML since we’re talking about fundamentally
generated styled documents from a structured document, but of course
it all depends where that data is coming from and where it might go
and how much overhead is really practical for functionality needs
today and future functionality needs. Ideally if it is a valid xhtml
document, that should be more translatable to pdf.
There certainly is something to be said for generating at least
vanilla xml first, since it leaves you with something far more
malleable.
I don’t know the full spec on the various pdf specs (the multitude is
the problem at times) but I do know that there is a lot there in
common with xml/xhtml type documents.
Here are some links on just such things.

http://www.antennahouse.com/XSLsample/XSLsample.htm
http://www.w3.org/Style/XSL/
http://www.digitaljunkies.ca/dompdf/

The last link is to a PHP version of such a utility.
I know these are not reporting tools per se, but they could certainly
provide ideas!
I know many reporting tools are taking data straight out of a database.
But if it is tranformed first into an xml format, then it does become
highly reusable.
I’m big finger wagger with sites that offer info for download as a
pdf but not viewable online.

[naething]

(apologies for the resend, my subject was mysteriously stripped from my
first reply)

I’ve looked at Ruport before but as far as I can tell, it doesn’t really
try to solve the part of the problem I’m most interested in. Ruport
seems to tackle pulling data from a variety of sources in a unified way,
and rendering the output to a variety of formats, but doesn’t provide
any help for modelling the reports themselves.

Maybe some concrete examples might help clarify what I’m driving at.
I’ve writing an interactive system for generating reports on a variety
of different library things - circulation activity, public computer use,
etc. The range of the reports is user-defined - this month, the previous
quarter, etc. The domain (?) of the reports is as well - maybe
system-wide, maybe by region, or by branch, or by individual terminal or
computer. For graphs, the variable in question, etc.

I’ve taken a few stabs at writing a report model object hierarchy that
encapsulates all this, with varying levels of satisfaction and despair.
I guess this is the sort of thing I’d’ve hoped that ruport would help
with, even if only by example. I’ve been meaning to write to the ruport
list for advice about this for a while, but this is as good as
opportunity as any.

• donald

[naething]

Gregory B. wrote:

On 4/27/07, Mariusz P�kala [email protected] wrote:

Maybe generating TeX and then PDF from it may be good solution?
TeX can do ‘styling’.

However, I don’t know anything about Ruport (yet) so forgive me if what I
said is stupid.

It’s not stupid. I just don’t think TeX styling is going to be any
higher level than our PDF::Writer wrappers, and it introduces a huge
toolchain. We at one point were doing TeX output for tables but
dropped it.

Have you ever looked at tioga. It creates figures directly in pdf and
then uses tex for text formatting. It is very flexible and allows you to
create beautiful figures in pdf, so it might be something to look at.

http://www.kitp.ucsb.edu/~paxton/tioga.html

Edwin

[naething]

On 4/27/07, Ball, Donald A Jr (Library) [email protected]
wrote:

(apologies for the resend, my subject was mysteriously stripped from my
first reply)

I’ve looked at Ruport before but as far as I can tell, it doesn’t really
try to solve the part of the problem I’m most interested in. Ruport
seems to tackle pulling data from a variety of sources in a unified way,
and rendering the output to a variety of formats, but doesn’t provide
any help for modelling the reports themselves.

This is a real pain point for us for sure. We’re looking at our
Report class and seriously considering dropping it before 1.0, if we
can’t think of a way to make it fairly elegant.

Maybe some concrete examples might help clarify what I’m driving at.
I’ve writing an interactive system for generating reports on a variety
of different library things - circulation activity, public computer use,
etc. The range of the reports is user-defined - this month, the previous
quarter, etc. The domain (?) of the reports is as well - maybe
system-wide, maybe by region, or by branch, or by individual terminal or
computer. For graphs, the variable in question, etc.

So what we really need is a way for you to define a bunch of variable
featuresets, and then hook those up to the right data manipulations
and renderers, more or less.

I’ve been asking folks on the Ruport list for a syntax (even
imaginary, so long as it’s valid ruby) for doing this in a general
way, and I’ve unfortunately yet to see anything. Maybe folks on this
list will have some insight.

I tend to use camping as the primary platform for my Ruport work, so I
do all of my process definition in the controllers and helpers. In
command line reports, I do similar, but it’s a fair bit of Ruby and
little help from Ruport.

I want to make this better, but I need some concrete ideas for what
the interface would look like, because I’m yet to think of a good one
(after way too long)