Metaruby, BFTS, Cardinal and Rubicon - State of play?

Austin Z. wrote:

Look. We know that the shootout is crap. We’ve known this for three
benchmarks, most other people don’t either (see the "For
* [email protected] * You are in a maze of twisty little passages, all alike. // halo • statue
* [email protected]

I just read the comment you made on Pat Eyler’s blog.
You do seem to be engaged in a personal vendetta.
That’s a shame.

On 9/15/06, Isaac G. [email protected] wrote:

Look. We know that the shootout is crap. We’ve known this for three
years now. But we still have people come in and use it for a variety
of reasons, most of which are completely bogus.
I just read the comment you made on Pat Eyler’s blog.
You do seem to be engaged in a personal vendetta.
That’s a shame.

It wouldn’t be necessary if you made the changes that I recommended
two years ago, so that people take what you’ve done a hell of a lot
less seriously, or that you take it a lot more seriously and do some
rigorous analysis and reject invalid submissions.

As it stands, you refuse to do either. So, yes, I will bring up the
fact that the shootout is a “silly place” every time anyone refers to
it for serious matters. It’s not worth looking at because you pimp it
yet refuse to take responsibility for ensuring quality results.

The shootout is utterly worthless, Ed Borasky’s opinions
notwithstanding. To make it useful, you need to start weeding out bad
implementations and bad assumptions, which you have yet to do despite
being told about this problem at least two years ago.

-austin

Austin Z. wrote:

vendetta.
rigorous analysis and reject invalid submissions.
Austin, you’ve stepped from criticism to personal vendetta.

You make abusive comments without any reference to what is actually
shown on the shootout website. In a similar tirade, six months ago, you
complained about a benchmark that, even then, hadn’t been shown in the
shootout for at least 6 months.

And now once again your making abusive comments based on your memories
about something that no longer exists
“(There’s a Perl example I looked at a couple of years ago …”

Where exactly is that Perl program shown in the current benchmarks?
Here’s the link http://shootout.alioth.debian.org/gp4/

On 9/15/06, Austin Z. [email protected] wrote:

fact that the shootout is a “silly place” every time anyone refers to
it for serious matters. It’s not worth looking at because you pimp it
yet refuse to take responsibility for ensuring quality results.

Austin, I hear you. I think many people on this list hear you too. I
bet most of us agree with you. But this is incredibly off topic, and
if you must go on about it, please do so somewhere more suitable.

I fear your message is slowly getting lost in the amount of noise you
create each time this comes up.

On 9/15/06, Isaac G. [email protected] wrote:

Austin, you’ve stepped from criticism to personal vendetta.

Not in the least. You demonstrated then that you were wholly
uninterested in being responsible.

You make abusive comments without any reference to what is actually
shown on the shootout website. In a similar tirade, six months ago, you
complained about a benchmark that, even then, hadn’t been shown in the
shootout for at least 6 months.

Not shown?

You do prominently link these from the shootout home page.

Where exactly is that Perl program shown in the current benchmarks?
Here’s the link http://shootout.alioth.debian.org/gp4/

http://shootout.alioth.debian.org/old/benchmark.php?test=ackermann&lang=perl&id=3

You may have changed the set of benchmarks. I’ll do some research now
and see if you’ve actually cleaned up your methodology. I doubt it,
but if you have, I’ll publicly post an apology. If you haven’t, you’ll
clean up your methodology. Otherwise, I’ll update my criticisms, but
you won’t get anything positive from me about the shootout.

It will be about two weeks before I can look at this, but I will do so.

Do we have an agreement there?

-austin

Austin Z. wrote:

Not shown?

You do prominently link these from the shootout home page.

Where exactly is that Perl program shown in the current benchmarks?
Here’s the link http://shootout.alioth.debian.org/gp4/

http://shootout.alioth.debian.org/old/benchmark.php?test=ackermann&lang=perl&id=3

So you couldn’t find that Perl program in the current benchmarks.

We do still show “the obsolete 2001 Doug Bagley Benchmarks”, and we
do prominently label them “the obsolete 2001 Doug Bagley Benchmarks”,
and of course they were not referred to by Kevin T…

You may have changed the set of benchmarks.

There’s no /may/ - the benchmarks were changed a year ago.

I’ll do some research now
and see if you’ve actually cleaned up your methodology. I doubt it,
but if you have, I’ll publicly post an apology. If you haven’t, you’ll
clean up your methodology. Otherwise, I’ll update my criticisms, but
you won’t get anything positive from me about the shootout.

It will be about two weeks before I can look at this, but I will do so.

Do we have an agreement there?

Of course not.

Your opinion of the shootout “methodology” is no more than your opinion
of the shootout “methodology”. When you have up-to-date information,
argue the merits of your opinion.

Gregory B. wrote:

create each time this comes up.
Here’s somewhere more suitable
http://alioth.debian.org/forum/forum.php?forum_id=999

On 9/15/06, Isaac G. [email protected] wrote:

http://shootout.alioth.debian.org/old/benchmark.php?test=ackermann&lang=perl&id=3
So you couldn’t find that Perl program in the current benchmarks.

We do still show “the obsolete 2001 Doug Bagley Benchmarks”, and we
do prominently label them “the obsolete 2001 Doug Bagley Benchmarks”,
and of course they were not referred to by Kevin T…

This is not the 2001 benchmark that we’re talking about, Mr. Gouy.
This is the one that you had that replaced the 2001 benchmark.

You may have changed the set of benchmarks.
There’s no /may/ - the benchmarks were changed a year ago.

You snipped the context in your split of the paragraph. The
implication here is that I don’t think you’ve changed your
methodology, which is what the problem was. Not the benchmarks
themselves.

Of course not.
Then I see no reason to research the issue aside from my own sense of
honour. I’ll do it, but you have confirmed for me that I’ll find
exactly what I’m expecting to find and that you haven’t done anything
to improve your methodology, so you’ll be getting no apology unless I
find that – as unlikely as it appears to be – you have actually
changed the methodology you use.

Cluebat: your methodology is and always has been the problem. Until
that improves, the shootout remains completely useless.

-austin

I’ve changed the subject line - my apologies for not having done so
sooner.

Austin Z. wrote:

On 9/15/06, Isaac G. [email protected] wrote:

http://shootout.alioth.debian.org/old/benchmark.php?test=ackermann&lang=perl&id=3
So you couldn’t find that Perl program in the current benchmarks.

We do still show “the obsolete 2001 Doug Bagley Benchmarks”, and we
do prominently label them “the obsolete 2001 Doug Bagley Benchmarks”,
and of course they were not referred to by Kevin T…

This is not the 2001 benchmark that we’re talking about, Mr. Gouy.
This is the one that you had that replaced the 2001 benchmark.

I don’t know what you mean.
Those “obsolete 2001 Doug Bagley Benchmarks” /are/ the same tasks that
appeared on Doug Bagley’s old Great Computer Language Shootout. That’s
simple enough to confirm, just check with the Wayback Machine archive
http://web.archive.org/web/20010124090400/http://www.bagley.org/~doug/shootout/

You may have changed the set of benchmarks.
There’s no /may/ - the benchmarks were changed a year ago.

You snipped the context in your split of the paragraph. The
implication here is that I don’t think you’ve changed your
methodology, which is what the problem was. Not the benchmarks
themselves.

No Austin, none of your posting was snipped, the whole posting was
shown.

that improves, the shootout remains completely useless.
Are you usually given a blank check? :slight_smile:

Anyway, I look forward to a diatribe against /the current/ Computer
Language Shootout.

As a practical matter

On 9/16/06, Isaac G. [email protected] wrote:

two years ago, so that people take what you’ve done a hell of a lot
if you must go on about it, please do so somewhere more suitable.

I fear your message is slowly getting lost in the amount of noise you
create each time this comes up.

Here’s somewhere more suitable
http://alioth.debian.org/forum/forum.php?forum_id=999

Although I have a strong opinion about the shoutout I think whoever will
not have the last word on this between Austin and Isaac will get a big
+100.

Robert


Deux choses sont infinies : l’univers et la bêtise humaine ; en ce qui
concerne l’univers, je n’en ai pas acquis la certitude absolue.

  • Albert Einstein