Forum: JRuby bug in #map

I have run across a bug in Enumerable#map. It affects 1.5-dev but is 
okay in 1.4.0.

Unfortunately, I can't figure out how to reproduce it to a simple 
example. I'll show in pseudo-code what I'm doing and the incorrect 
behavior I am seeing. I hope that is enough to find the problem.

loop do
      ladders = @strategies.map { |strategy|  strategy.crank(bar) }
      if verbose && ladders.any? { |ladder| ladder.changed? }
        ladder = Ladder.consolidate ladders
        write ladder
      end

     # unrelated work
end

This code calls #crank on my strategy class and returns a Ladder object. 
Obviously, it should assign a new array to the +ladders+ variable each 
time. However, I see a situation where on one pass through the loop 
+ladders+ gets 2 Ladder objects assigned. The *next* time through one 
strategy goes away so #crank is called *once* and returns a single 
Ladder object. This time the +ladders+ variable is assigned [Ladder, 
nil] for some reason. This causes my code to crash because I don't check 
for nil (it should *never* return a nil).

I tried to "fix" this by calling #compact after the map operation but 
that crashes jruby with an exception [1] which proves to me it isn't my 
code.

I also tried to shine some light on this by creating another local +i+ 
and incrementing it inside the map block, but when I run it that way the 
block is never called. It's all very weird.

# map block never gets called!
      i = 0
      ladders = @strategies.map { |strategy|
        i += 1
        strategy.crank(bar)
      }
      puts "ladders.size [#{ladders.size}] i [#{i}]" # always prints 
'ladders.size [0] i [0]'
      if verbose && ladders.any? { |ladder| ladder.changed? }
        ladder = Ladder.consolidate ladders
        write ladder
      end

Any ideas?

cr

[1] http://gist.github.com/302710

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Feb 12, 2010, at 10:30 AM, Chuck Remes wrote:

>        ladder = Ladder.consolidate ladders
>        write ladder
>      end

Oops, this was my mistake. I reused the variable +i+ which caused this 
particular issue.

When I use a new local variable, ithe code prints:

ladders.size [2] j [1]

This proves the block is called only once yet it somehow has a length of 
2 and members [Ladder, nil] which isn't possible.

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Feb 12, 2010, at 10:30 AM, Chuck Remes wrote:

> 
>     # unrelated work
> end

I have a little more information on this problem, but unfortunately not 
enough to create a JIRA.

I reduced the code to a simple call to #each to expose the problem. (The 
+ladders+ array is still generated by a call to #map where I believe the 
fault originates.)

  ladders.each { |ladder| ladder.reset }

This was failing with the same error as the original. I then changed the 
code to check for nil in the block.

  ladders.each { |ladder| ladder.reset unless ladder.nil? }

This now prints a warning.

 warning: multiple values for a block parameter (0 for 1)

This indicates to me that there is something screwed up with the 
underlying array. Somehow #each is yielding NO ARGUMENT to the block 
which then raises this warning.

There's a bug here somewhere but for the life of me I cannot reduce it 
to a simple case.

cr



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi Chuck,

Do you have complete example for me to try?

Thanks,
  --Vladimir

On Sat, Mar 6, 2010 at 8:49 PM, Chuck Remes <cremes.devlist@mac.com> 
wrote:
>>        ladder = Ladder.consolidate ladders
>  ladders.each { |ladder| ladder.reset }
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Mar 6, 2010, at 2:08 PM, Vladimir Sizikov wrote:

> Hi Chuck,
> 
> Do you have complete example for me to try?


I wish I did. This happens inside of a much larger program. I have 
tried, without success, in reducing it to a small sample program.

I'll try again to reduce it. :(

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Fri, Feb 12, 2010 at 10:30 AM, Chuck Remes <cremes.devlist@mac.com> 
wrote:
> I have run across a bug in Enumerable#map. It affects 1.5-dev but is okay in 1.4.0.
>
> Unfortunately, I can't figure out how to reproduce it to a simple example. I'll show in pseudo-code what I'm doing and the incorrect behavior I am seeing. I hope that is enough to find the problem.

I'll take the case, Watson!

On your end, you should of course keep trying to get a reduced case,
but you could also help us a *lot* by bisecting to a specific
revision. Given either a reproducible case (which we could bisect) or
a specific revision (that you have bisected) we could have a fix in no
time at all.

And you should file a bug right now anyway since it *is* a regression.

> This code calls #crank on my strategy class and returns a Ladder object. Obviously, it should assign a new array to the +ladders+ variable each time. However, I see a situation where on one pass through the loop +ladders+ gets 2 Ladder objects assigned. The *next* time through one strategy goes away so #crank is called *once* and returns a single Ladder object. This time the +ladders+ variable is assigned [Ladder, nil] for some reason. This causes my code to crash because I don't check for nil (it should *never* return a nil).
I assume you've confirmed that #crank only gets called once and
@strategies only contains one element on that second pass. If we're
still creating a two-element array, that would certainly be a bug.

> I tried to "fix" this by calling #compact after the map operation but that crashes jruby with an exception [1] which proves to me it isn't my code.

Your line numbers don't line up with master, but looking at
compact_bang there's only a few things that could be null and cause
this:

* The values array (highly unlikely)
* One of the array elements (more likely, since it tries to call 
.isNil())

Is @strategies an Array or some other Enumerable collection?

There have been a few changes to RubyArray in the past couple months,
and some of them were against map/collect. There have been fewer
changes to Enumerable#map, though quite a few additions for 1.9
behavior. Are you running in 1.9 mode?

>        ladder = Ladder.consolidate ladders
>        write ladder
>      end

Now that's peculiar. What is @strategies.size at that point? If it's
empty, this would be expected behavior (of course).

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Sat, Mar 6, 2010 at 1:49 PM, Chuck Remes <cremes.devlist@mac.com> 
wrote:
> This now prints a warning.
>
>  warning: multiple values for a block parameter (0 for 1)
>
> This indicates to me that there is something screwed up with the underlying array. Somehow #each is yielding NO ARGUMENT to the block which then raises this warning.

This points more and more toward the array size being off (realSize
incorrect, allowing it to read pass the actual end of the array) or to
a null getting into the array's backing store.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email