Forum: Ruby-core [ruby-trunk - Feature #5903][Open] Optimize st_table (take 2)

419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (Guest)
on 2012-01-17 15:38
(Received via mailing list)
Issue #5903 has been reported by Yura Sokolov.

----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (Guest)
on 2012-01-20 13:49
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


I've updated pool_allocation patch to use more efficient pool algorithm.
No separate commit, I just rewrite branch.
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-01-30 18:19
(Received via mailing list)
Issue #5903 has been updated by Yukihiro Matsumoto.


I am positive about this patch.  Nobu, could you review the patch and
check it in unless you find any problem?

Matz.
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2012-01-30 23:08
(Received via mailing list)
Issue #5903 has been updated by Nobuyoshi Nakada.


What's "_black_magick"?
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-01-31 00:59
(Received via mailing list)
Hi,

I meant 83.patch.  I think 84.patch (pool allocation) requires more
discussion.

              matz.

In message "Re: [ruby-core:42277] [ruby-trunk - Feature #5903] Optimize
st_table (take 2)"
    on Tue, 31 Jan 2012 07:07:15 +0900, Nobuyoshi Nakada
<nobu@ruby-lang.org> writes:
|
|
|Issue #5903 has been updated by Nobuyoshi Nakada.
|
|
|What's "_black_magick"?
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-01-31 01:03
(Received via mailing list)
Hi,

In message "Re: [ruby-core:42279] Re: [ruby-trunk - Feature #5903]
Optimize st_table (take 2)"
    on Tue, 31 Jan 2012 08:59:06 +0900, Yukihiro Matsumoto
<matz@ruby-lang.org> writes:

|I meant 83.patch.  I think 84.patch (pool allocation) requires more
|discussion.

Oops, 83.patch was pool allocation.  I meant 84.patch.

              matz.
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-01-31 06:09
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


Nobuyoshi Nakada wrote:
> What's "_black_magick"?

On my computer, pool allocator work by 1% faster when I keep those two
assignment to this "magic" field.

May be it is a case of my computer, cause when I remove `heaps_freed`
from gc.c (`objspace->heap.freed`
http://bugs.ruby-lang.org/projects/ruby-trunk/repo...
), it start to work slower too.
But it is not ever assigned (cause it is initialized with zero, and
`last < heaps_freed` is never true
http://bugs.ruby-lang.org/projects/ruby-trunk/repo...
)
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-01-31 06:47
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


Hi,

I could work on questions about pool allocation.

Regards,

Yura.
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2012-02-01 05:05
(Received via mailing list)
Issue #5903 has been updated by Nobuyoshi Nakada.


Another question about packing.
Why are PKEY_POS and PVAL_POS from the tail?
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-02-01 07:23
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


Nobuyoshi Nakada wrote:
> Another question about packing.
> Why are PKEY_POS and PVAL_POS from the tail?

It allows hash values to be very close to each other, so that while loop
in `find_packed_index` runs through them very fast and does not touch
another cache line of cpu.
And only when it found equal hash it jumps to check key. This allows
searching in packed hash be even slightly faster than in not packed hash
of same size.

Initially I experiment with variable sized packed hashes, so that
`num_bins` is used and they goes from tail to avoid division by 3.
With fixed size this could be simplified.

I pushed a commit which places PKEY_POS and PVAL_POS after hashes, but
in forward order.

They could be placed altogether (like `i*3`, `i*3+1`, `i*3+2`).
`remove_packed_entry` should be changed accordantly. I think, this could
improve iteration over hash.
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-02-05 15:51
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


Table packing slows down `st_foreach` a bit, and GC suffers from it.
So that I add a `st_foreach_nocheck` to fix GC
https://github.com/funny-falcon/ruby/commit/84d08a...

----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-02-09 15:48
(Received via mailing list)
Issue #5903 has been updated by Yura Sokolov.


I've updated `table packing` patch to correlate with trunk
https://github.com/ruby/ruby/pull/84
https://github.com/ruby/ruby/pull/84.patch

Also there is additional commit which increases usage of
`st_foreach_nocheck`
https://github.com/funny-falcon/ruby/commit/95d6ddcb2b91
https://github.com/funny-falcon/ruby/commit/95d6dd...
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903

Author: Yura Sokolov
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-03-24 22:39
(Received via mailing list)
Issue #5903 has been updated by funny_falcon (Yura Sokolov).


As far this ticket not closed, I'll post hash related patch here:

https://github.com/ruby/ruby/pull/107
https://github.com/ruby/ruby/pull/107.patch

1. remove some unused code from st.c and hash.c
2. change rb_hash_modify to rb_hash_modify_check when st_table
allocation is not necessary
3. move part of safe iteration logic to st.c to make it clearer
   This is arguable change, cause it clearly do not have positive impact
on performance,
   but `make check` consumes 592.2 second before this change and 595.4
after - less than 1 percent,
   so that I suppose, difference is negligible.
4. introduce st_shift to optimize Hash#shift


----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903#change-25087

Author: funny_falcon (Yura Sokolov)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
419b54e4b0c805f8ed671451ea536e19?d=identicon&s=25 Yura Sokolov (funny_falcon)
on 2012-03-26 17:18
(Received via mailing list)
Issue #5903 has been updated by funny_falcon (Yura Sokolov).


Add couple of commits to pull request
https://github.com/ruby/ruby/pull/107 :

1. Removal of ST_CHECK .
   ST_CHECK were always returned instead of ST_CONTINUE inside of some
st_foreach loops.
   Now such calls to st_foreach are converted to calls to
st_foreach_check.
   So that, there is no reason to differentiate ST_CHECK and
ST_CONTINUE, which simplifies calling code a bit.
   Also, it allows to simplify st_foreach_check code.
2. Traditionally, ultrapacking for empty and one element tables
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903#change-25183

Author: funny_falcon (Yura Sokolov)
Status: Open
Priority: Normal
Assignee:
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-03-29 17:44
(Received via mailing list)
Issue #5903 has been updated by mame (Yusuke Endoh).

Status changed from Open to Assigned
Assignee set to nobu (Nobuyoshi Nakada)


----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903#change-25381

Author: funny_falcon (Yura Sokolov)
Status: Assigned
Priority: Normal
Assignee: nobu (Nobuyoshi Nakada)
Category: core
Target version: 2.0.0


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
F24ff61beb80aa5f13371aa22a35619c?d=identicon&s=25 mame (Yusuke Endoh) (Guest)
on 2012-11-24 05:09
(Received via mailing list)
Issue #5903 has been updated by mame (Yusuke Endoh).

Target version changed from 2.0.0 to next minor

I'll postpone the pool allocation to next minor.  Sorry.

--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #5903: Optimize st_table (take 2)
https://bugs.ruby-lang.org/issues/5903#change-33776

Author: funny_falcon (Yura Sokolov)
Status: Assigned
Priority: Normal
Assignee: nobu (Nobuyoshi Nakada)
Category: core
Target version: next minor


Given some of preparations to this patches already merged into
ruby-trunk,
I suggest patches for improving st_table second time (first were #5789):

1) Usage of packing for st_table of any kind, not only for numeric
hashes.

Most of hashes, allocated during page render in Rails are smaller than 6
entries.
In fact, during rendering "Issues" page of Redmine, 40% of hashes not
even grows
above 1 entry. They are small options hashes, passed to numerous helper
methods.

This patch packs hashes upto 6 entries in a way like numeric hashes from
trunk.
Also it pack hashes of size 0 and 1 into `st_table` inself, so that
there is no
need to allocate any "bins" at all.

https://github.com/ruby/ruby/pull/84.patch
https://github.com/ruby/ruby/pull/84

2) Usage of specialized pool for allocating st_table, st_table_entry
structures
and st_table.bins of smallest size (11)

Usage of specialized pool for this allocations give great speedup for
hash creation.
Also it gives countable reduction of memory consumption.

https://github.com/ruby/ruby/pull/83.patch
https://github.com/ruby/ruby/pull/83

First patch gives little overhead for creating hashes bigger than 6
entries when applied alone.
But both patches combined are not slower than ruby-trunk for hashes of
any size.

Performance testing is here https://gist.github.com/1626602
This topic is locked and can not be replied to.