Forum: Ruby on Rails rails 3.1 assets:precompile deployment

18a4cfd669d998263f9824ed5efeab41?d=identicon&s=25 Michael Rigart (damick)
on 2011-09-09 12:19
Hi all,

I'm just deploying a  few rails apps after upgrading to rails 3.1. I
have implemented the asset pipeline and everything works ok.

I'm deploying using Capistrano with the pre fabricated deploy/assets
recipe.

The problem I encounter is that the rake task takes like forever to
precompile all my assets. I have to admit that there are a lot of assets
that need to be precompiled +300 (images / js / css ) , becouse they
include things like jquery ui, tiny mce, etc .

I didn't think this might cause a problem, because when I run the
assets:precompile rake task on my dev machine, all assets get
precompiles within a few seconds, while doing this during deployment, it
take almost more then 10minutes and the server load get really high!

I haven't done any custom changes in the way I implemented the asset
pipeline whatsoever.
Hoping that someone can clarify if there are differences between
precompiling in dev <-> prod, because I definitely need to optimize
this.

Thank you in advance
A9b3b6f3ea2e49d18d5bb14dbc0a1e13?d=identicon&s=25 Jim Morris (Guest)
on 2011-09-09 22:40
(Received via mailing list)
I experienced the same thing, although in my case it takes so long to
precompile on my server I don';t know if it ever finishes, waited 15
minutes
one time and it still had not finished.

My solution was to precompile locally which takes a few seconds, then
copy
the public/assets directory to my server. My cap task is...



    desc "deploy the precompiled assets"
    task :deploy_assets, :except => { :no_release => true } do
       run_locally("rake assets:clean && rake precompile")
       upload("public/assets", "#{release_path}/public/assets", :via =>
:scp, :recursive => true)
    end

Note I don't use the supplied assets:precompile due to a bug in it which
has
not yet been released but the task has see the rails bug lists for that.
30d8312a5dc254f7545b470c983be70b?d=identicon&s=25 Aldo Nievas (Guest)
on 2011-09-09 23:29
(Received via mailing list)
I run into the same problem.

add this in Capfile:

load 'deploy/assets'

it will automatically do the deploy_assets task ( assets:precompile )

it will create a link under public directory
assets -> $application_path/production/shared/assets/

for this to work you  must use:
gem 'rails'
gem 'sprockets', :git => 'git://github.com/sstephenson/sprockets.git'

and in environments/production.rb (otherwise it won't work )

#config.action_dispatch.x_sendfile_header = "X-Sendfile" # Use
'X-Accel-Redirect' for nginx
config.action_dispatch.x_sendfile_header = "X-Accel-Redirect"

config.assets.compile = true
config.assets.digest = true

Hope this helps.


-- Aldo Nievas
SATIO Internet Solutions
www.satio.com.ar
aldo@satio.com.ar
M: 54 11 3487 4900
18a4cfd669d998263f9824ed5efeab41?d=identicon&s=25 Michael Rigart (damick)
on 2011-09-10 12:15
@jim: precompiling them locally seems indeed the best way at the moment.
But I'm still curious what the reason is that precompiling on server
takes that long. I'm setting up a test so I can investigate it a bit
further. But thanks for the tip, it's a nice workaround.

@aldo: I also use the default capistrano recipe, so that can't be the
actual problem.  Can I ask why you include the sprockets gem through
source in your Gem file? sprockets should already be loaded through
rails, or is there a bugfix in the sprockets head that causes this
problem?
30d8312a5dc254f7545b470c983be70b?d=identicon&s=25 Aldo Nievas (Guest)
on 2011-09-10 16:44
(Received via mailing list)
@aldo: I also use the default capistrano recipe, so that can't be the
actual problem.  Can I ask why you include the sprockets gem through
source in your Gem file? sprockets should already be loaded through
rails, or is there a bugfix in the sprockets head that causes this
problem?

Yes there was a bug in the sprockets head so I should include gem
through
source as recommended in Rails issues list in github.

-- Aldo Nievas
SATIO Internet Solutions
www.satio.com.ar
aldo@satio.com.ar
M: 54 11 3487 4900
18a4cfd669d998263f9824ed5efeab41?d=identicon&s=25 Michael Rigart (damick)
on 2011-09-10 19:36
@aldo, thanks for the info. I'll look into it. But it seems that I don't
find any problems atm.

Meanwhile, I've been playing a bit around to see what happens during the
precompilation task. (I used the task from rails head for testing, not
the buggy one included in rails 3.1).

I have noticed the 2 loops:

      config.assets.precompile.each do |path|
        env.each_logical_path do |logical_path|
            ........
        end
      end

if first starts to loop over all files that need to be precompiled. My
setting was a bit different. I had 4 extra files that needed to be
precompiled. So that meant, 4 extra loops.

In the inner loop, it starts to loop over all the assets available.

First I thought this was the error. But it seemed like it only matches
files that are ok by the result of the first loop. Not that big of a
deal, the penalty for looping multiple times over the same asset list is
pretty small.

I started to calculte the time it took to run the task localy:

real  0m31.001s
user  0m28.357s
sys  0m2.330s

So I started to tinker a bit with the config.assets.precompile. I don't
use the application.css/js so they can be removed. So came up with this:

config.assets.precompile -= [/application.(css|js)$/]
config.assets.precompile += [/^(juggler|frontend).(css|js)$/]

Using this config setting, I was able to almost remove a whole second of
the precompilation time :p

real  0m30.606s
user  0m28.409s
sys  0m2.324s

What I did noticed, was that precompiling my biggest js manifes took a
lot af time. So I created a custom js compressor based on the jsmin gem
and used that for precompilation:

config.assets.js_compressor  = JsminTransformer.new

The result was tremendous:

real  0m22.895s
user  0m20.884s
sys  0m2.008s

It takes almost 8 seconds less to precompile using jsmin instead of
uglifier!

Since I was impressed, I retook a look at the 2 loops we encountered
earlier. It seems like env.each_logical_path always calls up the same
assets. But why do all the checks over again (see the sprockets base
class)? Either way you put it, it's a IO penalty, when you check for
files.

So I changed the rake task a bit. I call each_logical_path once and
store the result (an Enumerable object) in a variable that I'll loop:

asset_list = env.each_logical_path

config.assets.precompile.each do |path|
    asset_list.each do |logical_path|
        ....
     end
end

So I ran the altered precompile task again and got this result:

real  0m18.404s
user  0m17.211s
sys  0m1.158s

Again, almost 4 seconds off the clock!

As a final test, I wanted to see if limiting the asset list while the
script ran had some performance gain. So I tried to remove the assets
that where already precompiled:

asset_list = env.each_logical_path.to_a
remove_list = []
config.assets.precompile.each do |path|
    asset_list -= remove_list
    remove_list = []

    asset_list.each do |logical_path|
        ....

        if asset = env.find_asset(logical_path)
      ...

            remove_list << logical_path
        end
     end
end

The end result was a bit slim, but still:

real  0m17.772s
user  0m16.548s
sys  0m1.189s


Now that I altered the deploy recipe to use my altered precompile task,
everything seems to be in order, except the high cpu usage during the
rake task. But this is due to I/O I guess while copying the files
around.
At this point, I think I'll use my custom recipe, and precompile the
localy like Jim suggested, so that I can avoid the the high load on the
production machine.

At this point, I'm still curious of other findings that improve the
precompilation of assets. So any info is welcome.
E086e3dedc4876dd565174da32a65808?d=identicon&s=25 Donald Piret (Guest)
on 2011-10-01 16:34
(Received via mailing list)
I'm actually getting this same slowdown issue but both on my
production and local machines. The asset precompilation task takes
about 10 minutes to complete, with the same setup using jammit and
manually generating the hashes for the assets (using asset_hash) took
much less than a minute.
There's definitely something wrong somewhere but I haven't been able
to figure out what exactly is causing the slowdown.
Any hints on how I could debug this issue?
49e98c4a13a5203ecff5d2d0774f8991?d=identicon&s=25 Matthijs Langenberg (mlangenberg)
on 2011-10-13 16:34
I experience the same.
Running rake assets:precompile on my local machine takes about a minute.
Running it on my production machine (which is MUCH faster), takes
forever. Over 20 minutes.
Is there a specific Sprockets commit I can point to in my gemfile to
solve this issue? Otherwise I need to locally generate the assets and
then upload them to my server.
A9b3b6f3ea2e49d18d5bb14dbc0a1e13?d=identicon&s=25 Jim Morris (Guest)
on 2011-10-14 02:36
(Received via mailing list)
This issue seems to have been fixed with 3.1.1, at least for me I was
able
to deploy using capistrano's deploy/assets additions, it worked well,
and
the assets were compiled on the server pretty quickly. Whereas with
3.1.0 it
never completed the compile on the server.
49e98c4a13a5203ecff5d2d0774f8991?d=identicon&s=25 Matthijs Langenberg (mlangenberg)
on 2011-10-14 19:38
It must be linked to Capistrano.
I did a fresh checkout on the same server. No issues at all.
When I run 'rake assets:precompile' from a capistrano deployed release,
it takes 20 minutes. This is on the same server, using bundler.

@Jim I am also using Rails 3.1.1, so that can't be it.

Or it is caused by the directory structure used by capistrano. Or the
way bundler is used in capistrano is causing a difference in a gem
version.

I am not sure where to look anymore.
49e98c4a13a5203ecff5d2d0774f8991?d=identicon&s=25 Matthijs Langenberg (mlangenberg)
on 2011-10-15 01:12
*FINALLY!*

Okay, so when setting, config.assets.compress = true, this uses a
JavaScript (V8) runtime, right?
Somewhere in the compressor, your whole Rails app is being loaded
(multiple times!). In my case, this include features/**/*.rb
These Ruby files have some references to the constant Capybara. Because
I don't have the Capybara gem available on my server (it's in the 'test'
group in Gemfile), Rails will be doing its magic constant missing
trickery, which is crazy expensive.
This brought the total time to run the 'rake assets:precompile' task to
20 minutes! I could reproduce it locally by removing the Capybara gem
from my Gemfile.

I really hope Google will index this well, and nobody has to spend
another full day tracing this bug.
Now let's fix this!

By the way, here is some output from the process sampler:

1 rb_gc_mark  (in ruby) + 55  [0x100039d47]
    1457 Thread_33491: SamplerThread
      1457 thread_start  (in libsystem_c.dylib) + 13  [0x7fff8bdf8b75]
        1457 _pthread_start  (in libsystem_c.dylib) + 335
[0x7fff8bdf58bf]
          1457 _ZN2v88internalL11ThreadEntryEPv  (in v8.bundle) + 85
[0x102dd4d75]
            1457 v8::internal::SamplerThread::Run()  (in v8.bundle) + 55
[0x102dd55c7]
              1457
v8::internal::RuntimeProfilerRateLimiter::SuspendIfNecessary()  (in
v8.bundle) + 120  [0x102dedbc8]
                1457 semaphore_wait_trap  (in libsystem_kernel.dylib) +
10  [0x7fff8d2f96b6]

Total number in stack (recursive counted multiple, when >=5):
        27300       rb_gc_mark  (in ruby) + 190  [0x100039dce]
        12110       st_foreach  (in ruby) + 98  [0x1000878a2]
        11346       mark_entry  (in ruby) + 12  [0x100039eec]
        5576       gc_mark_children  (in ruby) + 124  [0x10003959c]
        5312       gc_mark_children  (in ruby) + 1778  [0x100039c12]
        5181       gc_mark_children  (in ruby) + 1524  [0x100039b14]
        4558       gc_mark_children  (in ruby) + 1769  [0x100039c09]
        4017       rb_call  (in ruby) + 188  [0x1000237ac]
        3100       rb_call0  (in ruby) + 3267  [0x1000233c3]
        2648       eval_call  (in ruby) + 347  [0x10002d95b]
        2186       gc_mark_children  (in ruby) + 1386  [0x100039a8a]
        1461       rb_eval  (in ruby) + 525  [0x10001e95d]
        1257       rb_yield_0  (in ruby) + 1135  [0x10002174f]
        1014       eval_iter  (in ruby) + 1029  [0x10002e1c5]
        1010       eval_fcall  (in ruby) + 330  [0x10002d69a]
        926       rb_call0  (in ruby) + 3436  [0x10002346c]
        775       gc_mark_children  (in ruby) + 1796  [0x100039c24]
        748       mark_keyvalue  (in ruby) + 25  [0x100039ec9]
        496       blk_mark  (in ruby) + 50  [0x100017622]
        495       gc_mark_children  (in ruby) + 1993  [0x100039ce9]
        480       rb_ary_each  (in ruby) + 46  [0x1000015ce]
        430       rb_eval  (in ruby) + 1464  [0x10001ed08]
        381       gc_mark_children  (in ruby) + 1539  [0x100039b23]
        357       blk_mark  (in ruby) + 76  [0x10001763c]
        357       rb_eval  (in ruby) + 3331  [0x10001f453]
        335       gc_mark_children  (in ruby) + 0  [0x100039520]
        294       rb_gc_mark  (in ruby) + 0  [0x100039d10]
        293       eval_call  (in ruby) + 461  [0x10002d9cd]
        274       eval_rescue  (in ruby) + 311  [0x100024b87]
        273       rb_eval  (in ruby) + 853  [0x10001eaa5]
        272       proc_invoke  (in ruby) + 1181  [0x1000224fd]
        270       gc_mark_children  (in ruby) + 1515  [0x100039b0b]
        264       block_pass  (in ruby) + 868  [0x100029494]
        254       eval_call  (in ruby) + 114  [0x10002d872]
        253       blk_mark  (in ruby) + 97  [0x100017651]
        174       eval_while  (in ruby) + 268  [0x1000207fc]
        174       garbage_collect  (in ruby) + 79  [0x10003b00f]
        174       garbage_collect_0  (in ruby) + 200  [0x10003a138]
        174       rb_str_resize  (in ruby) + 94  [0x10008e00e]
        174       ruby_xrealloc  (in ruby) + 215  [0x10003b9c7]
        174       strio_extend  (in stringio.bundle) + 60  [0x1001f190c]
stringio.c:757
        174       strio_putc  (in stringio.bundle) + 184  [0x1001f2cb8]
stringio.c:1135
        166       rb_eval  (in ruby) + 1901  [0x10001eebd]
        165       eval_slit  (in ruby) + 81  [0x100024cb1]
        165       rb_eval  (in ruby) + 1987  [0x10001ef13]
        143       rb_eval  (in ruby) + 3144  [0x10001f398]
        141       rb_eval  (in ruby) + 2205  [0x10001efed]
        133       st_foreach  (in ruby) + 0  [0x100087840]
        132       rb_eval  (in ruby) + 3073  [0x10001f351]
        117       eval_attrasgn  (in ruby) + 397  [0x10002dc4d]
        99       eval_ensure  (in ruby) + 231  [0x1000204a7]
        53       gc_mark_children  (in ruby) + 642  [0x1000397a2]
        49       rb_mark_generic_ivar  (in ruby) + 57  [0x100099639]
        37       rb_fast_mark_table_contains  (in ruby) + 0
[0x100036eb0]
        31       mark_keyvalue  (in ruby) + 17  [0x100039ec1]
        28       gc_mark_children  (in ruby) + 331  [0x10003966b]
        27       gc_mark_children  (in ruby) + 1556  [0x100039b34]
        24       rb_gc_mark  (in ruby) + 105  [0x100039d79]
        23       mark_source_filename  (in ruby) + 0  [0x100037000]
        22       rb_fast_mark_table_add  (in ruby) + 0  [0x100036e90]
        14       blk_mark  (in ruby) + 41  [0x100017619]
        13       gc_mark_children  (in ruby) + 1473  [0x100039ae1]
        12       gc_mark_children  (in ruby) + 1486  [0x100039aee]
        11       rb_fast_mark_table_add_filename  (in ruby) + 0
[0x100036ef0]
        10       blk_mark  (in ruby) + 121  [0x100017669]
        10       rb_gc_mark  (in ruby) + 118  [0x100039d86]
        8       rb_load  (in ruby) + 1476  [0x100030184]
        7       mark_entry  (in ruby) + 0  [0x100039ee0]
        7       rb_f_load  (in ruby) + 53  [0x1000307f5]
        5       blk_mark  (in ruby) + 0  [0x1000175f0]
        5       gc_mark_children  (in ruby) + 463  [0x1000396ef]

Sort by top of stack, same collapsed (when >= 5):
        semaphore_wait_trap  (in libsystem_kernel.dylib)        1457
        gc_mark_children  (in ruby)        399
        rb_gc_mark  (in ruby)        386
        rb_fast_mark_table_heap_contains  (in ruby)        224
        st_foreach  (in ruby)        222
        garbage_collect_0  (in ruby)        78
        rb_fast_mark_table_contains  (in ruby)        37
        mark_source_filename  (in ruby)        24
        rb_fast_mark_table_add  (in ruby)        22
        rb_fast_mark_table_heap_remove  (in ruby)        16
        rb_fast_mark_table_add_filename  (in ruby)        11
        mark_entry  (in ruby)        7
        blk_mark  (in ruby)        5
        rb_fast_mark_table_contains_filename  (in ruby)        5
A9b3b6f3ea2e49d18d5bb14dbc0a1e13?d=identicon&s=25 Jim Morris (Guest)
on 2011-10-15 07:45
(Received via mailing list)
good job, I  suggest you file this as a bug in the rails bug list so it
can
be fixed
D4bbfcd90239da7ed2ac00d417931fb7?d=identicon&s=25 Joseph C. (joseph_c)
on 2012-02-10 19:51
Ah, great to know what's causing the problem.

Were you suggesting that adding Capybara to the gemfile should reduce
the precompile time? I tried it and it still took 10minutes.
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.