Let’s say I have the following threading in my Rails web application:
class MyController
def my_action
count = 0
arr = []
10.times do |i|
arr[i] = Thread.new {
sleep(rand(0)/10.0)
Thread.current["mycount"] = count
count += 1
}
end
arr.each {|t| t.join; print t["mycount"], ", " }
puts "count = #{count}"
end
end
As you can see, the ‘count’ variable is shared across all threads.
Now, what I want to do is share ‘count’ across multiple httpd
requests and allow my_action in MyController to have access to that
variable. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.
Using memcached, a database, and session variables is out of the
question. Ultimately ‘count’ will actually be a resource object…an FTP
connection.
Is this possible? Perhaps using Apache/Passenger workers like at
A way to access common FTP connection resource pool in Ruby across AJAX calls? - Stack Overflow?
Example code would be appreciated.
On 9 Mar 2011, at 17:47, Chad J. [email protected] wrote:
Thread.current["mycount"] = count
count += 1
}
end
This looks thread dangerous to me - i don’t think += is atomic.
variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.
Using memcached, a database, and session variables is out of the
question. Ultimately ‘count’ will actually be a resource object…an FTP
connection.
Is this possible?
I wouldn’t go down this road if I were you. For starters presumably the
objects in question will be modified by those workers, you’ll need to
synchronise access to them etc. You might also be painting yourself into
a corner for the day you need to scale beyond 1 server.
Personally, if I had to do this, I’d these objects in a separate
long-lived process (or processes) and have your controller actions talk
to these processes (exactly how depends on what you’ll be doing, you
might consider drb, message queues etc)
Fred
Frederick C. wrote in post #986539:
On 9 Mar 2011, at 17:47, Chad J. [email protected] wrote:
Thread.current["mycount"] = count
count += 1
}
end
This looks thread dangerous to me - i don’t think += is atomic.
variable**. For instance, maybe whatever spawns the ruby process to
serve httpd process could hold the variable count in its scope, and then
the ruby processes spawned for httpd processes could then access that
variable.
Just example code I pulled from someone online. I’d never actually use
this, and my stuff would be as thread-safe as possible.
Personally, if I had to do this, I’d these objects in a separate
long-lived process (or processes) and have your controller actions talk
to these processes (exactly how depends on what you’ll be doing, you
might consider drb, message queues etc)
You’re talking about something like a daemon to act as a proxy, and
I’d communicate with it via some protocol I define? That was my thought
too, but it’s extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.
So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?
So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?
Regardless of that: I think that passenger has a callback that is
executed before new worker instance is spawned. Maybe that could
somehow help you.
Robert Pankowecki
On 9 Mar 2011, at 19:34, Chad J. [email protected] wrote:
serve httpd process could hold the variable count in its scope, and then
You’re talking about something like a daemon to act as a proxy, and
I’d communicate with it via some protocol I define? That was my thought
too, but it’s extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.
I think you’d find that drb is very easy to use.
So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?
I doubt it - while with something like passenger your workers do share
initial resources (since they are forked from the application spawner
(at least by default)), that’s a one way thing: if one worker changes
the state of the object then that happens in that process only. You do
share file descriptors, but that’s more likely to be a nuisance - if two
workers try use the same socket as the same time, chaos will ensue
Fred
On Mar 9, 2:34pm, Chad J. [email protected] wrote:
variable**. For instance, maybe whatever spawns the ruby process to
might consider drb, message queues etc)
You’re talking about something like a daemon to act as a proxy, and
I’d communicate with it via some protocol I define? That was my thought
too, but it’s extra work, and I was trying to avoid spending the extra
time on this and just use the FTP connection directly.
So, regardless of whether this is a good or bad idea (I will assess that
later), is there any way to directly access a shared resource object?
I don’t think there’s any question this is a bad idea. For instance,
how are you planning on arbitrating the FTP connections to avoid
situations like this:
Process 1: cd foo
Process 2: cd bar
Process 1: put somefile.txt
Now somefile.txt is in the wrong place… There are similar issues
with keeping the connections alive, etc plus the total mess that
happens once you move beyond a single box.
In the Stackoverflow question, you mention that you’re only listing
directories. Maybe it would make more sense to write a fairly basic
proxy in a smaller framework (Sinatra maybe?) that just takes JSON
specifying what to list and spits back results? Such a server would
solve the issues mentioned above, and could even do basic caching.
–Matt J.