# Synchronize between download and save in multi-threads?

**URL:** <https://www.ruby-forum.com/t/synchronize-between-download-and-save-in-multi-threads/193250>\
**Category:** Ruby\
**Created:** [August 14, 2010, 3:02am UTC](https://www.ruby-forum.com/t/synchronize-between-download-and-save-in-multi-threads/193250 "2010-08-14T03:02:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![luofeiyu](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/9de0a6/32.png) [@luofeiyu](https://www.ruby-forum.com/u/luofeiyu)\
**Post date:** [August 14, 2010, 3:02am UTC](https://www.ruby-forum.com/t/synchronize-between-download-and-save-in-multi-threads/193250/1 "2010-08-14T03:02:55Z")

</div>

i want to download all usa listed company historial data from yahoo  
finance,  
at first i download all compamy name list and save it in  
/home/pt/usastock/totalname(amex nyse nasdaq,6000 companies),here is my  
program to dowload their historial data from yahoo finance with  
multi-threads:  
require ‘rubygems’  
require ‘open-uri’  
threads = []  
puts time0=Time.now  
i=0  
j=0  
str=[]  
open(‘/home/pt/usastock/totalname’,‘r’){|item|  
while line=item.gets  
line=line.chomp  
i=i+1  
str\<\<“[http://ichart.finance.yahoo.com/table.csv?s=](http://ichart.finance.yahoo.com/table.csv?s=)”+line  
if i==10 or item.eof == true then  
begin  
for page\_to\_fetch in str  
threads\<\<Thread.new(page\_to\_fetch) do |url|  
open(url){|urlfile|  
datafile=“/home/pt/test/”+url.split(“=”)[1].to\_s  
open(datafile,‘w’){|file|  
file.puts urlfile.read  
j=j+1  
puts “#{url} #{j} is over”  
}}  
end  
end  
threads.each {|thr| thr.join}  
i=0  
str=[]  
threads = []  
rescue Timeout::Error =\> e  
rescue =\> e  
end  
else  
end  
end  
}  
puts Time.now-time0  
it can run seccessfully,when i change  
if i==10 or item.eof == true then (which means i open 10 threads)  
into  
if i==100 or item.eof == true then (which means i open 100  
threads)  
there is a problem,some data will not saved in my designated position,  
(datafile=“/home/pt/axel/”+url.split(“=”)[1].to\_s)  
some of them are saved such as “/home/pt/table.csv?s=YHOO”  
according to my program,it should be saved as “/home/pt/test/YHOO”  
i think that maybe it is a problem of synchronizimg between download  
and save  
in my program.  
would you mind to fix it?  
thinks.

---

<div class="post-metadata">

**Author:** ![luofeiyu](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/9de0a6/32.png) [@luofeiyu](https://www.ruby-forum.com/u/luofeiyu)\
**Post date:** [August 17, 2010, 2:23am UTC](https://www.ruby-forum.com/t/synchronize-between-download-and-save-in-multi-threads/193250/2 "2010-08-17T02:23:58Z")

</div>

> i think that maybe it is a problem of synchronizimg between download  
> and save  
> in my program.  
> would you mind to fix it?

The “Mutex” class might be useful to you.