# Yet another text parser **URL:** \ **Category:** Ruby\ **Created:** [July 18, 2007, 10:49am UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720 "2007-07-18T10:49:08Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 10:49am UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/1 "2007-07-18T10:49:08Z")
Hi, I have yet another question about how to write a specific text parser in ruby… So, without further ado - this is what the source file looks like: Query= gi|23510597|emb|CAD48982.1| ring-infected erythrocyte surface antigen precursor [Plasmodium falciparum 3D7] (1085 letters) Database: KOG 112,920 sequences; 47,500,486 total letters Searching…done ``` Score ``` E Sequences producing significant alignments: (bits) Value At2g21510 96 3e-19 At4g39150 95 1e-18 At1g76700 and so on… What I want to do is the following: Read the source file - and if a line starts with “Query=”, strip everything from the line but the expression “gi|xxxxx”. That part was no problem with gsub, mind you. But, now the tricky thing (or not, I guess…). Go from there until you find a line starting with “Sequence”, skip this line and the following and puts the third line together with the “gi|xxxxx” So from the above example it would look like this: gi|23510597 At2g21510 No, ideally I wouldnt have to include this skip-lines part, but I cant find a regexp, that lets me reliably identify the first line of the results block (not all possible results start with At…). How I tried to do it: def stripname line s = line.gsub(/Query=/, ‘’) u = s.gsub(/|emb.\*/, ‘’) end count = 0 # initializing variables t = nil v = nil ARGF.each do |l| puts l unless count.zero? count = [0, count-1].max if l.match(/^Query=/) t = stripname l elsif l.match(/^Sequences/) l = $1 count = 2 puts “#{t}#{l}” else end end But the output looks terrible: gi|23510597 At2g21510 96 3e-19 gi|23510599 At5g14980 58 3e-08 gi|23510600 And no matter what I try, I cant get the gi|xxxx and the corresponding “best hit” in the same line. Tried it with hashes, but frankly dont know enough about those yet. So If anyone has a helpful comment or solution, I would be extremely grateful! Cheers, Marc ---
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 11:48am UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/2 "2007-07-18T11:48:48Z")
I’d throw it all into one big ugly regex: s.match(/Query= (.+?|.+?)|.+?(bits)\s+Value\s+(.+?)\s+/m).to\_a[1…2].join(’ ') =\> “gi|23510597 At2g21510” ---
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 12:51pm UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/3 "2007-07-18T12:51:21Z")
Andreas S. wrote: > I’d throw it all into one big ugly regex: > s.match(/Query= > (.+?|.+?)|.+?(bits)\s+Value\s+(.+?)\s+/m).to\_a[1…2].join(’ ') > =\> “gi|23510597 At2g21510” Thanks for the suggestion! However, if someone has a suggestion regarding the following code and how to fix it, I’d be happy…its almost working and I just need to understand why it is behaving a bit odd. So here is the code. def stripname line s = line.gsub(/Query=/, ‘’) u = s.gsub(/|emb.\*/, ‘’) end count = 0 gene = nil store = Array.new ARGF.each do |l| store.push(l) unless count.zero? count = [0, count-1].max if l.match(/^Query=/) gene = stripname l elsif l.match(/^Sequences/) count = 2 puts “#{gene.strip} #{store.last.to\_s.strip}” else end end Problem: Reads: If line is found that starts with “Query=”, use the method stripname on it and store it in the variable “gene”. Go further, and if you find a line that starts with “Sequence”, use the above specified procedure “count”. Now this is the problem right now. After I wasnt able to figure out to get the formatting right, I decided to stick to the skip-line approach and instead of having it printed, to store it in an array. From there I simply read the last entry. BUT: instead of printing every stored hit to the corresponding “gene”, it shifts the whole thing 1 line. So that each “gene” is associated with the “best hit” of the previous match to “Query=”. gi|23510597 gi|23510599 At2g21510 gi|23510600 At5g14980 Now, I could solve that easily with a capable text editor, but I think there must be an easy solution to this…right? Cheers, Marc ---
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 1:22pm UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/4 "2007-07-18T13:22:33Z")
> Now this is a Ruby ML, right, so maybe you would accept that I Rubyish > the code a little bit 😉 > > gi = nil > ARGF.each do |line| > case line > when /Query=\s\*(gi|.\*?)|/ > gi = $1 > when /Sequence/ > puts gi.strip \<\< " " \<\< (1…2).map{ ARGF.readline > }.last.strip > end > end Very nice, thank you! ---
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 1:23pm UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/5 "2007-07-18T13:23:33Z")
On 7/18/07, Marc H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Database: KOG > > everything from the line but the expression “gi|xxxxx”. That part was no > find a regexp, that lets me reliably identify the first line of the > t = nil > l = $1 > 96 3e-19 > gi|23510599 > > At5g14980 > 58 3e-08 > gi|23510600 > > And no matter what I try, I cant get the gi|xxxx and the corresponding > “best hit” in the same line. > It is a terrible thing happens to me all the time, one tends to forget > these \n’s. > Well fortunately we have #chomp, but maybe you want to use #strip > which removes trailing (and leading) WS \n included. HTH Robert ---
**Author:** ![lasastard](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/58956e/32.png) [@lasastard](https://www.ruby-forum.com/u/lasastard)\ **Post date:** [July 18, 2007, 1:12pm UTC](https://www.ruby-forum.com/t/yet-another-text-parser/109720/6 "2007-07-18T13:12:26Z")
On 7/18/07, Marc H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > ARGF.each do |l| > else > you find a line that starts with “Sequence”, use the above specified > procedure “count”. Now this is the problem right now. After I wasnt able > to figure out to get the formatting right, I decided to stick to the > skip-line approach and instead of having it printed, to store it in an > array. From there I simply read the last entry. > > BUT: instead of printing every stored hit to the corresponding “gene”, > it shifts the whole thing 1 line. So that each “gene” is associated with > the “best hit” of the previous match to “Query=”. > Are you pushing before or after you use the last element of the array? > But you should go back to your original idea, which works just fine, > now that you have discovered #strip, before my post 🙂 Now this is a Ruby ML, right, so maybe you would accept that I Rubyish the code a little bit 😉 gi = nil ARGF.each do |line| case line when /Query=\s\*(gi|.\*?)|/ gi = $1 when /Sequence/ puts gi.strip \<\< " " \<\< (1…2).map{ ARGF.readline }.last.strip end end HTH Robert