# Automatic question generator libs in Ruby Language

**URL:** <https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666>\
**Category:** Ruby\
**Created:** [February 24, 2011, 6:04pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666 "2011-02-24T18:04:18Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\
**Post date:** [February 24, 2011, 6:04pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/1 "2011-02-24T18:04:18Z")

</div>

is there any Automatic question generator libraries in Ruby Language ?

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [February 25, 2011, 11:01am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/2 "2011-02-25T11:01:14Z")

</div>

class QuestionGenerator  
def self.generate  
“Huh?”  
end  
end

On Thu, Feb 24, 2011 at 6:04 PM, Sniper A. \<

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [February 25, 2011, 3:51pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/3 "2011-02-25T15:51:59Z")

</div>

Sniper,

Could you be a little more specific?  
What is a “question generator”?  
Perhaps we could help you.

Best regards,  
Abinoam Jr.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [February 27, 2011, 4:19am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/4 "2011-02-27T04:19:05Z")

</div>

On Sat, Feb 26, 2011 at 6:55 PM, Sniper A.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> suppose if i have a paragraph (arround 250 words)
> 
> i want to get all the possible question from that paragraph

?! _all possible questions_ ?

“Why are there some many 3-character words in this language?”  
“Was this bozo drunk when he wrote this?”  
“Why doesn’t the outside part of an ‘e’ go all the way around?”  
“This is really – wait, where did I put my car keys?”

def all\_possible\_questions  
?  
end

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [February 28, 2011, 7:18pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/5 "2011-02-28T19:18:19Z")

</div>

i don’t know were hit the sauce,bozo you type the letter with an e  
awersome cookie

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [February 27, 2011, 3:55am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/6 "2011-02-27T03:55:46Z")

</div>

suppose if i have a paragraph (arround 250 words)

i want to get all the possible question from that paragraph

Thanks

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 11:57am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/7 "2011-03-01T11:57:09Z")

</div>

On Tue, 1 Mar 2011 19:31:36 +0900, Shadowfirebird wrote:

> ```
> ans << sn if sn ~= /\? $/
> end
> 
> return ans
> 
> ```
> 
> end
> 
> (That won’t cope with brackets.) Said the Vicar: “Or quotes.” But  
> it’s a starting point – assuming that that was what you meant.
> 
> Presumably someone smarter than me could do better with a single  
> regex.

def questions(para)  
para.scan(/([^!.]+??|.+?[!.])\w\*/m).flatten.select { |m| m[-1] == ??  
}.map &:strip  
end

ruby-1.9.2-p136 :063 \> str  
=\> “This is a sentence? Yes. And this too? Definitely!\n”  
ruby-1.9.2-p136 :064 \> questions(str)  
=\> [“This is a sentence?”, “And this too?”]

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 12:02pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/8 "2011-03-01T12:02:48Z")

</div>

On Tue, Mar 1, 2011 at 10:55 AM, Peter Z.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)wrote:

> 

This could be a long rabbit hole.

questions(“What does ‘?’ mean?”) #=\> [“What does '?”, “’ mean?”]

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 1:29pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/9 "2011-03-01T13:29:33Z")

</div>

On Tue, 1 Mar 2011 20:02:13 +0900, Adam P. wrote:

> This could be a long rabbit hole.

Well, you’ve started this. Now I shall show you what a rabbit hole is  
😉

#encoding:utf-8

# requires 1.9

QUESTION\_REGEXP = Regexp.compile(\<\<‘END’.strip, Regexp::EXTENDED |  
Regexp::MULTILINE)  
(?\g[!?.]?\s\*){0}(?\g[!?.]\s\*){0}(?[^!.?‘]+?){0}  
(?(?["’])\g\k\<pq+0\>\g|«\g»\g|\g\g|){0}  
\g  
END

def questions(para)  
questions =   
while para =~ QUESTION\_REGEXP  
para = $’  
questions \<\< $& if $&.strip[-1] == ??  
end  
questions  
end

TEXT = \<\<‘END’  
Nested ‘questions?’ are supported.  
This is a text. A question?  
This «kind?» is supported too, really?  
The quotes «“shall 'be” matching?'»?  
END

p questions(TEXT)

–  
WBR, Peter Z..

P.S. You definitely should run the code on a real ruby. It is  
somehow… curious.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 11:32am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/10 "2011-03-01T11:32:03Z")

</div>

> i want to get all the possible question from that paragraph

Do you mean that you want to extract all the sentences that end in a  
question mark?

Untested (so don’t laugh, all you Ruby grownups):

def questions(para)  
ans =   
para.split(/[.?!] /).each do |sn|  
ans \<\< sn if sn ~= /? $/  
end

```
return ans

```

end

(That won’t cope with brackets.) Said the Vicar: “Or quotes.” But  
it’s a starting point – assuming that that was what you meant.

Presumably someone smarter than me could do better with a single regex.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 2:16pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/11 "2011-03-01T14:16:04Z")

</div>

On Tue, Mar 1, 2011 at 12:28 PM, Peter Z.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)wrote:

> 

Hm!

opine = “You think ’ is an excellent grapheme?” # arguably this is a  
valid  
fully-formed question…

questions(opine) # endless loop!

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 2:44pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/12 "2011-03-01T14:44:22Z")

</div>

On Tue, 1 Mar 2011 22:15:45 +0900, Adam P. wrote:

> > is 😉  
> > questions(opine) # endless loop!  
> > A debug modification has accidentally slipped through. The regexp  
> > should have been defined this way:

QUESTION\_REGEXP = Regexp.compile(\<\<‘END’.strip, Regexp::EXTENDED |  
Regexp::MULTILINE)  
(?\g[!?.]?\s\*){0}(?\g[!?.]\s\*){0}(?[^!.?]+?){0}  
(?(?["'])\g\k\<pq+0\>\g|«\g»\g|\g\g|){0}  
\g  
END

(look at the last in first line: it has ’ removed).

I am aware of this problem. (I hadn’t written about it because of  
several reasons: I don’t know what exactly causes it, and I wanted to  
see if someone would trigger it — congratulations. Given the time this  
regexp executes even on a simple strings, the code is already not  
suited  
for production, and this small misinformation would not hurt the OP.)

I’ve triggered it several times while trying different forms of the  
regexp, and sometimes it occurs when using one of seemingly equivalent  
constructs, but does not occur with other one. A quick look at some  
backtraces taken at random times during the execution suggests that it  
really loops (and not just executes for a lot of time), but oniguruma  
is  
really huge and complex, and I don’t have enough time to debug this.

Also, as I am not a regular expression guru, it may contain blatant  
errors. (I’ve seen once a book on optimizing regexps. It was scary.)  
I’ve tried to use my experience with LALR parsers here, but sometimes  
oniguruma behaves in a completely different way. E.g. at the some point  
I’ve tried to replace an empty token at the end of (?…) group  
with a plain \g, which of course includes empty string, it would  
not compile due to _indefinite recursion_. It works perfectly with  
current variant. The (?…) token always was a terminal, of  
course.

The ideal way of accomplishing this task would be using a lexer and a  
parser; it is easy enough to write them manually in this case (as  
opposed to using a tool like RACC). Thus, one would create a finite  
state machine, which is what regular expression compiler does, too, but  
the former will be optimized properly.

I’ll leave this as a homework to someone else 🙂

–  
WBR, Peter Z..

P.S. Given that Oniguruma RE can compile a full LALR parser, I wonder  
how a regular expression matching a regular expression look like. And  
the amount of time and memory it will use.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 3:09pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/13 "2011-03-01T15:09:31Z")

</div>

> opine = “You think ’ is an excellent grapheme?” # arguably this is a valid  
> fully-formed question…
> 
> questions(opine) # endless loop!

So is: “What is the capital of Tunisia!”

Unless you can get Ruby to understand English, the parsing will always  
be imperfect.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 3:15pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/14 "2011-03-01T15:15:10Z")

</div>

On Tue, Mar 1, 2011 at 8:59 AM, Shadowfirebird  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)wrote:

> –  
> Majorities, of course, start with minorities.  
> – Robert Moses

But that’s not a question! It’s an exuberant and wrong statement about a  
fictitious city named “What”! (cue Abbott and Costello)

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 3:18pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/15 "2011-03-01T15:18:17Z")

</div>

On Tue, Mar 1, 2011 at 1:59 PM, Shadowfirebird  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)wrote:

> > opine = “You think ’ is an excellent grapheme?” # arguably this is a  
> > valid  
> > fully-formed question…
> 
> So is: “What is the capital of Tunisia!”
> 
> Unless you can get Ruby to understand English, the parsing will always be  
> imperfect.

Sure. Although my opine-variable question is more syntactic, I think, in  
question-ness than the semantic-focused Tunisia one. But yes, I agree;  
it’s  
really just about coverage of needs.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 5:04pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/16 "2011-03-01T17:04:16Z")

</div>

> But that’s not a question! It’s an exuberant and wrong statement about a  
> fictitious city named “What”! (cue Abbott and Costello)

Imagine a very angry teacher. And remember that you can’t end a  
sentence with ‘?!’.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 5:23pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/17 "2011-03-01T17:23:06Z")

</div>

On Tue, Mar 1, 2011 at 5:03 PM, Shadowfirebird  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > But that’s not a question! It’s an exuberant and wrong statement about a  
> > fictitious city named “What”! (cue Abbott and Costello)
> 
> Imagine a very angry teacher. And remember that you can’t end a sentence with  
> ‘?!’.

Even an angry teacher will state a question (a rising inflection on  
the last word/syllable of a sentence).

Punctuation reflects these changes in inflection/speech patterns (like  
periods signaling long breaks, as are common to end a sentence, comma  
signaling shorter pauses within a sentence which usually imply  
additional information about the preceding topic, and so on).

TL;DR: Parsing a natural language is _hard_.

–  
Phillip G.

Though the folk I have met,  
(Ah, how soon!) they forget  
When I’ve moved on to some other place,  
There may be one or two,  
When I’ve played and passed through,  
Who’ll remember my song or my face.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 1, 2011, 5:31pm UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/18 "2011-03-01T17:31:53Z")

</div>

On Wed, 2 Mar 2011 01:22:11 +0900, Phillip G. wrote:

> the last word/syllable of a sentence).
> 
> Punctuation reflects these changes in inflection/speech patterns  
> (like  
> periods signaling long breaks, as are common to end a sentence, comma  
> signaling shorter pauses within a sentence which usually imply  
> additional information about the preceding topic, and so on).
> 
> TL;DR: Parsing a natural language is _hard_.

Indeed, this is the second derivative of original question.

---

<div class="post-metadata">

**Author:** ![mokkai](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/c2a13f/32.png) [@mokkai](https://www.ruby-forum.com/u/mokkai)\
**Post date:** [March 2, 2011, 10:15am UTC](https://www.ruby-forum.com/t/automatic-question-generator-libs-in-ruby-language/203666/19 "2011-03-02T10:15:14Z")

</div>

> Indeed, this is the second derivative of original question.

ROFL – indeed. We don’t really know what the OP actually wants as  
yet. And if it’s difficult for _people_ to parse a question…!
