Forum: Ruby Twitter Personalities (#208)

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
33117162fff8a9cf50544a604f60c045?d=identicon&s=25 Daniel X Moore (yahivin)
on 2009-06-05 22:57
(Received via mailing list)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

The three rules of Ruby Quiz:

1.  Please do not post any solutions or spoiler discussion for this
quiz until 48 hours have elapsed from the time this message was
sent.

2.  Support Ruby Quiz by submitting ideas and responses
as often as you can!
Visit: http://rubyquiz.strd6.com/suggestions

3.  Enjoy!

Suggestion:  A [QUIZ] in the subject of emails about the problem
helps everyone on Ruby Talk follow the discussion.  Please reply to
the original quiz message, if you can.

RSS Feed: http://rubyquiz.strd6.com/quizzes.rss

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

## Twitter Personalities (#208)

Merhaba Rubyists,

This week's quiz is to create a program that will generate messages
140 characters in length. There primary use will be to create a
Twitter "personality". At the end of the quiz period these
"personalities" will be unleashed on the internet and we'll see how
they do in the wild.

The programs will consist of two parts: a component for interacting
with Twitter, and a top secret "personality" module.

For the Twitter interface component there will be no no-spoiler
period. Please feel encouraged to discuss different libraries or
methods on the mailing list. Let's all work together to find the best
interface.

The "personality" component can take any inputs and will produce a 140
character message when called. The "personality" may remember state.
The no-spoiler period applies for the "personality" component; please
save them until everyone has had a chance to consider their own
implementations.


Have Fun!
282d4190e23301bbaa8a2343370a6c20?d=identicon&s=25 Joshua Collins (Guest)
on 2009-06-06 02:27
(Received via mailing list)
This Quiz sounds fun!

I wish  I knew more about Ruby than I do now. I would participate in
this
one, but I am a beginner rubyist :-(

Anyhow, I can not wait to see the results. Good luck to everyone
participating!


JC
4299e35bacef054df40583da2d51edea?d=identicon&s=25 James Gray (bbazzarrakk)
on 2009-06-06 06:25
(Received via mailing list)
On Jun 5, 2009, at 7:26 PM, Joshua Collins wrote:

> This Quiz sounds fun!
>
> I wish  I knew more about Ruby than I do now. I would participate in
> this one, but I am a beginner rubyist :-(
>
> Anyhow, I can not wait to see the results. Good luck to everyone
> participating!

This is an almost perfect quiz for a beginner!  Don't give up!

Use it as a chance to stretch your skills.

Try this:

1.  Build a funny saying in a String
2.  Replace some words in that saying with variables that hold the
words you removed
3.  Adjust each variable so it gets two or three different words that
all could fit the saying, selected randomly

It's just like reverse Madlibs.  You may get more ideas to keep
improving it too.

It's not a spoiler to get help on the Twitter side, or you can just
skip that part completely.  Don't be afraid to change the task to suit
your skill.  If you learn something, you win.

You can even ask us questions on the personality side, just be clever
in avoiding spoiler material.  "Hey guys, I'm building a new
navigation system for NASA's next generation rocket and it needs to
randomly choose between three directions…"  We're here to help.  :)

"Never give up!  Never surrender!"

James Edward Gray II
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2009-06-06 07:22
(Received via mailing list)
From: "James Gray" <james@grayproductions.net>
>
> "Never give up!  Never surrender!"

http://grabthar.ytmnd.com/

;D


SCNR,

Bill
4feed660d3728526797edeb4f0467384?d=identicon&s=25 Bill Kelly (Guest)
on 2009-06-06 07:30
(Received via mailing list)
From: "Daniel Moore" <yahivin@gmail.com>
>
> The "personality" component can take any inputs and will produce a 140
> character message when called. The "personality" may remember state.
> The no-spoiler period applies for the "personality" component; please
> save them until everyone has had a chance to consider their own
> implementations.

So... to clarify, the personality component is not replying
to--or seeing--other messages as input, but is merely
producing a sequence of 140 character messages determined
solely according to its internal black box state?


Regards,

Bill
6e366eb5a71be2bad7f383d42aeb4788?d=identicon&s=25 Justin Collins (Guest)
on 2009-06-06 09:33
(Received via mailing list)
Bill Kelly wrote:
> to--or seeing--other messages as input, but is merely
> producing a sequence of 140 character messages determined
> solely according to its internal black box state?
>
>
> Regards,
>
> Bill

I don't think that is a necessary restriction, as it "can take any
inputs." I will interpret that to mean you can feed in whatever you
would like - a dictionary or Shakespeare, for example.

-Justin
33117162fff8a9cf50544a604f60c045?d=identicon&s=25 Daniel X Moore (yahivin)
on 2009-06-06 18:44
(Received via mailing list)
Joshua,

James has some great suggestions and I couldn't have said it better.
One of the best ways to learn more about Ruby is to dive in. You may
find out that it is easier than you imagine. If you get stuck on a
particular issue you can search the web or ask on the mailing list;
there are many great resources and lots of people interested in
helping out.
33117162fff8a9cf50544a604f60c045?d=identicon&s=25 Daniel X Moore (yahivin)
on 2009-06-06 18:46
(Received via mailing list)
Bill,

The personality component can see other messages as input and use
those to determine the next message to provide. Of course, in order
for the component to see the replies and other messages they would
need to be gathered and passed in as arguments in some form.

If you like you can share the techniques for gathering these messages
with the mailing list. The goal is to make it as easy as possible to
focus on the "personality" part without worrying about the
nitty-gritty of interfacing with Twitter. Additionally sharing
information on available libraries or general techniques will help all
the solutions to be better.
282d4190e23301bbaa8a2343370a6c20?d=identicon&s=25 Joshua Collins (Guest)
on 2009-06-07 02:02
(Received via mailing list)
I will give the Quiz a shot. After all, I will surely learn from it :-)

Thanks for the encouragement guys!
0df4a6c75caf1bd9b01d2dcbfb085ee4?d=identicon&s=25 Sandro Paganotti (Guest)
on 2009-06-07 17:12
(Received via mailing list)
Wow !! Absolutely great quiz ! :D
1e736afdcdfe0753d67a81d449aca590?d=identicon&s=25 Srijayanth Sridhar (Guest)
on 2009-06-08 08:47
(Received via mailing list)
So the "personality" is just a random message, perhaps a Clint Eastwood
quote or something?

Jayanth

On Sun, Jun 7, 2009 at 8:42 PM, Sandro Paganotti
<sandro.paganotti@gmail.com
6e366eb5a71be2bad7f383d42aeb4788?d=identicon&s=25 Justin Collins (Guest)
on 2009-06-08 12:16
(Received via mailing list)
Daniel Moore wrote:
> Visit: http://rubyquiz.strd6.com/suggestions
>
> The programs will consist of two parts: a component for interacting
> save them until everyone has had a chance to consider their own
> implementations.
>
>
> Have Fun!
>

Posting to Twitter, I found, was pretty straightforward using the
Twitter4r gem, although it only worked for me under Ruby 1.9:

require 'twitter'
client = Twitter::Client.new :login => "myname", :password =>
"mypassword"
client.status :post, "Working on Ruby Quiz."


I hope that helps anyone who might have been hesitant because of the
Twitter interface part.

-Justin
4299e35bacef054df40583da2d51edea?d=identicon&s=25 James Gray (bbazzarrakk)
on 2009-06-08 16:46
(Received via mailing list)
On Jun 8, 2009, at 1:46 AM, Srijayanth Sridhar wrote:

> So the "personality" is just a random message, perhaps a Clint
> Eastwood
> quote or something?

It is if you want it to be.  :)

If you would prefer to be trickier, you could try to build intelligent
replies to things people say.

James Edward Gray II
0df4a6c75caf1bd9b01d2dcbfb085ee4?d=identicon&s=25 Sandro Paganotti (Guest)
on 2009-06-08 23:32
(Received via mailing list)
Ok, I've just finished mine... it was a really crazy experience... now
I'm
going to create a twitter account for it and generate some random
phrases
with different personalities :D
Aafa8848c4b764f080b1b31a51eab73d?d=identicon&s=25 Phlip (Guest)
on 2009-06-08 23:41
(Received via mailing list)
Sandro Paganotti wrote:
> [Note:  parts of this message were removed to make it a legal post.]
>
> Ok, I've just finished mine... it was a really crazy experience... now I'm
> going to create a twitter account for it and generate some random phrases
> with different personalities :D

You mean like "The Policeman's Beard is Half-Constructed"?
0df4a6c75caf1bd9b01d2dcbfb085ee4?d=identicon&s=25 Sandro Paganotti (Guest)
on 2009-06-08 23:53
(Received via mailing list)
Sort of :P (here's the bot account: https://twitter.com/twsapiens)

Sandro
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-06-09 00:03
(Received via mailing list)
Phlip wrote:
> You mean like "The Policeman's Beard is Half-Constructed"?

Racter ftw!
6e366eb5a71be2bad7f383d42aeb4788?d=identicon&s=25 Justin Collins (Guest)
on 2009-06-09 01:08
(Received via mailing list)
Justin Collins wrote:
>> as often as you can!
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>
>> The no-spoiler period applies for the "personality" component; please
> require 'twitter'
> client = Twitter::Client.new :login => "myname", :password =>
> "mypassword"
> client.status :post, "Working on Ruby Quiz."
>
>
> I hope that helps anyone who might have been hesitant because of the
> Twitter interface part.
>
> -Justin
>


So, here is my full solution, which just pulls transcriptions of
Dijkstra's notes and tries to extract decent quotes. Also attempts to
find relavent replies. Works most of the time. You can see some results
at http://twitter.com/ewdbot and get a nicer-looking version from
http://gist.github.com/125781

#A little Twitter thing to post random quotes from Edsger Dijkstra

require 'yaml'
require 'open-uri'
require 'twitter'  #twitter4r gem
require 'hpricot'

class DijkstraQuote
    class << self

        #Match the website's naming scheme
        def random_ewd
            (rand(1289) + 30).to_s.rjust(4, "0")
        end

        #Get a random quote
        def get_quote
            quote = nil
            while quote.nil? do
                quote = fetch random_ewd
            end

            quote
        end

        #Get a random quote from the specified set of notes
        def fetch ewd

            #Check if we've already retrieved this set of notes
            if File.exists? "ewd#{ewd}"
                quotes = YAML.load_file "ewd#{ewd}"
                quotes[rand quotes.length]
            else
                $stderr.puts "Fetching EWD#{ewd}..." if $DEBUG

                #Fetch a transcript of Dijkstra's notes
                begin
                    file =
open("http://www.cs.utexas.edu/~EWD/transcriptions/EWD#{...)
{ |f| Hpricot(f) }
                rescue OpenURI::HTTPError
                    $stderr.puts "Not found" if $DEBUG
                    return nil
                end

                text = file.to_plain_text

                #Pick some sentences that seem good
                lines = text.split(/\.(?:\s+|\n+)/).map do |l|
                    l.gsub(/\t|\n|\r/, " ").squeeze(" ").strip
                end.select do |l|
                    l.length > 40 and l.length < 140 and l[0,1] ==
l[0,1].upcase and l[-3, 3] != "viz"
                end

                #Cache them for later
                File.open "ewd#{ewd}", "w" do |f|
                    YAML.dump lines, f
                end

                #Return a random sentence
                lines[rand lines.length]
            end
        end

        #(Very crudely) tries to find a quote that is related to the
message.
        #Note that this only searches the cache.
        def find_related message

            #Get the longer words
            words = message.gsub(/[^a-zA-Z ]/, "").split.select { |w|
w.length > 4 }

            quote = nil
            if words.length > 0
                word = /#{Regexp.union(words)}/i

                #Check local files for the search word
                Dir.glob("ewd*").find do |f|
                    text = File.read f
                    if text =~ word

                        #Get the list of quotes and grab one
                        quotes = YAML.load_file f
                        matches = quotes.grep word
                        quote = matches[rand(matches.length)]
                    else
                        false
                    end
                end
            else
                quote = get_quote
            end

            quote
        end
    end
end

#Post to Twitter
class DijkstraTwitter

    def initialize
        @twitter = Twitter::Client.new :login => "?", :password => "?"
    end

    #Post a random quote. If _ask_ is true, asks for approval first
    def post_random ask = false
        quote = DijkstraQuote.get_quote

        if ask and ask_permission("Would you like to post this:
\"#{quote}\"") or not ask
            @twitter.status :post, quote
        end
    end

    #Check unanswered @ewd messages and come up with responses
    def post_replies ask = false
        require 'set'

        #Keep track of what has been replied to already
        if File.exist? "ewdreplies"
            replied = YAML.load_file "ewdreplies"
        else
            replied = Set.new
        end

        replies = @twitter.status(:replies)

        replies.each do |status|
            status_id = status.id.to_s
            next if replied.include? status_id
            message = status.text
            user = status.user.screen_name
            if reply_to message, user, ask
                replied << status_id
            end
        end

        File.open "ewdreplies", "w" do |f|
            YAML.dump replied, f
        end
    end

    #Send a reply if we can find one
    def reply_to message, sender, ask = false
        puts "Responding to \"#{message}\""

        quote = DijkstraQuote.find_related message
        if not quote
            puts "Nothing related. Skipping."
            return false
        elsif ask and ask_permission("Would you like to post this:
\"#{quote}\"") or not ask
            response = "@#{sender} #{quote}"
            if response.length > 140
                response = response[0,140]
            end

            @twitter.status(:post, "@#{sender} #{quote}")
            true
        else
            if ask and ask_permission "Ignore this reply?"
                true
            else
                false
            end
        end
    end

    #Ask if the user would like to post the quote which was found
    def ask_permission message
        puts message

        response = nil
        until response =~ /^(y|n)/i
            print "(Y/N): "
            response = gets
        end

        $1.downcase == "y"
    end
end

#Try it out
puts DijkstraQuote.get_quote

#dt = DijkstraTwitter.new
#dt.post_random true
#dt.post_replies true


My favorite so far: "In this sense, Programming = Mathematics + Murphy's
Law"

-Justin
017e05d1a49ffa59ea03e149e7af720b?d=identicon&s=25 Chris Shea (chrisshea)
on 2009-06-09 01:15
(Received via mailing list)
On Jun 5, 2:54 pm, Daniel Moore <yahi...@gmail.com> wrote:
> Visit:http://rubyquiz.strd6.com/suggestions
>
> The programs will consist of two parts: a component for interacting
> save them until everyone has had a chance to consider their own
> implementations.
>
> Have Fun!
> --
> -Danielhttp://rubyquiz.strd6.com

Here's the twitter "personality" I made: http://twitter.com/iloveweather

It posts about the weather conditions of a random US city.  The phrase
list needs some expanding, and there's a little code to clean up
before I post it.  It was pretty fun, and I'm really enjoying seeing
tweets like "Step outside, Bliss. It's nice." and "It's days like this
that make people happy to be in Beverly."

It doesn't do replies (yet).

More to come,
Chris
D91f9adf63acfa73b50ebab43f10d7ee?d=identicon&s=25 Thorsten Hater (Guest)
on 2009-06-09 16:34
(Received via mailing list)
Hi,

here my implementation of a Markov chain based text generator for
Twitter.

class MarkovText

  def initialize file
    @word_map = {}
    do_preprocessing file
  end

  def do_preprocessing file
    text = File.readlines(file).collect{|l| l.chomp}.join
    text.gsub! /([a-zA-Z0-9])([,:.!?;])/, '\1 \2'
    text.squeeze! ' '
    words = text.split ' '
    for i in (0..words.size-2)
      # if we encounter this word first initialize with empty Hash
      @word_map[words[i]] ||= {}
      @word_map[words[i]][words[i+1]] ||= 0.0
      @word_map[words[i]][words[i+1]]  += 1.0
    end

    # generate prob distribution for text
    @word_map.keys.each do |k|
      sum = @word_map[k].values.inject(&:+)
      @word_map[k].keys.each{|l| @word_map[k][l] /= sum}
    end
  end

  def generate start_word = nil
    text = [start_word || @word_map.keys[rand(@word_map.keys.size-1)] ]
    while !(/[.!?:;]/ === text.last) do
      w = choose_next(text.last)
      break if !w
      text << w
    end
    text
  end

  def choose_next word
    roll = rand
    cand = @word_map[word]
    return nil if !cand
    i = 0
    while roll > cand.values[i]
      roll -= cand.values[i]
      i += 1
    end
    cand.keys[i]
  end
end

And the inteface code which maintains a collection of text generators
and
does interfacing with the output (stdout at the moment) and restriction
to
140 chars etc:

# load Markov chain text generator
require 'markov_text'

class TwPerson

  def initialize min, max, *files
    @max  = max
    @min  = min

    @name = "Mark V. Shaney"
    @pwd  = "markovianSplitPersonality"

    @gens = []
    files.each do |f|
      @gens << MarkovText.new(f)
    end

    @pool = []
    generate_text
  end

  def run
    loop do
      make_post
      time = rand(@max-@min) + @min
      puts "> Made my post, sleeping for #{time}s"
      sleep(time)
    end
  end

  def make_post
    puts "> Trying to post"
    text = ""
    while  (text.length + @pool.last.length) < 139
      text += " " + get_next
    end
    text.squeeze! ' '
    text.gsub!(/( )([.,;:?!])/) do |m|
      m[1]
    end
    do_post text
  end

  def generate_text
    puts "> Generating new text pool"
    m = @gens.shift
    @pool += m.generate
    @gens << m
  end

  def do_post txt
    puts txt
  end

  def get_next
    generate_text if @pool.size < 2
    @pool.shift
  end

end

This solution is quick and dirty and does not claim to be either elegant
or free from errors ;)

Thorsten
0df4a6c75caf1bd9b01d2dcbfb085ee4?d=identicon&s=25 Sandro Paganotti (Guest)
on 2009-06-10 10:14
(Received via mailing list)
Here's my solution: http://github.com/sandropaganotti/Twitter-Sapiensit
uses
Wordnet to create a group of words semantically-related to a starting
sentence (its personality);
the uses this words to generate a new sentence.

Sandro
0df4a6c75caf1bd9b01d2dcbfb085ee4?d=identicon&s=25 Sandro Paganotti (Guest)
on 2009-06-10 10:27
(Received via mailing list)
Uops. real link:
http://github.com/sandropaganotti/Twitter-Sapiens<...

On Wed, Jun 10, 2009 at 8:13 AM, Sandro Paganotti <
33117162fff8a9cf50544a604f60c045?d=identicon&s=25 Daniel X Moore (yahivin)
on 2009-06-14 21:13
(Received via mailing list)
Attachment: 208.tar.gz (8 KB)
There were two main components to this week's quiz, the interface to
Twitter and the personality that generates messages.

Let's start with the Twitter interfaces.

Justin Collins' @ewdbot uses the Twitter4r gem[1]. The usage is
straightforward. When given the account credentials it creates an
object that provides methods to post status updates and get
information from Twitter.

    def initialize
      @twitter = Twitter::Client.new :login => "?", :password => "?"
    end

    ...
    # Posting a quote
    @twitter.status :post, quote

    # Getting replies
    replies = @twitter.status(:replies)


Sandro Pagonatti's @twsapiens uses the Twitter gem[2].

    # Initialize Twitter credentials
    base = Twitter::Base.new(Twitter::HTTPAuth.new('twsapiens', <psw
here>))

    # Get a random status text from the account's friends timeline
    base.friends_timeline.sort{|a,b| rand()<=>rand()}.first.text


One comment about randomizing arrays is that Ruby 1.9.1 provides a
`shuffle` method to `Array`, so now you no longer need to use
`sort_by{ rand }` or the like if you are on 1.9.1.

Both of the gems make connecting to Twitter via HTTP a breeze, though
Twitter4r seems a little bit simpler to use. No matter which one you
choose it shouldn't be more than a couple of lines to connect.

Now that we're all connected to Twitter let's examine some ways to
generate messages to send.

@ewdbot by Justin selects a random Edsger Dijkstra quotation from the
Edsger W. Dijkstra Archive[3]. The quotation is selected by
downloading a random page from the archive, selecting sentences that
meet certain size and regex requirements, then choosing a random
sentence from that list. The trimmed word lists are cached locally to
make future access easier. Justin's solution is well written and
definitely worth examining if you are interested in learning more.

Remember that Sandro was selecting a random status message from
@twsapiens friends timeline? Well that message is used to seed the
personality. Sandro uses the Linguistics gem[4] to create a collection
of words related to the selected message. Those words are then
arranged according to grammar rules from Ola Bini's port[5] of Peter
Norvig's _Paradigms of Artificial Intelligence Programming_. Some of
these sentences come out a little crudely constructed, but it is a
very difficult problem for a program to construct it's own sentences.
This sentence in particular is rather thought provoking:

    the character in table on a variation by he by volume with he on a
variation typecast a notebook computer

Thorsten Hater submitted a Markov chain text generator. I tested it
out on some of the EWD quotations and it had some successful results.
It would be interesting to combine the Markov chain text generation
with trending topics in an attempt to create a popularity bot. Let the
mailing list know when your bot hits 1,000,000 followers!

Twitter Personality Accounts:

* @iloveweather - Chris Shea
* @ewdbot - Justin Collins
* @twsapiens - Sandro Pagonatti

Thank you everyone for your great responses to this week's quiz!

[1]: http://twitter4r.rubyforge.org/
[2]: http://twitter.rubyforge.org/
[3]: http://www.cs.utexas.edu/~EWD/transcriptions/trans...
[4]: http://www.deveiate.org/projects/Linguistics/
[5]:
http://github.com/olabini/paipr/tree/7334d50b3b629...
This topic is locked and can not be replied to.