# Web crawler - spider and Amazon Web Servces (AWS)

**URL:** <https://www.ruby-forum.com/t/web-crawler-spider-and-amazon-web-servces-aws/143424>\
**Category:** Rails\
**Created:** [July 18, 2008, 11:28am UTC](https://www.ruby-forum.com/t/web-crawler-spider-and-amazon-web-servces-aws/143424 "2008-07-18T11:28:45Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abel](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/2acd7d/32.png) [@Abel](https://www.ruby-forum.com/u/Abel)\
**Post date:** [July 18, 2008, 11:28am UTC](https://www.ruby-forum.com/t/web-crawler-spider-and-amazon-web-servces-aws/143424/1 "2008-07-18T11:28:45Z")

</div>

I need to create a web crawler and the closest thing to a tutorial  
I’ve found so far is this article:

[http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1182](http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1182)  
wich I think I found via RubyInside or some blog.

It uses some of Amazone Web Services, mainly SQS, but this would be my  
first time outsourcing a process to a third party and I would like to  
know if someone in the group has experience with AWS and what they  
think about this kind of architecture.