# How to design a system to centralize data? -- using ruby **URL:** \ **Category:** Ruby\ **Created:** [September 27, 2006, 12:16am UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137 "2006-09-27T00:16:10Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\ **Post date:** [September 27, 2006, 12:16am UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137/1 "2006-09-27T00:16:10Z")
Hi I work in a psychology lab which used to test subjects one on one. A subject would be tested on 1 to 20 tests each generating a summary value or text data file. Now in addition, we are getting thousands of responses on single or multiple tests from folks on the web in multiple websites. We can’t possibly copy/paste that volume of data into an excel file. So we need to automate. One thought is to have a centralized depository with both data and data format so folks can analyse the data easily. Plus a way of filtering the data: for example extract data from subjects matched on some criteria… So I think that means a database. Because the high volume data is accumulated on the web and can be sent by email, that suggests a webapplication. ruby on rails can probably do something like that. Is this something a programmer with a little ruby and rails can design, or should we consult with some folks to figure out what the format/system should look like, and if so what kind of folks would be able to advise us? (folks knowing ruby or rails of course) ---
**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\ **Post date:** [September 27, 2006, 12:32am UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137/2 "2006-09-27T00:32:12Z")
> I work in a psychology lab which used to test subjects one on one. A If you’re lab is at harvard as your email suggests, I’d see if there aren’t some CS students that need a project to complete for their coursework and talk to them… -p ---
**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\ **Post date:** [September 27, 2006, 3:06am UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137/3 "2006-09-27T03:06:23Z")
On 9/26/06, anne001 [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > format so folks can analyse the data easily. Plus a way of filtering > the data: for example extract data from subjects matched on some > criteria… So I think that means a database. Yup. > Because the high volume data is accumulated on the web and can be sent > by email, that suggests a webapplication. ruby on rails can probably do > something like that. Well, you mention “multiple websites”, as if the data is already being generated and you need to pick it up and do something with it. Correct? In that case, you may not need a webapp per se, since (it sounds like) the webapps are already in place. If they’re already sending you auto-generated emails, then you’re set: you just need to hack together a script to periodically scan the relevant inbox, pull out those messages, parse them for the data, and insert that data into your db. Then you’ll probably use a 2nd program to mine that data and do something useful with it. > Is this something a programmer with a little ruby and rails can design, > or should we consult with some folks to figure out what the > format/system should look like, and if so what kind of folks would be > able to advise us? (folks knowing ruby or rails of course) I like Philip’s suggestion about finding a CS student or two to do it. Just post a note on the bulletin board in the CS department and you should get some help that way. —John ---
**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\ **Post date:** [September 27, 2006, 5:50am UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137/4 "2006-09-27T05:50:02Z")
On Wednesday, September 27, 2006, at 10:05 AM, John G. wrote: > > One thought is to have a centralized depository with both data and data > > Well, you mention “multiple websites”, as if the data is already being > > something useful with it. > > —John Students can be a fine solution to problems like this, but as with most things, you get what you pay for. \_Kevin [www.sciwerks.com](http://www.sciwerks.com) ---
**Author:** ![anne001](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/34f0e0/32.png) [@anne001](https://www.ruby-forum.com/u/anne001)\ **Post date:** [September 27, 2006, 3:22pm UTC](https://www.ruby-forum.com/t/how-to-design-a-system-to-centralize-data-using-ruby/70137/5 "2006-09-27T15:22:54Z")
Thank you for your post, Unfortunately it is almost impossible to communicate with Computer Science students at Harvard. I have tried in the past, Harvard has no intra posting ability, only computer science students can belong to computer science association, etc. but we get bright student coming through our courses, so we can hook students in that way. Your response is very helpful in that it suggests that for now, we don’t need to change the way we acquire and store the data, and can develop the interface seperately. This will reduce the complexity of the project – if it ain’t broke don’t fix it – . anne