Im creating a screenscraper app that takes a users search term, scrapes
several 3rd party sites and returns the aggregated results after a few
Rather than wait for all the sites scraping to be completed before
showing the results, I want to show the results as they are scraped, ie.
when one sites results have been scraped and prepared, they are shown
immediately whilst the next set are being done.
Initially I thought id just make a scraper function like so
sites.each do |a_site|
a_sites_results = scrape_and_parse_this_site(a_site)
Then in my controllers js response I could take the results and render
but then i found out htat you cant make multiple render calls in one go.
How do i go about doing this? Does this behaviour have a name like say
“live search” etc