It takes forever to scrap

Questions and answers about anything related to Helium Scraper
Post Reply
bbee
Posts: 2
Joined: Thu Mar 03, 2011 5:46 am

It takes forever to scrap

Post by bbee » Mon Mar 07, 2011 1:46 am

One page, it takes about 5 minutes. I better do it by hand. I'll finish waaaay faster.

Any ideas Why?

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: It takes forever to scrap

Post by webmaster » Mon Mar 07, 2011 2:43 am

The page is probably taking forever to load. If you want send me the page, or even better, your project, and I'll take a look at it. You can attach Helium Scraper projects in this forum.
Juan Soldi
The Helium Scraper Team

bbee
Posts: 2
Joined: Thu Mar 03, 2011 5:46 am

Re: It takes forever to scrap

Post by bbee » Mon Mar 07, 2011 6:56 pm

Thank you for your reply.

Yesterday, I spent several hours trying to get this program to scrap. Today, I tried one more time before replying to you, and it started scraping fast.

I'll be doing more tests. Maybe it was the proxies. I'm not sure. I was scraping with and without proxies.

Speaking about proxies, can you adjust the program so we can upload proxies in bunches? not one by one.

Thank you again! I'm starting to like this program. I'll let you know about my progress.

Best wishes.

caliman
Posts: 21
Joined: Tue May 31, 2011 5:12 pm

Re: It takes forever to scrap

Post by caliman » Tue May 31, 2011 5:18 pm

Hello Juan, I am scraping a site but it has many pictures, images an banners, so it takes a while to finish and the HS waits until everything is loaded. Can I do something? The important information loads very fast, so it is not necessary waiting until all the stuff loads. thanks.

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: It takes forever to scrap

Post by webmaster » Tue May 31, 2011 7:38 pm

Hi,

Try changing the navigation timeout at Project -> Options. Make sure your timeout is no less that what it takes for your data to load. Also, as a precaution, I would set a requirement in the "Extract" action. This way if the data haven't loaded yet, Helium Scraper will let you know by popping up a message box on which you can instruct it to try again (which should continue the execution as normally if the data already loaded).
Juan Soldi
The Helium Scraper Team

Post Reply