Help for creating project

Questions and answers about anything related to Helium Scraper
Post Reply
dcmumbai
Posts: 2
Joined: Mon Sep 19, 2011 1:55 pm

Help for creating project

Post by dcmumbai » Mon Sep 19, 2011 2:09 pm

I have created one project , but it is giving problem , after extracting some pages it is taking blank pages..
kindly check attached project file and let me know what is problem...
Attachments
infoline-ver-1.hsp
(414.93 KiB) Downloaded 578 times

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Help for creating project

Post by webmaster » Mon Sep 19, 2011 7:50 pm

Hi,

I tested it for 10 pages and it worked just fine. What exactly do you mean by "taking blank pages"? If the problem is that you are getting blank pages when navigating to a page, then the problem is on the server side. It's simply serving you blank pages. This could be because it is detecting too many requests from your IP. The only solution here would be to use proxies.

You can set up Helium Scraper to use proxies from Project -> Proxies and have your Repeat action rotate them.
Juan Soldi
The Helium Scraper Team

dcmumbai
Posts: 2
Joined: Mon Sep 19, 2011 1:55 pm

Re: Help for creating project

Post by dcmumbai » Tue Sep 20, 2011 6:42 pm

Thanks , yes you are right the blank pages was due to site problem,
Now it is working fine , the only problem is that after 10 -12 pages the scraper slow down,
and extracting process becomes very slow and it take lot of time to extract data.
Kindly help me for this problem
Thanks
Regrds
dcmubmai

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Help for creating project

Post by webmaster » Wed Sep 21, 2011 7:56 pm

Hi,

Again, this is because the server gets slower at serving pages. Now, you might want to try setting a shorter navigation timeout at Project -> Options. This will cause Helium Scraper to do as if the page has finished loading even if it has not whenever that timeout is reached.

Just make sure is not the whole page but just a section you are not interested in that is taking too long to complete loading. If you see that when navigating to one of these pages, the browser stays on the previous page or you see an empty page for a long time, this would mean is the whole page that is taking too long to complete and another solution, such as using proxies, will be required.
Juan Soldi
The Helium Scraper Team

Post Reply