Turning pages without "Next" button
Turning pages without "Next" button
There are a few cases on which a set of results pages won't provide a "Next" button, but instead, just a list of page numbers.
The attached project is a simple solution to this problem. To use it, just import it into the project you are working on, or use it as a starting point, and then create a kind that selects the page number that comes after the current page. Then navigate to another page and add the page number that comes after that page. So if you are, for instance, at page 1, select the link that takes you to page 2, and create a kind called "Next" with it. Then navigate to page 2, select the link that takes you to page 3 and add it to your "Next" kind.
After this, when you navigate to any other page, this kind will always select the next page number just as if it were a "Next" button and you can use it as such for every automation purpose. It will also work with the Go through all pages premade project.
Without this project, if you try to follow the steps I describe above, your kind will most likely end up selecting all page numbers, instead of just the next one. This is because this project contains a set of JavaScript gatherers that gather properties from elements that are located before the element you select when creating a kind, which on this case means that it will gather properties from the current page number when you select the next page number. Since the current page number is different that all other page numbers (i.e. is not a clickable link), Helium Scraper will be able to distinguish between the next page number and any other page number.
I've tested it on several search engines and sites and it worked OK. But if you notice that your kind starts selecting every page number instead of just the next one, let me know and I'll add whatever JavaScript gatherer is necessary to have it working properly.
The attached project is a simple solution to this problem. To use it, just import it into the project you are working on, or use it as a starting point, and then create a kind that selects the page number that comes after the current page. Then navigate to another page and add the page number that comes after that page. So if you are, for instance, at page 1, select the link that takes you to page 2, and create a kind called "Next" with it. Then navigate to page 2, select the link that takes you to page 3 and add it to your "Next" kind.
After this, when you navigate to any other page, this kind will always select the next page number just as if it were a "Next" button and you can use it as such for every automation purpose. It will also work with the Go through all pages premade project.
Without this project, if you try to follow the steps I describe above, your kind will most likely end up selecting all page numbers, instead of just the next one. This is because this project contains a set of JavaScript gatherers that gather properties from elements that are located before the element you select when creating a kind, which on this case means that it will gather properties from the current page number when you select the next page number. Since the current page number is different that all other page numbers (i.e. is not a clickable link), Helium Scraper will be able to distinguish between the next page number and any other page number.
I've tested it on several search engines and sites and it worked OK. But if you notice that your kind starts selecting every page number instead of just the next one, let me know and I'll add whatever JavaScript gatherer is necessary to have it working properly.
- Attachments
-
- PageNumberProperties.hsp
- (291.48 KiB) Downloaded 2238 times
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
-
- Posts: 10
- Joined: Tue May 03, 2011 9:15 pm
Re: Turning pages without "Next" button
Hello, i would like to know how can I import this premade to my project. thanks
Re: Turning pages without "Next" button
Just use the File -> Import command from the main menu.
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
Re: Turning pages without "Next" button
hi
I have given my more than 60 mins for this online premedes not solutions arises to me
could you give me step by step instruction in this forum as I have to navigate the page with out next
i had imported the go through the page number or also tried turn pages with out next button I am but going crazy
please do help me
I have given my more than 60 mins for this online premedes not solutions arises to me
could you give me step by step instruction in this forum as I have to navigate the page with out next
i had imported the go through the page number or also tried turn pages with out next button I am but going crazy
please do help me
Re: Turning pages without "Next" button
this action tree is not called by any actiion tree
Re: Turning pages without "Next" button
Hi,
You'll see the "This action tree is not called by any action tree" message when you attempt to run a premade actions tree that is supposed to be run from an Execute Actions Tree action in another actions tree. When you add an Execute Actions Tree action you get to select which actions tree to run. Here you would select the premade actions tree you just imported.
Regarding your other post above, I'm not sure I understand what the problem is. Can you attach the project you're working on? Also, which premade are you using, is it the one from File -> Online Premade or is it the one attached on this post?
You'll see the "This action tree is not called by any action tree" message when you attempt to run a premade actions tree that is supposed to be run from an Execute Actions Tree action in another actions tree. When you add an Execute Actions Tree action you get to select which actions tree to run. Here you would select the premade actions tree you just imported.
Regarding your other post above, I'm not sure I understand what the problem is. Can you attach the project you're working on? Also, which premade are you using, is it the one from File -> Online Premade or is it the one attached on this post?
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
Re: Turning pages without "Next" button
Hi
I am trying to extract business name with phone linked to business name through all the pages from page 1
you can see the in the link http://www.hotfrog.co.uk/Products/wedding-cars there is business name and phone no are linked inside with out the next only page number give
please help me
I am trying to extract business name with phone linked to business name through all the pages from page 1
you can see the in the link http://www.hotfrog.co.uk/Products/wedding-cars there is business name and phone no are linked inside with out the next only page number give
please help me
Re: Turning pages without "Next" button
Hi,
What I would do here is use the URL Variations premade since result pages include the page number in the URL. Trying to do it by clicking on the next page up to page 10 and then clicking on Next 10 Pages each time would be a harder way of doing it.
Just import the premade by going File -> Online Premades and select URL Variations (make sure you read the description) and click Import. Then, add an Execute Actions Tree action to a new actions tree and select Navigate URLs. For the URL, enter this (note the dollar symbol at the end; this will be replaced by the page number):
http://www.hotfrog.co.uk/Products/wedding-cars/$
On First Page enter 1, and on Last Page enter 106 (which is the past page for wedding cars). Then press OK and add whatever actions you need to perform inside each results page as children of the Execute Actions Tree action. I assume you want to navigate inside each business' details page, so you'd add as children action a Navigate Each that navigates through each link that takes you to a details page, and then as a children of the Navigate Each action an Extract action that extracts from the details page whatever data you need.
Let me know if you need any more help.
What I would do here is use the URL Variations premade since result pages include the page number in the URL. Trying to do it by clicking on the next page up to page 10 and then clicking on Next 10 Pages each time would be a harder way of doing it.
Just import the premade by going File -> Online Premades and select URL Variations (make sure you read the description) and click Import. Then, add an Execute Actions Tree action to a new actions tree and select Navigate URLs. For the URL, enter this (note the dollar symbol at the end; this will be replaced by the page number):
http://www.hotfrog.co.uk/Products/wedding-cars/$
On First Page enter 1, and on Last Page enter 106 (which is the past page for wedding cars). Then press OK and add whatever actions you need to perform inside each results page as children of the Execute Actions Tree action. I assume you want to navigate inside each business' details page, so you'd add as children action a Navigate Each that navigates through each link that takes you to a details page, and then as a children of the Navigate Each action an Extract action that extracts from the details page whatever data you need.
Let me know if you need any more help.
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
Re: Turning pages without "Next" button
Hey Juan,
I'm also having trouble importing this to my current project. I have downloaded the project and gone to FIle>Import, and click import. It then prompts that this is from an outside source, and I allow it. However, I am then unable to find it in my current project. Any suggestions?
Thanks,
David
I'm also having trouble importing this to my current project. I have downloaded the project and gone to FIle>Import, and click import. It then prompts that this is from an outside source, and I allow it. However, I am then unable to find it in my current project. Any suggestions?
Thanks,
David
Re: Turning pages without "Next" button
This project only adds a few javascript gatherers (you can see their code at Project –> JavaScript Gatherers). Perhaps you were expecting to see new actions or kinds? As the project’s description states, you’d import the project and then create a kind that selects the next page number. Please refer to the project’s description for more details.
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team