I am trying to scrape inventory data from a website.
The website does not have a next button and I am using the premade listed under 'Turning pages without "Next" button'
Currently, I have repeat set to 10 times.
My understanding of how this works is that the process starts, navigates to each link, extracts the data, and then turns the page.
Then, repeats so the repeat counter is now 2/10.
The number of pages could be as small as 24 or as large as 1000 or more, so setting the repeat to 10 will prevent me from getting all the inventory. However if I set it to 100000, it might cycle through the inventory 100 times and over write or duplicate inventory.
Is there a way to set this up so it only goes through the inventory on all the pages until it gets to the end?
I updated my current (updated project file) if looking at it is helpful.
Questions and answers about anything related to Helium Scraper
2 posts • Page 1 of 1
- Project Rev1
- (59.83 KiB) Downloaded 272 times
I got this working. This is fixed using the "Go through all pages" premade.