Trying to get wine info on this page but Infinite scroll stop after a couple scrolls.
https://www.vivino.com/explore?e=eJwNxE ... ficYG1M%3D
It helps increasing the scrollDelay but over a certain milliseconds delay, wathever the delay specified, it stops. I tried to increase navigation timeout, remove image handling and increasing max retries in project settings.
Same same, I cant get more than 175 lines (items).
I think it may be a protection on the page itself, because the offscreen browser show a page without loading new paage scroll button at bottom.
Here my code :
https://i.imgur.com/Df4tgZ6.png
Infinite scroll stop after a couple scrolls
Infinite scroll stop after a couple scrolls
- Attachments
-
- Df4tgZ6.png (78.14 KiB) Viewed 7409 times
Last edited by Stark8026 on Sat Mar 11, 2023 8:38 pm, edited 1 time in total.
Re: Infinite scroll stop after a couple scrolls
I also tried with slowscroll script. Let it go for 5 hours. It slow as **8 and I think there a bug again cause 7000 results divided / 25 = 280 scrolls whihc is not so long. In the ends no results were collected.
See attached screens for settings i used for the Scrollto bottom tryout
.
See attached screens for settings i used for the Scrollto bottom tryout
.
- Attachments
-
- Capture d’écran 2023-03-11 153529.png (84.9 KiB) Viewed 7410 times
Re: Infinite scroll stop after a couple scrolls
The slow scroll premade won't help here since that scrolls just once. I've noticed that the site sometimes won't load more items after scrolling to the bottom, even when manually scrolling, so when this happens Helium will think there are no more items and stop scrolling.
Note sure if there's much that can be done in this case other than figuring out which requests the site is making and getting the data from there. Check this video for an example on how to do this, or if you know some TypeScript you can also use this feature.
Note sure if there's much that can be done in this case other than figuring out which requests the site is making and getting the data from there. Check this video for an example on how to do this, or if you know some TypeScript you can also use this feature.
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team