Page 1 of 1

Infinite scroll stop after a couple scrolls

Posted: Fri Mar 10, 2023 10:03 pm
by Stark8026
Trying to get wine info on this page but Infinite scroll stop after a couple scrolls.
https://www.vivino.com/explore?e=eJwNxE ... ficYG1M%3D

It helps increasing the scrollDelay but over a certain milliseconds delay, wathever the delay specified, it stops. I tried to increase navigation timeout, remove image handling and increasing max retries in project settings.

Same same, I cant get more than 175 lines (items).

I think it may be a protection on the page itself, because the offscreen browser show a page without loading new paage scroll button at bottom.

Here my code :

https://i.imgur.com/Df4tgZ6.png

Re: Infinite scroll stop after a couple scrolls

Posted: Sat Mar 11, 2023 8:38 pm
by Stark8026
I also tried with slowscroll script. Let it go for 5 hours. It slow as **8 and I think there a bug again cause 7000 results divided / 25 = 280 scrolls whihc is not so long. In the ends no results were collected.

See attached screens for settings i used for the Scrollto bottom tryout
.

Re: Infinite scroll stop after a couple scrolls

Posted: Wed May 10, 2023 8:44 am
by webmaster
The slow scroll premade won't help here since that scrolls just once. I've noticed that the site sometimes won't load more items after scrolling to the bottom, even when manually scrolling, so when this happens Helium will think there are no more items and stop scrolling.

Note sure if there's much that can be done in this case other than figuring out which requests the site is making and getting the data from there. Check this video for an example on how to do this, or if you know some TypeScript you can also use this feature.