Helium Scraper

To use this template, place it on an empty folder and open it with Helium Scraper 3. Check this video for a quick walkthrough. The template has two globals that extract different pieces of information: ProductLinks : This global extracts top-level information without visiting any actual product page...

All scrolling functions scroll the currently selected element, which defaults to the whole page. So if you select the scrollable element it will scroll that element instead of the whole page: Select.ScrollableElement InfiniteScroll · Select.ListItem · 1000 · true To select the scrollable element, ju...

We've just updated Common Crawler to include the Sequence.WriteFile function. If you don't get an update prompt, this may be because we've migrated the publish location to AWS. If so, just uninstall it and reinstall it from here . Once you have the latest version (3.2.4.9) you can do this to save th...

There's no way to automatically flush the data, but you can save the project with File -> Save while it's still running without having to stop the extraction. Regarding the dot, not 100% sure about this but I think you can use brackets like "[Some.Thing]" in Access. Anyway, it'd make sense to be abl...

You can use Gather.HTML to get the current page HTML (or the HTML of any particular element when the element is selected), and since version 3.2.4.8 you can use Sequence.WriteFile to write files with arbitrary text content. In your case, you could do something like this, supposing all the pages you'...

Since version 3.2.4.4 you can override the user agent at Project -> Settings -> User Agent.

Import this function to extract meta data . To use this on your project, follow these instructions . { "api": "1.0.2.1", "name": "MetaData", "parameters": { "name": { "caption": "Name", "type": "text", "default": "description" } } } function (name) Browser.EvalScript · "try { return document.querySe...

There are a couple of ways to do this. One is to create a selector for each kind of element, and then use the + operator whenever you want to select any of the elements: + · Select.ButtonA · Select.ButtonP If you're using a wizard you will only be able to select a specific selector, so just select a...

Instructions on how to integrate Infatica proxies into Helium Scraper have been published here.

Yes, Select.ByScript must return a DOM element or a list of DOM elements. I guess the documentation should be explicit about this. If you just need to get a string from a script, you can use Browser.EvalScript like this: Browser.EvalScript · "return 'Hello World!';" · 0 as myString extract test mySt...