Bulk Register Domain Scraper

Questions and answers about anything related to Helium Scraper
Post Reply
Teedium
Posts: 6
Joined: Tue Jun 05, 2012 11:19 pm

Bulk Register Domain Scraper

Post by Teedium » Fri Jun 15, 2012 1:07 am

Hello!
Thanks to the help from this forum I was able to make a crude scraper for my needs. I'm using it to pull all our host records from bulkregister in preparation for switching domain registers. Its very crude, only grabbing the Host Record and making a note of the domains that have custom DNS and EMAIL settings. I found that some kinds I set one day were not working the next day.... maybe bulkregister changes something?

How it works:
manually log into bulkregister.com

from http://www.bulkregister.com/domains/Domain-Manager.aspx, it searches using write from database from Table1.
if the search turns up multiple results, it makes a note for manual work: SearchedFor in table ExtractedData. (see "Issues")
Wait 7 seconds for page to load.
From the domain overview, it finds the currentSite, DNSTYPE and EMAILTYPE. If they're custom then you'll have to manually get them after. (see "Issues")
it navigates into NAVHOST
wait 3 seconds
get the host name, record type and address.

Issues:
bulkregister uses drop down menus for a few essential navigations. I don't know how to change their values. If I could, I would change the Search Type from "Starts with" to "Matching", which would eliminate the multiple results issue.
For now, I'll use the DNSTYPE and EMAILTYPE column to make a list and run new scrapes for all custom settings and match them up manually. I couldn't find a way to navigate from Host Records to DNS Settings and Email Settings without the drop down menu.

I'm open to criticism! This was my first scraper, and while it probably took me just as long as to do the job manually, it was definitely more fun :)
Let me know if you can help with the Issue.
Attachments
BulkRegisterScrape.7z
had to zip for file size..
(36.28 KiB) Downloaded 275 times

webmaster
Site Admin
Posts: 510
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Bulk Register Domain Scraper

Post by webmaster » Sat Jun 16, 2012 2:05 am

Hi,

I cannot test this since I don't have an account, but here are a few observations:
  • About the Repeat 1 times atop Execute tree: Write From Database: did you mean to put the Execute tree: Write From Database as a children of the Repeat?
  • You shouldn't need to use Wait actions at all. If this contents loads dynamically (after the page completes loading) you can always use a Force Select online premade, which waits just as long as it takes for the required content to load.
  • Regarding the drop down menus, I just happened to upload a premade that may be just what you need at http://www.heliumscraper.com/forum/view ... ?f=3&t=523.
Juan Soldi
The Helium Scraper Team

Post Reply