Automatic exporting at end of execution [update]

[vodkasushi]

Trying out this program, great work!
Have a question: is there a way to automatically export CSV at the end of an execution, via command line or otherwise?
[triple edit]Ah, there's a thread that already answers this... didn't find it on Search for some reason. Well, I hope the feature gets implemented ASAP!

I want to elaborate on this feature request - I'm currently using the MS Access db as a temporary data store, which I then pass on to a C# app to process the data. When processing finishes, I wipe the database (e.g. "DELETE * FROM Table1") file. Now for some unfathomable reason, Access does not clean up after itself, so even with the data gone, the metadata still exists, so the file size only goes up. To solve this, I programatically compact the database using the MS JET library. This compacts the database, but it produces a copy of the database as a result. The original file is left intact. I replace the old file with the new one, BUT the new one is set up as read-only. Running Helium Scraper again with this database relation causes a "Operation must use an updateable query" error (incidentally is there a "automatically end and close program when encountering error and also log the error" feature somewhere on your roadmap? ).

I haven't solve this yet but things would be so much easier if a CSV file could be automatically made. I'd just delete that file after using it!

[edit]Is there also a way to remove/rename actions trees? [double edit] Doh... there's a button!

[webmaster]

How are you saving to the Ms Access database? Do you have your project connected to an external database, or do you export the database each time? If connected to an external database (in the database panel, Connect to Database File button), you'll be able to programmatically clear the database every time you start a new extraction. Is this what you are doing? How much bigger does the database get every time you clear it?

[vodkasushi]

I'm trying to fully automate the scraping process, so the scraping project is connected to an external database. How may I programatically clear the database through Helium? I've been able to do it via Microsoft's JET library but as I've stated, the solution I've found does not reduce filesize, it expands upon it.

However, after further tests, I think the database filesize only goes up by 1MB at most (I'm scraping the same details over and over for the test) after every scrape so it's managable for now, I'd still like a way to export automatically to a different, simpler format though.

[webmaster]

You can run SQL queries from an Execute JavaScript action like this:

Code: Select all

Global.DataBase.Query("DELETE * FROM [MyTable]");