ScrapingNinja API
ScrapingNinja is a Web Scraping API that handles proxies and Headless browser for you, so you can focus on extracting the data you want, and nothing else.
View ScrapingNinja API docs
Use the ScrapingNinja Connector
Add the scrapingninja connector on your Zenaton dashboard,
enter the credentials and authorize Zenaton to call the scrapingninja API from your workflows.
Then add code snippet and URL path to your workflow.
const { workflow } = require("zenaton")
module.exports = workflow("MyWorkflow", function* () {
const scrapingninja = this.connector(
'scrapingninja',
'connector-id')
const response = yield scrapingninja.get('path_to_API')
const params = {body: {param1: '...'}}
yield scrapingninja.post('path_to_API', params)
})The Zenaton Engine
We handle the complexity of orchestrating ScrapingNinja API calls and related logic in your code.
Out of the Box API Call
Add the connector to your Zenaton workflow and we manage the authentication, oauth1 and oauth2 flow including token refreshing.
Automatic Retries & Alerting
If your ScrapingNinja API call fails, it can automatically be retried and you will receive an alert with error details.
Monitoring and Troubleshooting
View the Zenaton dashboard for execution history, scheduled tasks, errors and logs or retry failed ScrapingNinja API calls.
Integrate ScrapingNinja into your application logic
The Zenaton connector is a pre-configured task for calling the ScrapingNinja API inside your workflow with one line of code. Build custom integration logic by adding a workflow directly into your application using the functions in the Zenaton SDK.
// The Zenaton engine orchestrates scrapingninja API calls and related logic via the Zenaton agent. Every step is executed at the right moment on your servers and monitored on Zenaton dashboard.
const { workflow } = require("zenaton");
// scrapingninja authentification on Zenaton
module.exports = workflow("ParallelWorkflow", function* () {
const scrapingninja = this.connector(
'scrapingninja',
'your-connector-id-from-zenaton-dashboard'
);
// execute parallel tasks handled automatically by the Zenaton engine
const [a, b] = yield this.run.task(["TaskA"],["TaskB"]);
if (a > b) {
const response = yield scrapingninja.get('path_to_API');
} else {
yield this.run.task('TaskD');
}
});
const { workflow } = require("zenaton");
// scrapingninja authentification on Zenaton
module.exports = workflow("AsynchronousWorkflow", function* () {
const scrapingninja = this.connector(
'scrapingninja',
'your-connector-id-from-zenaton-dashboard'
);
this.run.task('TaskA');
this.scrapingninja.post('path_to_API');
yield run.task('TaskB');
yield this.run.task('TaskD');
});
// Tasks can be automatically (or manually) retried and executions are displayed in real-time on the Zenaton dashboard.
const { workflow, duration } = require("zenaton");
// scrapingninja authentification on Zenaton
module.exports = workflow("WaitWorkflow", function* () {
const scrapingninja = this.connector(
'scrapingninja',
'your-connector-id-from-zenaton-dashboard'
);
yield this.run.task('TaskA');
// The 'wait function is managed by the Zenaton engine and will be executed on your worker at the right time.
yield this.wait.for(duration.days(7));
yield scrapingninja.get('path_to_API');
});
const { workflow, duration } = require("zenaton");
module.exports = workflow("WaitEventWorkflow", function*() {
const scrapingninja = this.connector(
'scrapingninja',
'your-connector-id-from-zenaton-dashboard'
);
// Wait for up to 24 hours for the event using the Zenaton Wait Function.
const event = yield this.wait.event("MyEvent").for(duration.hours(24));
if (event) {
// If event has been triggered within 24 hours
yield this.run.task('TaskA');
} else {
// else calls scrapingninja API
yield scrapingninja.get('path_to_API');
}
}
});
Start building workflows
Sign-up and run a sample project Learn more
Zenaton
Product
Resources
Zenaton Examples
