line

ScrapingNinja API

ScrapingNinja is a Web Scraping API that handles proxies and Headless browser for you, so you can focus on extracting the data you want, and nothing else.

View ScrapingNinja API docs

Use the ScrapingNinja Connector

Add the scrapingninja connector on your Zenaton dashboard,
enter the credentials and authorize Zenaton to call the scrapingninja API from your workflows.
Then add code snippet and URL path to your workflow.

View documentation
const { workflow } = require("zenaton");

module.exports = workflow("MyWorkflow", function* () {
const scrapingninja = this.connector(
'scrapingninja', 'your-connector-id-from-zenaton-dashboard');

const response = yield scrapingninja.get('path_to_API');
yield scrapingninja.post('path_to_API', {param1: '...'});
});

The Zenaton Engine

We handle the complexity of orchestrating ScrapingNinja API calls and related logic in your code.

Out of the Box API Call

Add the connector to your Zenaton workflow and we manage the authentication, oauth1 and oauth2 flow including token refreshing.

Automatic Retries & Alerting

If your ScrapingNinja API call fails, it can automatically be retried and you will receive an alert with error details.

Monitoring and Troubleshooting

View the Zenaton dashboard for execution history, scheduled tasks, errors and logs or retry failed ScrapingNinja API calls.

Integrate ScrapingNinja into your application logic

The Zenaton connector is a pre-configured task for calling the ScrapingNinja API inside your workflow with one line of code. Build custom integration logic by adding a workflow directly into your application using the functions in the Zenaton SDK.

// The Zenaton engine orchestrates scrapingninja API calls and related logic via the Zenaton agent. Every step is executed at the right moment on your servers and monitored on Zenaton dashboard.
const { workflow } = require("zenaton");
// scrapingninja authentification on Zenaton
module.exports = workflow("ParallelWorkflow", function* () {
  const scrapingninja = this.connector(
    'scrapingninja',
    'your-connector-id-from-zenaton-dashboard'
  );
// execute parallel tasks handled automatically by the Zenaton engine 
  const [a, b] = yield this.run.task(["TaskA"],["TaskB"]);
  if (a > b) {
    const response = yield scrapingninja.get('path_to_API');
  } else {
    yield this.run.task('TaskD');
  }
});

const { workflow } = require("zenaton");
// scrapingninja authentification on Zenaton
module.exports = workflow("AsynchronousWorkflow", function* () {
  const scrapingninja = this.connector(
    'scrapingninja',
    'your-connector-id-from-zenaton-dashboard'
  );
  this.run.task('TaskA');
  this.scrapingninja.post('path_to_API');
  yield run.task('TaskB');
  yield this.run.task('TaskD');
});
// Tasks can be automatically (or manually) retried and executions are displayed in real-time on the Zenaton dashboard.
const { workflow, duration } = require("zenaton");

// scrapingninja authentification on Zenaton
module.exports = workflow("WaitWorkflow", function* () {
  const scrapingninja = this.connector(
    'scrapingninja',
    'your-connector-id-from-zenaton-dashboard'
  );

  yield this.run.task('TaskA');

// The 'wait function is managed by the Zenaton engine and will be executed on your worker at the right time.
  yield this.wait.for(duration.days(7));

  yield scrapingninja.get('path_to_API');
});
const { workflow, duration } = require("zenaton");
module.exports = workflow("WaitEventWorkflow", function*() {
  const scrapingninja = this.connector(
    'scrapingninja',
    'your-connector-id-from-zenaton-dashboard'
  );
    // Wait for up to 24 hours for the event using the Zenaton Wait Function.
    const event = yield this.wait.event("MyEvent").for(duration.hours(24));
    if (event) {
      // If event has been triggered within 24 hours
      yield this.run.task('TaskA');
    } else {
      // else calls scrapingninja API
      yield scrapingninja.get('path_to_API');
    }
  },
  id() {
    return this.id;
  }
});