line

Scrape & Analyze Restaurant Reviews

This simple ETL Workflow scrapes all of the Tripadvisor reviews for a specific restaurant for the current month and then runs a sentiment analysis on each review to determine the positive or negative aspects. Then, the results are saved to a Google sheet where a report is generated.

Visual of Workflow

This flowchart helps us to visualize the tasks and logic of the workflow.
line

Workflow Steps

  • First, it gets the current month
  • Then we pull all of the tripadvisor urls for that restaurant for that month
  • Then a sentiment analysis script is launched for each review (in parallel) to determine if it is positive or negative
  • Finally, the data is saved to a google sheet

Workflow Code

This workflow is the code that orchestrates tasks (through the Zenaton workflow engine) and executes them on your servers.

const spreadsheetId = "";
const gsheetConnectorId = "";

module.exports = {
  *handle(restaurantName, tripAdvisorUrl) {
    // Get the current month to keep workflow idempotent.
    const currentMonth = yield this.run.task("GetCurrentMonth");

    // Scrape the tripAdvisorUrl to get the reviews of the month.
    const reviews = yield this.run.task("ScrapeReviewsOfTheMonth",
      tripAdvisorUrl,
      currentMonth
    );

    // Run SentimentAnalysis tasks in parallel for each review.
    tasks = reviews.map((review) => ["SentimentAnalysis", review]);
    const postiveAndNegativeAspects = yield this.run.task(...tasks)

    // Save results or data to a GoogleSheet
    yield* this.sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects);
  },
  *sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects) {
    const google_sheets = this.connector('google_sheets', gsheetConnectorId);

    // Format data to GoogleSheet format
    const cells = postiveAndNegativeAspects.map((aspect) => {
      // build here the cells content you want to save
    });

    return yield google_sheets.post(`v4/spreadsheets/${spreadsheetId}:batchUpdate`, cells);
  }
};

Schedule Scrape & Analyze Workflow

We can easily make this "ReviewAnalysisWorkflow" workflow recurrent by using the schedule method. For instance, the one-line code below schedules the workflow every month.

const { Client } = require("zenaton");
const client = new Client(app_id, api_token, app_env);

client.schedule("0 0 1 * *").workflow("ReviewAnalysisWorkflow");

You will have access on the Zenaton dashboard to all workflows scheduled:

cron-review-analysis

Workflow Executions

View a short snippet of the task executions from the dashboard.
line