Scrape & Analyze Restaurant Reviews

This simple ETL Workflow scrapes all of the Tripadvisor reviews for a specific restaurant for the current month and then runs a sentiment analysis on each review to determine the positive or negative aspects. Then, the results are saved to a Google sheet where a report is generated.

This flowchart helps us to visualize the tasks and logic of the workflow.

Workflow Code

This workflow is the code that orchestrates tasks (through the Zenaton workflow engine) and executes them on your servers.

const spreadsheetId = "";
const gsheetConnectorId = "";

module.exports = {
  *handle(restaurantName, tripAdvisorUrl) {
    // Get the current month to keep workflow idempotent.
    const currentMonth = yield this.run.task("GetCurrentMonth");

    // Scrape the tripAdvisorUrl to get the reviews of the month.
    const reviews = yield this.run.task("ScrapeReviewsOfTheMonth",
      tripAdvisorUrl,
      currentMonth
    );

    // Run SentimentAnalysis tasks in parallel for each review.
    tasks = reviews.map((review) => ["SentimentAnalysis", review]);
    const postiveAndNegativeAspects = yield this.run.task(...tasks)

    // Save results or data to a GoogleSheet
    yield* this.sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects);
  },
  *sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects) {
    const google_sheets = this.connector('google_sheets', gsheetConnectorId);

    // Format data to GoogleSheet format
    const cells = postiveAndNegativeAspects.map((aspect) => {
      // build here the cells content you want to save
    });

    return yield google_sheets.post(`v4/spreadsheets/${spreadsheetId}:batchUpdate`, cells);
  }
};

Workflow Steps

• Get the current month
• Pull all of the tripadvisor urls for the specified restaurant for that month
• Run a sentiment analysis script for each review (in parallel) to determine if it is positive or negative
• Save the data to a google sheet

Schedule Scrape & Analyze Workflow

We can easily make this "ReviewAnalysisWorkflow" workflow recurrent by using the schedule method. For instance, the one-line code below schedules the workflow every month.

const { Client } = require("zenaton");
const client = new Client(app_id, api_token, app_env);

client.schedule("0 0 1 * *").workflow("ReviewAnalysisWorkflow");

View all scheduled workflows and tasks on the Zenaton dashboard.

View real-time data from each step on the Zenaton dashboard - including errors, logs and automatic retries.