Scrape & Analyze Restaurant Reviews

This simple ETL Workflow scrapes all of the Tripadvisor reviews for a specific restaurant for the current month and then runs a sentiment analysis on each review to determine the positive or negative aspects. Then, the results are saved to a Google sheet where a report is generated.

Visual of Workflow

This flowchart helps us to visualize the tasks and logic of the workflow.

Workflow Steps

  • First, it gets the current month
  • Then we pull all of the tripadvisor urls for that restaurant for that month
  • Then a sentiment analysis script is launched for each review (in parallel) to determine if it is positive or negative
  • Finally, the data is saved to a google sheet

Workflow Code

This workflow is the code that orchestrates tasks (through the Zenaton workflow engine) and executes them on your servers.

const spreadsheetId = "";
const gsheetConnectorId = "";

module.exports = {
  *handle(restaurantName, tripAdvisorUrl) {
    // Get the current month to keep workflow idempotent.
    const currentMonth = yield"GetCurrentMonth");

    // Scrape the tripAdvisorUrl to get the reviews of the month.
    const reviews = yield"ScrapeReviewsOfTheMonth",

    // Run SentimentAnalysis tasks in parallel for each review.
    tasks = => ["SentimentAnalysis", review]);
    const postiveAndNegativeAspects = yield

    // Save results or data to a GoogleSheet
    yield* this.sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects);
  *sendDataToGoogleSheet(restaurantName, currentMonth, postiveAndNegativeAspects) {
    const google_sheets = this.connector('google_sheets', gsheetConnectorId);

    // Format data to GoogleSheet format
    const cells = => {
      // build here the cells content you want to save

    return yield`v4/spreadsheets/${spreadsheetId}:batchUpdate`, cells);

Schedule Scrape & Analyze Workflow

We can easily make this "ReviewAnalysisWorkflow" workflow recurrent by using the schedule method. For instance, the one-line code below schedules the workflow every month.

const { Client } = require("zenaton");
const client = new Client(app_id, api_token, app_env);

client.schedule("0 0 1 * *").workflow("ReviewAnalysisWorkflow");

You will have access on the Zenaton dashboard to all workflows scheduled:


Workflow Executions

View a short snippet of the task executions from the dashboard.