Distributing workflows on Docker - Scale with Zenaton

Scaling out your Zenaton workers across a cluster

While we put a lot of care into our tutorial to help you see what you can do with Zenaton, one part we didn’t really cover is how you would use Zenaton in a production-like environment. Ideally you’d want to be able to

• trigger a workflow execution from one service
• spread the processing of your tasks on many machines

In this post, we are going to create a very simple workflow in Ruby and use Docker to simulate a pool of workers. The only things you’ll need to follow along are your Zenaton credentials and Docker installed on your machine.

Building our application

If you want to skip ahead, you can copy clone our sample application from GitHub and jump straight to the next section.

Before starting out, here is an overview of the structure of the application we are going to be building in this section.

The Gemfile

source 'https://rubygems.org'
gem 'zenaton'

This is where we list our Ruby dependencies. For this example, we only really need to load the Zenaton gem.

The Task

class LetterIncrementer < Zenaton::Interfaces::Task
  include Zenaton::Traits::Zenatonable

  def initialize(letter, workflow_id)
    @letter = letter
    @workflow_id = workflow_id
  end

  def handle
    puts "This is task #{@letter}, from workflow: #{@workflow_id}"
    @letter.next
  end
end

Our task is rather simple. When building a new task object, we initialize it with a letter and a workflow ID. Then when we call #handle on the task object, it prints a message containing both variables to standard output and returns the next letter of the alphabet.

The Workflow

class Alphabet < Zenaton::Interfaces::Workflow
  include Zenaton::Traits::Zenatonable

  def initialize(id)
    @letter = 'A'
    @id = id
  end

  def handle
    @letter = LetterIncrementer.new(@letter, @id).execute while @letter != 'F'
  end
end

Our workflow keeps track of the current state, in our case, which letter of the alphabet we are at. We initialize the workflow with an ID and start at the letter ‘A’ . Then we repeat our created task, passing it through the current letter and the workflow ID until after we’ve reached the letter ‘E’.

The Configuration

Zenaton::Client.init(
  ENV['ZENATON_APP_ID'],
  ENV['ZENATON_API_TOKEN'],
  ENV['ZENATON_APP_ENV']
)

Here all we need to configure is the Zenaton client. We tell it to load our credentials from environment variables. We will set them when we configure our docker cluster.

The Boot File

require 'zenaton'
require './config/initializers/zenaton.rb'
require './app/workflows/alphabet'
require './app/tasks/letter_incrementer'

# Don't buffer when writing to stdout
STDOUT.sync = true

Finally, wiring everything together, we create a boot file. This will be the main entry point of our application.

Building our cluster

To begin, our plan is to create two container images. One will be used for our client instance, the other for our worker instances. In our case, the client will boot up in order to launch the workflow we’ve created and then power down. In practice this client could be a long-running process, like your web application, for example. Our workers will remain up, waiting to receive tasks to complete.

The Client

The single purpose of the client is to execute the following script which triggers the execution of a workflow with a random UUID.

#!/usr/bin/env ruby
require './app/boot'
require 'securerandom'
Alphabet.new(SecureRandom.uuid).dispatch

Let’s create a docker image for our client.

FROM debian:stretch AS zenaton-installer

RUN apt-get update \
 && apt-get install -y curl \
 && curl https://install.zenaton.com | sh \
 && apt-get remove -y curl \
 && apt-get clean

FROM ruby:stretch

COPY --from=zenaton-installer /root/.zenaton /root/.zenaton
RUN ln -s /root/.zenaton/zenaton /usr/local/bin/zenaton

WORKDIR /app
ADD Gemfile* ./
RUN bundle install

CMD ["./bin/launch_client"]

Here we download and install the Zenaton Agent binary into a separate layer. Then in our actual application image, we copy that binary to make it available to the user running our application.

If you are using this Dockerfile as a reference for your own application, please be aware that the installation script from install.zenaton.com will install the Agent in $HOME/.zenatonand then symlink the executable to be in the $PATH. If your application is configured to be run as a user other than root, you will need to amend the COPY and RUN commands so that they point to the correct home directory for that user.

Once the image is provisioned, we will execute a shell script that will start the Agent in client mode and then execute the ruby script to trigger the execution of our alphabet workflow.

#!/bin/sh
set -e
bundle check || bundle install
zenaton start
zenaton listen --app_id="$ZENATON_APP_ID" --ruby --client
./bin/launch_workflow

The Worker

Our worker image is going to be pretty similar to our client one. The main differences are

• we copy the application directory inside our image to prevent the workers from writing to the same zenaton.out file, and
• we launch a different script once the image is provisioned.

FROM debian:stretch AS zenaton-installer

RUN apt-get update \
 && apt-get install -y curl \
 && curl https://install.zenaton.com | sh \
 && apt-get remove -y curl \
 && apt-get clean

FROM ruby:stretch

COPY --from=zenaton-installer /root/.zenaton /root/.zenaton
RUN ln -s /root/.zenaton/zenaton /usr/local/bin/zenaton

WORKDIR /app
COPY . .
RUN bundle install

CMD ["./bin/launch_worker"]

The script to be launched once the worker instance is up consists of starting the installed Zenaton Agent and tailing the zenaton.log file so that we can see our workflow being executed.

#!/bin/sh
set -e
bundle check || bundle install
zenaton start
zenaton listen --app_id="$ZENATON_APP_ID" --boot=app/boot.rb
touch zenaton.out && truncate -s 0 zenaton.out
tail -f zenaton.out

Before continuing please make sure all the scripts inside the bin folder are executable. A quick chmod +x bin/* should do the trick.

The Cluster

We can now set up our different images by creating a docker-compose.yml file

version: "3"
services:
  client:
    environment:
      ZENATON_APP_ENV: ${ZENATON_APP_ENV}
      ZENATON_APP_ID: ${ZENATON_APP_ID}
      ZENATON_API_TOKEN: ${ZENATON_API_TOKEN}
    build:
      context: .
      dockerfile: docker/client/Dockerfile
    volumes:
      - client_bundle:/usr/local/bundle
      - .:/app
  worker:
    environment:
      ZENATON_APP_ENV: ${ZENATON_APP_ENV}
      ZENATON_APP_ID: ${ZENATON_APP_ID}
      ZENATON_API_TOKEN: ${ZENATON_API_TOKEN}
    build:
      context: .
      dockerfile: docker/worker/Dockerfile
    volumes:
      - worker_bundle:/usr/local/bundle
volumes:
  client_bundle:
  worker_bundle:

Executing workflows on our cluster

Credentials

Our application needs to authenticate with Zenaton and read the credentials from environment variables. To set them we are going to use the fact that by default docker-compose reads environment variables from a .env file if present. Go to your Zenaton dashboard, grab your credentials and create your .env file.

ZENATON_APP_ENV=production
ZENATON_APP_ID=your-zenaton-application-id
ZENATON_API_TOKEN=your-zenaton-api-token

Running the workflow on one worker

With this we can now build our images and execute our workflow. In a terminal, type:

docker-compose build && docker-compose up

Building the image takes a couple of minutes, but once this is done you will see the client and worker instances booting up. After dispatching the workflow, the client powers down and you will see the worker instance picking up the tasks sequentially.

While this is fine and shows that our setup works, what do we need to change if we want to have 5 worker instances processing our tasks?

Nothing.

Running the workflow on multiple workers

Simply spin more worker instances and Zenaton will take care of distributing the tasks across all your available workers. In our setup, we can trivially launch more worker instances. To make things more interesting we can also launch multiple workflows at the same time. In your terminal, type

docker-compose up --scale client=3 --scale worker=5

This will execute our workflow 3 times and the tasks will be processed by 5 instances. Once the workers are up and running, you should be able to see the tasks being executed in your terminal.

Note that we are still traversing the alphabet sequentially for each workflow, but any of the 5 available workers can pick up the task of displaying the current letter of a given workflow. Additionally the workflow tasks are automatically scheduled on our pool of workers.

All that without any changes.

While this is a very simplistic workflow that only processes tasks sequentially, with this example we were able to look at a few things:

• Starting a Zenaton Agent in client mode to be able to trigger workflow execution from one machine.
• Having many worker instances process the tasks from a given workflow.

For more realistic examples of workflows you can check out:

• Building a marketplace with Zenaton
• Sending an ETA notification using Zenaton
• Coding a Welcome Email Series

We are eager to know what your potential use cases look like and would enjoy the opportunity to discuss how Zenaton can help drastically simplify your applications. When creating your free account you will be invited to our community Slack. Come say hi! 👋