Zenaton white logo

Going to production

Before we start, let's define some vocabulary that will be used in the next step.

Web App
Web App represents servers that will receive HTTP traffic from your users. They are able to run your application code.
Worker
Workers are servers dedicated to the execution of workflows and tasks. They have access to the same code as the Web App servers. Technically, they are not so different from Web App servers. They have the necessary software needed to run your application. The only difference is that they should not handle HTTP traffic from your users. It's more a conceptual difference, to be able to separate the load generated from HTTP traffic and the load generated by workflows and tasks, to make sure one will not have any performance impact on the other.
Agent
The Zenaton Agent means the software we distribute which is running on your servers (Web App and Workers). It's a binary, run as a daemon.

Dedicated workers setup

On both your Web App and your workers, you will need to:

  • have Node-old installed;
  • have workflows and tasks' source code available;
  • have a Zenaton Agent installed, configured and always running.

Network traffic

Traffic is always initiated by the Agent to Zenaton. You must open the following ports for the Agent to work:

Outbound
  • 443/tcp: port used by the Agent to send requests to Zenaton API
  • 5672/tcp: port used by the Agent to send and receive instructions

Client Mode

Since we don't want any task execution to happen in your Web App, we will start the Agent there in "client mode". Client mode means that the Agent is able to dispatch workflows and tasks, send events, pause, kill and resume workflows, but will not handle any of the executions.

To start the Agent in client mode, just add the --client option during configuration:

zenaton listen --client --php
zenaton listen --client --node
zenaton listen --client --node
zenaton listen --client --ruby
zenaton listen --client --python
zenaton listen --client --go

On your Workers, configure the Agent as usual:

zenaton listen --boot=boot.php
zenaton listen --boot=boot.js
zenaton listen --boot=boot.js
zenaton listen --boot=boot.rb
zenaton listen --boot=boot.py
zenaton listen --boot=boot/boot.go

If you want to use your Web App as a Worker also, and thus avoid having more servers to manage, just configure your Agent in your Web App as if it was a Worker.

Environment

By default there are three environments created per applications: dev, staging and production.
They are just a label to isolate data into your dashboard.

Each developer on your team can create their own environment for testing - for example in their local environment: dev-david, dev-sonia, ...

View your plan details to see how many environments your team may create.

To use an environment, change the agent parameter ZENATON_APP_ENV.

When deploying to production, make sure to use the production environment.

ZENATON_APP_ENV=production

Create a new environment

To create a new environment, change the agent parameter ZENATON_APP_ENV

ZENATON_APP_ENV=my_new_env

Then you will need to unlisten, and then listen again, so that the agent will connect to this new environment.

zenaton unlisten
zenaton listen <parameters>

The Zenaton engine will identify and create the new environment and it will appear on your dashboard.
You can check that the agent configuration is what you were expecting in the agent monitoring dashboard, and also switch to this new environment using the environment select box.
(You might have to refresh the page to see the newly created environment).

Keep the Agent Running

To make sure your application is able to dispatch tasks and workflows at any time, you need to make sure the Agent is always running and listening using your application credentials.

To enforce this, you will need to use a software capable of monitoring the execution of a process and make sure it's always running. You can use systemd, supervisord, or any similar software to achieve this. Let's see how we can do that using systemd. You will need to run the following commands as user root.

First, create a systemd unit file:

touch /etc/systemd/system/zenaton-agent.service

Make sure the file permissions are correct:

chmod 664 /etc/systemd/system/zenaton-agent.service

Open the file we just created with your favorite text editor and add the following content:

[Unit]
Description=Zenaton Agent Daemon service
After=network-online.target

[Service]
Type=forking

# User and group that will be used to run your Zenaton Agent. User must have a Zenaton Agent installation
# in its home directory and have correct permissions to execute your sources.
User=...
Group=...

# Path to sources of your workflows & tasks.
WorkingDirectory=...

# Add your exact listen command
ExecStartPost=/usr/local/bin/zenaton listen --boot=...

ExecStart=/usr/local/bin/zenaton start
ExecStop=/usr/local/bin/zenaton stop
Restart=always

[Install]
WantedBy=multi-user.target

Make sure to replace ... with correct values.

Deploying to Heroku

To be able to use Zenaton in your Heroku hosted app, you will need to use our Heroku buildpack. The buildpack will install a Zenaton Agent inside your dynos, allowing your application to dispatch tasks and workflows, send events, etc. You will also be able to use your dynos as Zenaton workers.

Installation

To add the buildpack to your project, as well as the required environment variables, use the following commands:

cd <HEROKU_PROJECT_ROOT_FOLDER>

# If this is a new Heroku project
heroku create

# Add the appropriate language-specific buildpack
heroku buildpacks:add heroku/php

# Add Zenaton buildpack and set your Zenaton credentials
heroku buildpacks:add --index 1 zenaton/heroku-buildpack-zenaton
heroku config:set ZENATON_APP_ID=<ZENATON_APP_ID>
heroku config:set ZENATON_API_TOKEN=<ZENATON_API_TOKEN>
heroku config:set ZENATON_APP_ENV=production
# ZENATON_LISTEN_ARGS environment variable is used to give the parameters you want to use for the listen command of the Agent
heroku config:set ZENATON_LISTEN_ARGS="--boot=boot.php --php"

# Deploy to Heroku
git push heroku master
cd <HEROKU_PROJECT_ROOT_FOLDER>

# If this is a new Heroku project
heroku create

# Add the appropriate language-specific buildpack
heroku buildpacks:add heroku/nodejs

# Add Zenaton buildpack and set your Zenaton credentials
heroku buildpacks:add --index 1 zenaton/heroku-buildpack-zenaton
heroku config:set ZENATON_APP_ID=<ZENATON_APP_ID>
heroku config:set ZENATON_API_TOKEN=<ZENATON_API_TOKEN>
heroku config:set ZENATON_APP_ENV=production
# ZENATON_LISTEN_ARGS environment variable is used to give the parameters you want to use for the listen command of the Agent
heroku config:set ZENATON_LISTEN_ARGS="--boot=boot.js --node"

# Deploy to Heroku
git push heroku master

cd <HEROKU_PROJECT_ROOT_FOLDER>

# If this is a new Heroku project
heroku create

# Add the appropriate language-specific buildpack
heroku buildpacks:add heroku/ruby

# Add Zenaton buildpack and set your Zenaton credentials
heroku buildpacks:add --index 1 zenaton/heroku-buildpack-zenaton
heroku config:set ZENATON_APP_ID=<ZENATON_APP_ID>
heroku config:set ZENATON_API_TOKEN=<ZENATON_API_TOKEN>
heroku config:set ZENATON_APP_ENV=production
# ZENATON_LISTEN_ARGS environment variable is used to give the parameters you want to use for the listen command of the Agent
heroku config:set ZENATON_LISTEN_ARGS="--boot=boot.rb --ruby"

# Deploy to Heroku
git push heroku master
cd <HEROKU_PROJECT_ROOT_FOLDER>

# If this is a new Heroku project
heroku create

# Add the appropriate language-specific buildpack
heroku buildpacks:add heroku/python

# Add Zenaton buildpack and set your Zenaton credentials
heroku buildpacks:add --index 1 zenaton/heroku-buildpack-zenaton
heroku config:set ZENATON_APP_ID=<ZENATON_APP_ID>
heroku config:set ZENATON_API_TOKEN=<ZENATON_API_TOKEN>
heroku config:set ZENATON_APP_ENV=production
# ZENATON_LISTEN_ARGS environment variable is used to give the parameters you want to use for the listen command of the Agent
heroku config:set ZENATON_LISTEN_ARGS="--boot=boot.py --python"

# Deploy to Heroku
git push heroku master

Replace <ZENATON_APP_ID> and <ZENATON_API_TOKEN> with your Zenaton credentials.

When the deploy is finished, the Zenaton Agent will be automatically started when each dyno starts.
The Agent is started in client mode in order to avoid adding load to your web dyno.

Worker dynos

If you want to use some dynos to perform the execution of workflows and tasks, you have to make some dynos using type zenatonworker. These dynos will start the Zenaton Agent in regular mode, executing workflows and tasks. In order to make these worker dynos, add them in your Procfile:

# Procfile
web: vendor/bin/heroku-php-apache2 web/
zenatonworker: tail -f /dev/null
# Procfile
web: node index.js
zenatonworker: tail -f /dev/null
# Procfile
web: bundle exec puma -C config/puma.rb
zenatonworker: tail -f /dev/null
# Procfile
web: gunicorn gettingstarted.wsgi --log-file -
zenatonworker: tail -f /dev/null

Because the buildpack automatically starts the Zenaton Agent, you do not need to provide any specific command for the dyno, that's why we use tail -f /dev/null.

Deploying to Clever Cloud

To be able to use Zenaton in your Clever Cloud application, you just have to add a few environment variables on Clever Cloud dashboard.

Here is an easy tutorial on deploying your Zenaton tasks to your application:

Installing the Zenaton Agent on your Clever Cloud Servers

To be able to start workflows and execute tasks asynchronously, you will need to install the Zenaton Agent. The Agent will connect to the Zenaton infrastructure, execute jobs when instructed and send results back so they can be reported on your dashboard.

To activate the Agent in your Clever Cloud application, add the following environment variables:

variable: CC_WORKER_COMMAND
value: sh -c "zenaton start && zenaton listen --client"
variable: CC_WORKER_COMMAND
value: sh -c "zenaton start && zenaton listen --client"

OR if your servers hosting your web app are different than those processing tasks (which we recommend!), you should launch the Zenaton Agent in Client mode to avoid processing tasks. Client mode means that the Agent is able to dispatch workflows and tasks, send events, pause, kill and resume workflows, but will not handle any of the executions.
If you are launching your Agent in client mode, then add the following value instead:

variable: CC_WORKER_COMMAND
value: sh -c "zenaton start && zenaton listen --client"
variable: CC_WORKER_COMMAND
value: sh -c "zenaton start && zenaton listen --client"

AND

variable: CC_WORKER_COMMAND_SERVICE_TYPE
value: forking
variable: CC_WORKER_COMMAND_SERVICE_TYPE
value: forking

Its should look like this when you're done:

Clever Cloud Zenaton workers

Or like this if you are using Client Mode:

Clever Cloud Zenaton web app

You will also need to add the following environment variables to identify your Zenaton application and environment. These can be found on your Zenaton dashboard.

variable: ZENATON_API_TOKEN
variable: ZENATON_APP_ID
variable: ZENATON_APP_ENV 
(ie. production or staging or whatever you have called your environment)
        
variable: ZENATON_API_TOKEN
variable: ZENATON_APP_ID
variable: ZENATON_APP_ENV 
(ie. production or staging or whatever you have called your environment)
        

Once you have activated the Zenaton Agent in your application, the logic for the tasks and workflows that you have written into your code can be sent to the Zenaton workflow engine and can be executed on your servers.

Deploying to AWS

AWS EC2 instances can be used to run Zenaton workers. The following example will show you how to deploy your code to a Linux-based EC2 instance and have it execute your workflows and tasks. You will see how to:

  • Create an instance and set the security to allow your Agent to talk with Zenaton
  • Install the files needed to run Zenaton, and to execute your Workflows, and your Tasks.
  • Set you Agent to automatically run when the EC2 instance is started.

For this example, we will use a Linux-based instance. Specifically, the "Amazon Linux 2 AMI (HVM), SSD Volume Type" was used. Any version with systemctl installed should work, though you may need to change some of the file paths.

After selecting your AMI, you will be able to select your Instance Type. This example used the t2.micro, which is also free tier eligible.

Steps 3 (Instance Details), 4 (Storage), and 5 (Tags) can be set how you wish, though the defaults will all work. For Security, however, you will need to add two rules. You will need to allow outbound traffic from ports 443 and 5672. You can read more about this here.

AWS Outbound Ports

You can now launch your instance. Download the SSH key if needed, and make note of the Public DNS. Use that information to log into your new instance:

ssh -i <path to key> ec2-user@<public dns>

Once logged in you'll want to update any packages that may be out of date.

sudo yum update
curl --silent --location https://rpm.nodesource.com/setup_12.x | sudo bash - sudo yum -y install nodejs

Next, install Zenaton:

curl https://install.zenaton.com | sh

Now you can copy your project to your instance. Create a directory for it:

mkdir app

And then use scp to copy your files from your computer:

scp -i <path to key> -r app/* ec2-user@<public dns>:app/

Now you can log back into your instance, and install your project's dependencies:

cd app
npm install

Your project can now be run on your instance. Just make sure you have a valid .env in your project directory. All that remains is to set it to automatically start when your instance is launched. We will do this using systemd. As root (ie, via sudo), create the service file at /etc/systemd/system/zenaton-agent.service:

[Unit]
Description=Zenaton Agent Daemon service
After=network-online.target

[Service]
Type=forking

# User and group that will be used to run your Zenaton Agent. User must have a Zenaton Agent installation
# in its home directory and have correct permissions to execute your sources.
User=ec2-user
Group=ec2-user

# Path to sources of your workflows & tasks.
WorkingDirectory=/home/ec2-user/app

# Add your exact listen command
ExecStartPost=/usr/local/bin/zenaton listen --boot=boot.js --env=.env

ExecStart=/usr/local/bin/zenaton start
ExecStop=/usr/local/bin/zenaton stop
Restart=always

[Install]
WantedBy=multi-user.target

Finally, start the Agent, and tell systemd to start it each time your instance is launched:

sudo systemctl daemon-reload
sudo systemctl start zenaton-agent
sudo systemctl enable zenaton-agent

Your instance will now execute your Workflows and Tasks. To test, you can start Zenaton locally in client mode, and check zenaton.out on your EC2 instance to verify the tasks are being executed on the instance.

Deploying to Google Cloud

Google Cloud Compute VMs can be used to run Zenaton workers. The following example will show you how to deploy your code to a Linux-based VM and have it execute your workflows and tasks. You will see how to:

  • Set up a firewall rule to allow communication with Zenaton.
  • Create an instance and set the security to allow your Agent to talk with Zenaton
  • Install the files needed to run Zenaton, and to execute your Workflows and Tasks.
  • Set you Agent to automatically run when the VM is started.

Set Up Firewall Rule

You will need to set up a firewall rule to allow outbound traffic from ports 443 and 5672. To do this, go to the list of Firewall rules, and clilck the "Create Firewall Rule" button at the top.

Give the rule a name so you can easily identify it later (ie "zenaton") and then set it to allow outgoing traffic (egress) to ports 443 and 5672. You can read more about this here.

Firewall Example

If you set this to be a default rule, it will automatically apply to all VM instances in your project. You can also chose to have it only apply to those VMs on which you specifically enable it.

Set Up VM Instance

This example was tested with the "f1-micro" machine type. This was used because the example is simple, and it reduces the cost of running it. You may need to scale up, depending on the amount of power your Tasks need. Also note that, for this example, the default settings were used.

Setting Up Google Compute VM Instance

Once your instance is up and runnning, you can connect to it and set it up to run your Zenaton Tasks:

gcloud compute --project "<project-name>" ssh --zone <zone> <instance-name>

You will need to install Node.js:

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install -y nodejs

Next, install Zenaton:

curl https://install.zenaton.com | sh

Now you can copy your project to your instance. Create a directory for it:

mkdir app

And then use scp to copy your files from your computer:

gcloud compute --project "<project-name>" scp -r app/* <instance-name>:app/

Now you can log back into your instance, and install your project's dependencies:

cd app
npm install

Your project can now be run on your instance. Just make sure you have a valid .env in your project directory. All that remains is to set it to automatically start when your instance is launched. We will do this using systemd. Make sure you know your username on your instance; you can use the whoami command if you need to. As root (ie, via sudo), create the service file at /etc/systemd/system/zenaton-agent.service:

[Unit]
Description=Zenaton Agent Daemon service
After=network-online.target

[Service]
Type=forking

# User and group that will be used to run your Zenaton Agent. User must have a Zenaton Agent installation
# in its home directory and have correct permissions to execute your sources.
User=<username>
Group=<username>

# Path to sources of your workflows & tasks.
WorkingDirectory=/home/<username>/app

# Add your exact listen command
ExecStartPost=/usr/local/bin/zenaton listen --boot=src/boot.js --env=.env

ExecStart=/usr/local/bin/zenaton start
ExecStop=/usr/local/bin/zenaton stop
Restart=always

[Install]
WantedBy=multi-user.target

Finally, start the Agent, and tell systemd to start it each time your instance is launched:

sudo systemctl daemon-reload
sudo systemctl start zenaton-agent
sudo systemctl enable zenaton-agent

Your instance will now execute your Workflows and Tasks. To test, you can start Zenaton locally in client mode, and check zenaton.out on your VM instance to verify the tasks are being executed on the instance.