Technical Introduction to Zenaton

Problem

Creating workflows is a hassle for developers, creating frustrations in the whole team and slowing down product development and customer satisfaction.

Zenaton helps developers run, and scale background jobs that need to be orchestrated. So:

• if you are using a queue management system to offload your web app, you’ve probably had the need to recover or retry when a task fails;
• if some of your tasks last a long time or need some kind of human interaction, you’ve felt the difficulty of tracking the state of processing;
• if your workflows are a bit complex, you’ve encountered the difficulty of orchestrating all the tasks together and improving their flow;

Zenaton can help you.

Solution

A good habit when implementing a workflow is to ensure a clear separation between the control flow and the tasks. What makes Zenaton really shine is that you actually code the control flow in your preferred programming language, with all its versatility. For example, to write a workflow in nodejs that will request a price from 3 providers, wait for all results and then buy the cheapest, you may write:

var { Workflow } = require("zenaton");
var GetPriceFromProviderA = require("./GetPriceFromProviderA");
var GetPriceFromProviderB = require("./GetPriceFromProviderB");
var GetPriceFromProviderC = require("./GetPriceFromProviderC");
var OrderFromProviderA = require("./OrderFromProviderA");
var OrderFromProviderB = require("./OrderFromProviderB");
var OrderFromProviderC = require("./OrderFromProviderC");

module.exports = Workflow("OrderWorkflow", function() {

  var prices = [
    new GetPriceFromProviderA(this.item),
    new GetPriceFromProviderB(this.item),
    new GetPriceFromProviderC(this.item)
  ].execute();
  
  switch (prices.indexOf(Math.min(...prices))) {
    case 0:
      new OrderFromProviderA(this.item).execute();
      break;
    case 1:
      new OrderFromProviderB(this.item).execute();
      break;
    case 2:
      new OrderFromProviderC(this.item).execute();
      break;
    }
});

Other than coding the tasks, that’s really all you need to do! The first execute method will trigger parallel executions of GetPriceFromProviderA, GetPriceFromProviderB and GetPriceFromProviderC. When all results are obtained, the workflow automatically resumes and triggers the order from the cheapest provider.

In your web app, launching such a workflow is as simple as:

new OrderWorkflow({ item: “item_ref” }).dispatch();

As you may have noted, the syntax used is very similar to a queuing system. That’s not just a coincidence, as in many ways Zenaton is very similar — except that instead of dispatching simple tasks, you dispatch the orchestrated tasks using your workflow class.

How it works

When your web app launches a workflow through the Zenaton library, a queued message is sent to Zenaton Engine with instructions to start a OrderWorkflow with the given parameters — from there The Zenaton engine will maintain the state of the workflow and messages will be queued back and forth with workers to perform “decisions” or “tasks”.

You need to install some code in your servers to be able to execute those “decisions” and “tasks” . We call this code a Zenaton worker, and you can install it with this simple instruction:

curl https://install.zenaton.com | sh

A Zenaton worker will listen to queues managed by Zenaton Engine and execute Zenaton Engine instructions coming from there. There are two types of instructions:

• a “decision” is the task of using your workflow class (orderworkflow.js in the example above) to determine what to do next.
• a “task” is a actual task to process (eg. GetPriceFromProviderA in the example above). Those tasks are locally executed on your worker, and the result is serialized and sent back to the Zenaton Engine.

Note: Zenaton workers need to locally access your sources to be able to process them, but those sources are never moved or uploaded elsewhere.

To illustrate this, let’s consider a very simple workflow with two sequential tasks:

var { Workflow } = require("zenaton");
var TaskA = require("./TaskA");
var TaskB = require("./TaskB");

module.exports = Workflow("SequentialWorkflow", function() {
  new TaskA().execute();
  new TaskB().execute();
});

1. the Engine asks workers to perform a “decision” task using SequentialWorkflow implementation
2. a worker catches this message, determines what to do next and tells the Engine that the next thing to do is executing TaskA
3. the Engine asks workers to execute TaskA
4. a worker catches this message and processes TaskA, serializes and sends the output back to the Engine
5. the Engine asks workers to perform a “decision” task using SequentialWorkflow implementation
6. a worker catches this message, determines what to do next and tells the Engine that the next thing to do is executing TaskB
7. a worker catches this message and processes TaskB, serializes and sends the output back to the Engine,
8. the Engine asks workers to perform a “decision” task using SequentialWorkflow implementation
9. a worker catches this message, determines what to do next and tells the Engine there is nothing more to do!
10. the Engine closes the workflow.

This graph below illustrate this flow:

Conclusion

Zenaton makes it very easy to execute asynchronous long-running business or marketing logic:

• writing a workflow is very simple and does not usually take longer than a few dozens of lines of simple code;
• implementation is inherently scalable as every task is dispatched through a queue messaging system, and you can easily multiply the number of workers;
• the implementation is resilient, as the state is saved at each step, and if a task fails, it can easily be retried without losing the workflow state.

Zenaton is for technical teams that understand that their primary mission is to improve the business through quick iterations and new ideas, not spending most of their time solving purely technical issues.

If you have additional questions, feel free to contact me at gilles at zenaton.com or to ask them below 👇