Zenaton is a service for managing and monitoring all of your background processes whether it is a single task or a long running workflow - just by writing a few lines of code. We make it easy to implement external services into your app - and trigger a sequence of actions in your code based on responses - or a lack of response - from APIs and other data sources.
Instead of building and managing infrastructure to watch and manage your background jobs, you just write the business logic of your tasks into your code, install the Zenaton agent on your servers and the Zenaton workflow engine handles the rest. The agent will listen to a queuing system hosted for you by Zenaton, and when a task should be executed, the Agent will launch it and collect the output.
We provide a sophisticated backend engine to manage the orchestration of tasks by managing the queues and pushing the decisions down to your workers. And, It's easy to scale up quickly just by adding more workers.
Beside, the Zenaton dashboard offers real-time monitoring for all of your tasks and workflows.
We built Zenaton to give one developer the power and control of an entire team of software developers to manage sophisticated business and data processes.
Lets be honest. Building and managing infrastructure to manage background jobs is a pain. It includes spinning up a special server to manage the queuing system, managing states in a database and then writing cron jobs and a lot of logic in your code. All of this infrastructure must be maintained and there is very little visibility into how things are running. So when something goes wrong you have to read log files and check lots of places to figure out the problem and fix it.
With Zenaton, you can write the business logic for your tasks or workflows directly into your code in an easily readable format so that anyone on the team can understand the logic, troubleshoot and make changes.
We have made it easy to manage all of your background processes without having to worry about...well everything.
Here is how it works:
When running background processes with Zenaton you can either write a single task or a write a sequence of related tasks within a workflow.
Tasks are the basic building blocks of Zenaton. Tasks are where you code your actual work and there is no limitation on what you can do there. Processing of tasks are offloaded onto one of your servers (called a worker then), but usually different from the one from which they were dispatched.
A few examples of tasks:
Using Zenaton library, writing a task is as simple as
from zenaton.abstracts.task import Task from zenaton.traits.zenatonable import Zenatonable class MyTask(Task, Zenatonable): def __init__(self, ...): # initialize properties def handle(self): # Your task implementation
Once Zenaton configured, dispatching a task to be processed on one of your workers is as easy as
Workflows are a set of tasks intended to be processed in a specific order, despite the fact that each of them can have been processed on different machines. Things you can do with a workflow:
Implementing worklows implies dealing with - notoriously difficult - distributed systems, where you have to deal with:
With Zenaton, writing such sequences is very easy, whatever the complexity of your case, we handle all of the complexity for you:
Using Zenaton library, writing a workflow is as simple as
from zenaton.abstracts.workflow import Workflow from zenaton.traits.zenatonable import Zenatonable class MyWorkflow(Workflow, Zenatonable): def __init__(self, ...): # initialize properties def handle(self): # Your workflow implementation
Once Zenaton configured, dispatching a workflow to be processed on your workers is as easy as
With Zenaton, implementing a workflow is very easy as we use our prefered language. For example:
def handle(self): a = TaskA().execute() if a > 0: b = TaskB(a).execute() else: b = TaskC(a).execute() TaskD(b).execute()
We’re going to walk you through creating your first project on your local directory - or in your development environment.
If you are just looking for the easiest way to install and run some workflows to see how Zenton works, we recommend doing our Tutorial.
Here are the steps we will walk through:
First we will create a directory
zenaton-test and a
src repo inside of it.
mkdir -p zenaton-test/src && cd zenaton-test/src
Now using the Zenaton library,
we’re going to create 3 sample tasks into the
GetNameand it will return a string ‘world’
GetSentenceand use the name as the parameter and return ‘hello name’
SaySentenceand will display a sentence in the console
import time from zenaton.abstracts.task import Task from zenaton.traits.zenatonable import Zenatonable class GetName(Task, Zenatonable): def handle(self): time.sleep(3) # simulate a real task return "World"
import time from zenaton.abstracts.task import Task from zenaton.traits.zenatonable import Zenatonable class GetSentence(Task, Zenatonable): def __init__(self, name_): self.name_ = name_ def handle(self): time.sleep(1) # simulate a real task return "Hello " + self.name_ + '!'
import time from zenaton.abstracts.task import Task from zenaton.traits.zenatonable import Zenatonable class SaySentence(Task, Zenatonable): def __init__(self, sentence_): self.sentence_ = sentence_ def handle(self): time.sleep(4) # simulate a real task print(self.sentence_) return
Now, lets write a workflow orchestrating our three tasks. Our workflow will
GetName, to retrieve a string
GetSentence, to retrieve the string
SaySentence, to print it on the console.
from src.get_name import GetName from src.get_sentence import GetSentence from src.say_sentence import SaySentence from zenaton.abstracts.workflow import Workflow from zenaton.traits.zenatonable import Zenatonable class HelloWorld(Workflow, Zenatonable): def handle(self): name = GetName().execute() sentence = GetSentence(name).execute() SaySentence(sentence).execute()
If you want to do more with your workflows learn more about implementing workflows with Zenaton.
zenaton-test directory, install our library
pip install zenaton
If we wanted to run a quick test of our workflow locally,
we would just run our
HelloWorld workflow by calling the
handle method directly!
To do this, we will create the following file:
# load dependencies import zenaton.client from src.hello_world import HelloWorld # init Zenaton client zenaton.client.Client( 'AppId', 'ApiToken', 'dev' ) # processing a workflow in foreground HelloWorld().handle()
And now we can run this file with following command:
So far we’ve create 3 tasks and a workflow to orchestrate and run those three tasks locally.
Now we are going to install the Zenaton agent on our computer so that we can let Zenaton orchestrate our tasks and workflow in the background and dispatch them to our computer - via the agent.
The beauty of Zenaton is that all of this happens without having to write any additional code. You just have to install our Agent and deploy your sources wherever you want your tasks to be processed. When you are working locally, your sources are already in your computer, so we only need to install the agent locally.
If we wanted to deploy to another environment (like in production), we would need to install the Zenaton agent on the servers ("workers") where we wanted to execute tasks and (in client mode) on the servers from where tasks and workflows are dispatched. Zenaton automatically orchestrates the processing of our tasks and workflows on distributed workers and monitors all of the activity.
If you have not already installed the Zenaton agent on your computer, we will do that here.
curl https://install.zenaton.com | sh
Depending on your configuration, you may be asked for a password to allow global installation of Zenaton.
If you need to deploy the Agent on a particular system like Docker, Heroku, or Clever Cloud, we have some precooked solutions for you.
Since the Agent will process our tasks and workflows in the background, we will need to provide a boot file that will be loaded before any processing:
# load dependencies import zenaton.client from src.hello_world import HelloWorld zenaton.client.Client( 'AppId', 'ApiToken', 'dev' )
Finally, we need to provide our
which we will find on Zenaton API section,
so that the Agent can listen to our configuration and report processing status back
to our engine and your Zenaton dashboard:
zenaton listen --boot=boot.py --app_id=AppId --api_token=ApiToken --app_env=dev
You should see
Zenaton worker is now listening to app "AppId" on env "dev".
Note that once an agent is installed, we can always check the Agents monitoring section on the dashboard to view Agent settings and worker activity for each environment.
Ok we are now ready to run some background tasks and workflows! Let's make sure that the Zenaton library is initialized with our credentials before dispatching them, eg. we will create the following file:
# load dependencies import zenaton.client from src.get_name import GetName from src.hello_world import HelloWorld # init Zenaton client zenaton.client.Client( 'AppId', 'ApiToken', 'dev' ) # dispatching a task to be processed in background GetName().dispatch() # dispatching a workflow to be processed in background HelloWorld().dispatch()
And now we can run this file with following command:
Everything written to stdout during the processing in background of your your tasks will be in the
that you'll find in the same directory than our sources. For example, after first launch, you should see:
If we do not get the expected result, in stderr during the processing of our tasks will be in the
that you'll find in the same directory than our sources.
We can check the Agents section on our dashboard which allows us to monitor the real-time activity of each of our connected Agents:
We can check the Tasks section, of the dashboard to monitor the real-time processing of our individual tasks:
And we can also check the Workflows section of our dashboard to see an overview of all of our processed tasks and workflows, including real time-monitoring of tasks that are in progress.
And we only need to click on each task to see the full details:
The easiest way to get started using Zenaton is to do our tutorial.
The tutorial will walk you through installing the Zenaton agent on your computer, downloading our library for your preferred programming language and running all of the examples from our examples repo.
This will give you an opportunity to see what our syntax looks like for workflows and see how quickly you can set up your environment and start processing background tasks using the Zenaton service.
You will also have the opportunity to view all of your executed tasks and workflows displayed on your Zenaton dashboard.
In order to run the tutorial tutorial you need to have an active Zenaton account to access your unique Application Id and Api Token.
Download our examples repo wherever you want (on the same machine where you already launched a Zenaton agent). (If you have completed the tutorial, you would have already downloaded the examples repository.)
git clone https://github.com/zenaton/examples-python.git
Then, install all dependencies
pip3 install -r requirements.txt
Then, add a
cp .env.example .env
And update the
.env file with your application id and your api token found
here. Now that your
.env file is complete, you can launch Zenaton agent with these credentials:
zenaton listen --env=.env --boot=boot.py
.env is the default value for --env option. You can also omit it here.
A Zenaton agent is natively able to handle concurrent tasks. There is no need to launch it more than once on the same server. Your credentials will be used to listen to any task sent by your application to this agent, within the provided environment.
An environment is a way to isolate workflow instances between production and development. A workflow is always launched within a given environment - and its tasks are handled only by agents listening to this environment.
Now you can look at all provided examples - a launcher is provided for each of them, eg. launch_wait.py
# Get the configured Zenaton client import .client # Start a new instance of the workflow from .workflows.wait_workflow import WaitWorkflow WaitWorkflow().dispatch()
will execute this
from zenaton.abstracts.workflow import Workflow from zenaton.traits.zenatonable import Zenatonable from zenaton.tasks.wait import Wait from tasks.task_a import TaskA from tasks.task_b import TaskB class WaitWorkflow(Workflow, Zenatonable): def handle(self): TaskA().execute() Wait().seconds(5).execute() TaskB().execute()
These examples include:
launch_sequentialillustrates a sequential execution of tasks and how you can use a task result in a following task;
launch_parallelillustrates some parallel executions of tasks and how the workflow waits for all results before going further;
launch_asynchronousillustrates some asynchronous "fire and forget" executions;
launch_eventillustrates how you can retrieve a workflow instance, send an event to it, and handle this event to modify workflow execution;
launch_waitillustrates how you can use the provided
Waitclass to manage time in your workflow;
launch_wait_eventillustrates how you can use the provided
WaitEventclass to wait for an external event before going further (with a timeout);
Feel free to play with these examples, eg.
Last but not least, you can add an intentional error in these examples to see how the workflow stops and how you can see the error here. Fix your code and click 'retry' to resume the workflow execution without any additional effort!