Workers

Workers in Hatchet are the long-running processes that execute tasks. In the broadest sense, it may be helpful to think of a worker as a simple while loop that receives a new task assignment from Hatchet, executes the task, and reports the results back.

When workers are spun up - in any environment, be it locally, on a VM, etc. - they will register themselves with Hatchet to start receiving and executing tasks.

Declaring a worker

A worker needs a name and a set of tasks (or workflows, more on this later) to register:

def main() -> None:
    worker = hatchet.worker("dag-worker", workflows=[dag_workflow])

    worker.start()

import { hatchet } from '../hatchet-client';
import { simple } from './workflow';
import { parent, child } from './workflow-with-child';
import { simpleWithZod } from './zod';

async function main() {
  const worker = await hatchet.worker('simple-worker', {
    // 👀 Declare the workflows that the worker can execute
    workflows: [simple, simpleWithZod, parent, child],
    // 👀 Declare the number of concurrent task runs the worker can accept
    slots: 100,
  });

  await worker.start();
}

if (require.main === module) {
  main();
}

worker, err := client.NewWorker("simple-worker", hatchet.WithWorkflows(task))
if err != nil {
	log.Fatalf("failed to create worker: %v", err)
}

interruptCtx, cancel := cmdutils.NewInterruptContext()
defer cancel()

err = worker.StartBlocking(interruptCtx)
if err != nil {
	log.Fatalf("failed to start worker: %v", err)
}

def main
  worker = HATCHET.worker("dag-worker", workflows: [DAG_WORKFLOW])
  worker.start
end

When a worker starts, it registers each of its tasks and workflows with Hatchet. From that point on, Hatchet knows to route matching tasks to that worker.

One important note is that multiple workers can register the same task. In this scenario, Hatchet distributes work across all of them, allowing for simple horizontal scaling.

Starting a worker

hatchet worker dev

export HATCHET_CLIENT_TOKEN="<your-client-token>"

export HATCHET_CLIENT_TLS_STRATEGY=none

python worker.py

Once the worker starts, you will see logs confirming it is connected:

[INFO]  🪓 -- STARTING HATCHET...
[DEBUG] 🪓 -- 'test-worker' waiting for ['simpletask:step1']
[DEBUG] 🪓 -- acquired action listener: efc4aaf2-...
[DEBUG] 🪓 -- sending heartbeat

Slots

Every worker has a fixed number of slots that control how many tasks it can run concurrently, which can be configured with the slots option on the worker. For instance, if slots is set to 5, the worker will run up to five tasks concurrently at any time. Any additional tasks wait in the queue until a slot opens up. Slots are a local limit. They protect the individual worker from attempting to run more tasks concurrently than desired, which can help control resource usage by the worker.

The default slot count for workers in Hatchet is 100. In many cases, leaving the default as-is will be perfectly fine, especially when first getting set up with Hatchet.