We use cookies

We use cookies to ensure you get the best experience on our website. For more information on how we use cookies, please see our cookie policy.

By clicking "Accept", you agree to our use of cookies.
Learn more.

User GuideWorkers

Workers

Workers are the backbone of Hatchet, responsible for executing the individual tasks. They operate across different nodes in your infrastructure, allowing for distributed and scalable task execution.

How Workers Operate

In Hatchet, workers are long-running processes that wait for instructions from the Hatchet engine to execute specific steps. They communicate with the Hatchet engine to receive tasks, execute them, and report back the results.

Declaring a Worker

Now that we have a task declared we can create a worker that can execute the task.

Declare a worker by calling the worker method on the Hatchet client. The worker method takes a name and an optional configuration object.

def main() -> None:
  worker = hatchet.worker("test-worker", workflows=[simple])
  worker.start()
 
if __name__ == "__main__":
    main()

And that’s it! Once you run your script to start the worker, you’ll see some logs like this, which tell you that your worker is running.

For self-hosted users, you may need to set other gRPC configuration options to ensure your worker can connect to the Hatchet engine. See the Self-Hosting docs for more information.

[DEBUG] 🪓 -- 2025-03-24 15:11:32,755 - creating new event loop
[INFO]  🪓 -- 2025-03-24 15:11:32,755 - ------------------------------------------
[INFO]  🪓 -- 2025-03-24 15:11:32,755 - STARTING HATCHET...
[DEBUG] 🪓 -- 2025-03-24 15:11:32,755 - worker runtime starting on PID: 26406
[DEBUG] 🪓 -- 2025-03-24 15:11:32,758 - action listener starting on PID: 26434
[INFO]  🪓 -- 2025-03-24 15:11:32,760 - starting runner...
[DEBUG] 🪓 -- 2025-03-24 15:11:32,761 - starting action listener health check...
[DEBUG] 🪓 -- 2025-03-24 15:11:32,764 - 'test-worker' waiting for ['simpletask:step1']
[DEBUG] 🪓 -- 2025-03-24 15:11:33,413 - starting action listener: test-worker
[DEBUG] 🪓 -- 2025-03-24 15:11:33,542 - acquired action listener: efc4aaf2-be4a-4964-a578-db6465f9297e
[DEBUG] 🪓 -- 2025-03-24 15:11:33,542 - sending heartbeat
[DEBUG] 🪓 -- 2025-03-24 15:11:37,658 - sending heartbeat

Note that many of these logs are debug logs, which only are shown if the debug option on the Hatchet client is set to True

Understanding Slots

Slots are the number of concurrent task runs that a worker can execute, are are configured using the slots option on the worker. For instance, if you set slots=5 on your worker, then your worker will be able to run five tasks concurrently before new tasks start needing to wait in the queue before being picked up. Increasing the number of slots on your worker, or the number of workers you run, will allow you to handle more concurrent work (and thus more throughput, in many cases).

An important caveat is that slot-level concurrency is only helpful up to the point where the worker is not bottlenecked by another resource, such as CPU, memory, or network bandwidth. If your worker is bottlenecked by one of these resources, increasing the number of slots will not improve throughput.

Best Practices for Managing Workers

To ensure a robust and efficient Hatchet implementation, consider the following best practices when managing your workers:

  1. Reliability: Deploy workers in a stable environment with sufficient resources to avoid resource contention and ensure reliable execution.

  2. Monitoring and Logging: Implement robust monitoring and logging mechanisms to track worker health, performance, and task execution status.

  3. Error Handling: Design workers to handle errors gracefully, report execution failures to Hatchet, and retry tasks based on configured policies.

  4. Secure Communication: Ensure secure communication between workers and the Hatchet engine, especially when distributed across different networks.

  5. Lifecycle Management: Implement proper lifecycle management for workers, including automatic restarts on critical failures and graceful shutdown procedures.

  6. Scalability: Plan for scalability by designing your system to easily add or remove workers based on demand, leveraging containerization, orchestration tools, or cloud auto-scaling features.

  7. Consistent Updates: Keep worker implementations up to date with the latest Hatchet SDKs and ensure compatibility with the Hatchet engine version.