Embedded Hatchet

Hatchet can run in an embedded mode from inside your workers. The goal is to make local testing as easy as possible and be able to test workers end to end in ephemeral CI environments. There is no need to provision tenants, users, or API tokens to get started.

When using the embedded mode, you get a full Hatchet engine running locally without any external dependencies including Postgres. By default it uses embedded-postgres to provision a Postgres database.

How it works

• Go runs the engine in-process: the hatchet-embedded package registers itself with the Go SDK via a blank import, and the SDK boots the engine on NewClient.
• TypeScript and Python run the engine as a sidecar process. On first use the SDK downloads the hatchet-embedded-sidecar binary for your platform from the hatchet-embedded releases (signed and notarized on macOS), caches it under ~/.hatchet/embedded/<version>, and spawns it. The binary is verified against the release checksums on every start, and the sidecar shuts down with your process.

Usage

from hatchet_sdk import Hatchet

hatchet = Hatchet.from_embedded()

from hatchet_sdk import ClientConfig, EmbeddedHatchetConfig, Hatchet

hatchet = Hatchet.from_embedded(
    ClientConfig(
        embedded=EmbeddedHatchetConfig(
            version="vX.Y.Z",
            database_url="postgres://...",
            log_level="info",
        )
    )
)

See runnable examples for all three SDKs in the hatchet-embedded repo.

Options

* WithPostgresDataDir is an option on the embed package directly; the remaining Go options are hatchet.WithEmbedded* client options. The embed package also offers WithAdminUser(email, password), WithKeysets(...), and WithLogger(...) when you drive the engine yourself via embed.StartServer.

The Python options are fields of EmbeddedHatchetConfig, set via ClientConfig.embedded. Each of these can also be set through a HATCHET_CLIENT_EMBEDDED_* environment variable (for example, HATCHET_CLIENT_EMBEDDED_BINARY_PATH).

Serve the dashboard

Embedded instances do not ship a frontend. The hatchet embedded-ui CLI command serves the dashboard bundled in the CLI binary on your machine and proxies API requests to the instance's API server. With no flags it targets the default embedded API port (http://localhost:28243):

hatchet embedded-ui

This opens a browser to the locally served dashboard. The URL contains a one-time ui_token that is exchanged for a local session cookie; requests without the cookie are rejected, so other processes or machines cannot use the proxy port. The server binds to localhost unless you override --host.

If your instance is not on the default port (for example, a second instance on the same machine), pass its API server explicitly with --api-url, using the address from the engine's ready line. The target must report itself as an embedded instance (via /api/v1/meta); the command errors out for any other deployment.

Versioning

hatchet-embedded release tags correspond to the publicly released Hatchet engine versions.

The TypeScript and Python SDKs resolve the version in this order:

1. The version option
2. The HATCHET_CLIENT_EMBEDDED_VERSION environment variable
3. The latest hatchet-embedded release

Running multiple instances

Multiple embedded instances on one machine work out of the box: the first instance's API binds to the default port 28243 and the rest get random free ports (printed on each engine's ready line), gRPC and Postgres ports are auto-allocated, and the bundled Postgres keeps its data in a per-project directory (~/.hatchet-embedded/<hash of working dir>), which also persists across restarts.

Two instances started from the same working directory cannot share the bundled Postgres data directory concurrently. Give each its own via the Postgres data dir option, or point them at an external Postgres.

Run a fleet with a shared database

You can run many embedded engines as one fleet. Point each engine to the same external Postgres database. The engines then operate on the same tenant. They share the task queue. This works the same in Go, TypeScript, and Python, and you can mix them.

Set the database URL when you create each client:

from hatchet_sdk import ClientConfig, EmbeddedHatchetConfig, Hatchet

hatchet = Hatchet.from_embedded(
    ClientConfig(
        embedded=EmbeddedHatchetConfig(
            database_url="postgres://user:pass@db.internal:5432/hatchet"
        )
    )
)

Start more processes with the same database URL to make the fleet larger. Stop a process to make the fleet smaller. The other engines continue the work.

Each engine applies database migrations at startup. Do not let two engines apply migrations at the same time. Start the first engine fully before you start the others, or turn off migrations on all engines except one (WithoutEmbeddedMigrations(), runMigrations: false, run_migrations=False).

Encryption keysets

Without explicit keysets, embedded mode auto-manages them in a schema inside the same database. This is convenient for development, but at-rest encryption offers no additional protection when the keys live next to the data they encrypt. Pass WithKeysets (Go, via embed.StartServer) to manage keys externally.