# Embedded Hatchet

Hatchet can run in an [embedded mode](https://github.com/hatchet-dev/hatchet-embedded) from inside your workers. The goal is to make local testing as easy as possible and be able to test workers end to end in ephemeral CI environments. There is no need to provision tenants, users, or API tokens to get started.

When using the embedded mode, you get a full Hatchet engine running locally without any external dependencies including Postgres. By default it uses [embedded-postgres](https://github.com/fergusstrange/embedded-postgres) to provision a Postgres database.

## How it works

- **Go** runs the engine in-process: the `hatchet-embedded` package registers itself with the Go SDK via a blank import, and the SDK boots the engine on `NewClient`.
- **TypeScript and Python** run the engine as a sidecar process. On first use the SDK downloads the `hatchet-embedded-sidecar` binary for your platform from the hatchet-embedded [releases](https://github.com/hatchet-dev/hatchet-embedded/releases) (signed and notarized on macOS), caches it under `~/.hatchet/embedded/<version>`, and spawns it. The binary is verified against the release checksums on every start, and the sidecar shuts down with your process.

## Usage

#### Python

```python
from hatchet_sdk import Hatchet

hatchet = Hatchet.from_embedded()
```

Configure the embedded engine through `ClientConfig.embedded`:

```python
from hatchet_sdk import ClientConfig, EmbeddedHatchetConfig, Hatchet

hatchet = Hatchet.from_embedded(
    ClientConfig(
        embedded=EmbeddedHatchetConfig(
            version="vX.Y.Z",
            database_url="postgres://...",
            log_level="info",
        )
    )
)
```

> **Warning:** Python workers run tasks in subprocesses that re-import your main module.
>   `Hatchet.from_embedded()` handles this automatically, and subprocesses connect
>   to the parent's engine instead of booting their own. Keep worker startup and
>   task triggering under an `if __name__ == "__main__":` guard, as with any
>   Python program that uses multiprocessing.

#### Typescript

Embedded mode is a separate entry point, so it stays out of production bundles:

```ts
import { HatchetEmbeddedClient } from "@hatchet-dev/typescript-sdk/v1/embedded";

const hatchet = await HatchetEmbeddedClient.init();
```

Options are passed as the first argument:

```ts
const hatchet = await HatchetEmbeddedClient.init({
  version: "vX.Y.Z",
  databaseUrl: "postgres://...",
  logLevel: "info",
});
```

#### Go

```sh
go get github.com/hatchet-dev/hatchet-embedded
```

Import the package for side effects alongside the SDK; the SDK detects the embedded config and boots the engine on `NewClient`:

```go
import (
	hatchet "github.com/hatchet-dev/hatchet/sdks/go"
	_ "github.com/hatchet-dev/hatchet-embedded"
)

client, err := hatchet.NewClient(hatchet.WithEmbedded())
```

Options are passed to `WithEmbedded`, e.g. `hatchet.WithEmbedded(hatchet.WithEmbeddedDatabaseURL("postgres://..."))`.

#### Ruby

> **Info:** Embedded mode for the Ruby SDK is coming soon. Join our
>       <a href="https://hatchet.run/discord">Discord</a> to stay up to date.

See runnable examples for all three SDKs in the [hatchet-embedded repo](https://github.com/hatchet-dev/hatchet-embedded/tree/main/examples).

## Options

Go, TypeScript, Python, Effect

`WithEmbeddedDatabaseURL(url)`, `databaseUrl`, `database_url`, Use your own Postgres instead of the bundled one
`WithPostgresDataDir(dir)`\*, `postgresDataDir`, `postgres_data_dir`, Store the bundled Postgres runtime and data under this directory
`WithEmbeddedRabbitMQ(url)`, `rabbitmqUrl`, `rabbitmq_url`, Use RabbitMQ instead of the Postgres message queue
`WithEmbeddedAPIPort(port)` / `WithEmbeddedGRPCPort(port)`, `apiPort` / `grpcPort`, `api_port` / `grpc_port`, Bind the API / gRPC servers to specific ports
`WithoutEmbeddedAPI()`, `startApi: false`, `start_api=False`, Start only the engine + gRPC, no REST API
`WithoutEmbeddedMigrations()`, `runMigrations: false`, `run_migrations=False`, Skip running migrations on startup
`WithEmbeddedLogLevel(level)`, `logLevel`, `log_level`, Engine log level (default `warn`)
N/A, `version`, `version`, hatchet-embedded release tag to download (see [Versioning](#versioning))
N/A, `binaryPath`, `binary_path`, Use an existing sidecar binary, skips the download
N/A, `checksum`, `checksum`, Pinned sha256 of the sidecar binary, replaces checksums.txt as the trust anchor

\* `WithPostgresDataDir` is an option on the `embed` package directly; the remaining Go options are `hatchet.WithEmbedded*` client options. The `embed` package also offers `WithAdminUser(email, password)`, `WithKeysets(...)`, and `WithLogger(...)` when you drive the engine yourself via `embed.StartServer`.

The Python options are fields of `EmbeddedHatchetConfig`, set via `ClientConfig.embedded`. Each of these can also be set through a `HATCHET_CLIENT_EMBEDDED_*` environment variable (for example, `HATCHET_CLIENT_EMBEDDED_BINARY_PATH`).

## Serve the dashboard

Embedded instances do not ship a frontend. The `hatchet embedded-ui` CLI command serves the dashboard bundled in the CLI binary on your machine and proxies API requests to the instance's API server. With no flags it targets the default embedded API port (`http://localhost:28243`):

```sh
hatchet embedded-ui
```

This opens a browser to the locally served dashboard. The URL contains a one-time `ui_token` that is exchanged for a local session cookie; requests without the cookie are rejected, so other processes or machines cannot use the proxy port. The server binds to `localhost` unless you override `--host`.

If your instance is not on the default port (for example, a second instance on the same machine), pass its API server explicitly with `--api-url`, using the address from the engine's ready line. The target must report itself as an embedded instance (via `/api/v1/meta`); the command errors out for any other deployment.

Flag, Description

`--api-url`, API server URL to proxy to (defaults to `http://localhost:28243`).
`--profile`, Profile whose API server the UI targets.
`--port`, Port to serve the UI on (defaults to auto-detecting from `8080`).
`--host`, Host interface to bind the UI server to (defaults to `localhost`).
`--no-open`, Do not automatically open a browser.

## Versioning

hatchet-embedded release tags correspond to the publicly released [Hatchet engine](https://github.com/hatchet-dev/hatchet/releases) versions.

The TypeScript and Python SDKs resolve the version in this order:

1. The `version` option
2. The `HATCHET_CLIENT_EMBEDDED_VERSION` environment variable
3. The latest hatchet-embedded release

## Running multiple instances

Multiple embedded instances on one machine work out of the box: the first instance's API binds to the default port `28243` and the rest get random free ports (printed on each engine's ready line), gRPC and Postgres ports are auto-allocated, and the bundled Postgres keeps its data in a per-project directory (`~/.hatchet-embedded/<hash of working dir>`), which also persists across restarts.

Two instances started from the _same_ working directory cannot share the bundled Postgres data directory concurrently. Give each its own via the Postgres data dir option, or point them at an external Postgres.

## Run a fleet with a shared database

You can run many embedded engines as one fleet. Point each engine to the same external Postgres database. The engines then operate on the same tenant. They share the task queue. This works the same in Go, TypeScript, and Python, and you can mix them.

Set the database URL when you create each client:

#### Python

```python
from hatchet_sdk import ClientConfig, EmbeddedHatchetConfig, Hatchet

hatchet = Hatchet.from_embedded(
    ClientConfig(
        embedded=EmbeddedHatchetConfig(
            database_url="postgres://user:pass@db.internal:5432/hatchet"
        )
    )
)
```

#### Typescript

```ts
const hatchet = await HatchetEmbeddedClient.init({
  databaseUrl: "postgres://user:pass@db.internal:5432/hatchet",
});
```

#### Go

```go
client, err := hatchet.NewClient(
	hatchet.WithEmbedded(
		hatchet.WithEmbeddedDatabaseURL("postgres://user:pass@db.internal:5432/hatchet"),
	),
)
```

#### Ruby

> **Info:** Embedded mode for the Ruby SDK is coming soon. Join our
>       <a href="https://hatchet.run/discord">Discord</a> to stay up to date.

Start more processes with the same database URL to make the fleet larger. Stop a process to make the fleet smaller. The other engines continue the work.

Each engine applies database migrations at startup. Do not let two engines apply migrations at the same time. Start the first engine fully before you start the others, or turn off migrations on all engines except one (`WithoutEmbeddedMigrations()`, `runMigrations: false`, `run_migrations=False`).

## Encryption keysets

Without explicit keysets, embedded mode auto-manages them in a schema inside the same database. This is convenient for development, but at-rest encryption offers no additional protection when the keys live next to the data they encrypt. Pass `WithKeysets` (Go, via `embed.StartServer`) to manage keys externally.