Adopting Boundary and BAML

Recommended

Adopt BAML incrementally

BAML generates a typed SDK for most languages, so your app calls it like its own code.

Try it with

  1. 1
    baml_src/triage.baml
    enum Label {
  BILLING
  BUG
  OTHER
}

function classify_ticket(body: string) -> Label {
  client: "openai/gpt-5-mini"
  prompt: `
    Label this support ticket.
    ${body}
  `
}
    app/tickets.py
    from baml_sdk import classify_ticket, Label

label = await classify_ticket(ticket.body)
if label == Label.BUG:
    escalate(ticket)
    $ baml generate add python/pydantic && baml generatethen pip install baml_bridge
  2. 2

    BAML is open source (Apache 2.0), and on your machine it records every call with no account. For hyperobservability in production, get early access to Boundary Cloud.

What you get
Those functions get hyperobservability. Boundary can trace the rest of your code too, so you can move your whole tracing stack off OTel.

Send your OTel traces to Boundary Cloud

Works with whatever your backend is written in.

Good to know
Since it’s OTel, we can’t do much for performance or storage.

  1. 1

    Boundary Cloud is in early access. Get early access to start.

  2. 2

    Point the OpenTelemetry exporter your services already use at Boundary Cloud.

    OTEL_EXPORTER_OTLP_ENDPOINT=<your Boundary Cloud endpoint>
  3. 3

    Redeploy. No code changes, in any language.

What you get
Every service lands in Boundary Cloud: one place to see and query what your backend does.

Write new services in BAML

For new services, where there is nothing to migrate.

Why not rewrite yet
We don't recommend rewriting existing services yet. BAML's standard library and package ecosystem are still young, and a rewrite trades code that works for risk. Move the functions that benefit most first, and save full BAML for new services.

  1. 1

    Start a project.

    baml new my-service
  2. 2

    Run any function or script.

    baml run -f handle_ticket
  3. 3

    Ship it as one self-contained binary, no host language.

    baml pack -f handle_ticket
  4. 4

    BAML is open source (Apache 2.0), and on your machine it records every call with no account. For hyperobservability in production, get early access to Boundary Cloud.

What you get
Everything you write is hyperobservable from the first run. Nothing else to deploy next to it.

Under the hood

How BAML runs

For the curious. BAML compiles when you build and runs inside your app as a library, with no sidecar and no server. Pick where it runs, and the diagrams below follow.

Build, then run

baml generate type-checks and compiles your BAML, then writes an SDK in your language with the compiled program inside. Your app ships with it.

When you build

On your machine, or in CI.

baml_src/

triage.baml · invoices.baml

$ baml generate

baml_sdk/

__init__.pyyour functions, typed

__init__.pyitypes for your editor

_inlinedbaml.pyyour compiled program

_typemap.pyBAML types ↔ Python types

py.typedmarks the package as typed

ships with your app →

When it runs

BAML runs inside your Python process and loads when you import the SDK. Your code keeps its threads, BAML runs on its own, and the bridge connects them.

your Python process

your code, on your threads

main threadthread 2

baml_sdk (generated)

baml_bridge · native extension (pyo3)

A call crosses to BAML’s workers. Its result crosses back.

BAML runtime, on its own threads

baml-worker 1baml-worker 2baml-worker 3

What happens at release

What ships with your app, and what to set so production runs the same as your machine.

The runtime is inside the baml_bridge wheel, for Linux (glibc and Alpine), macOS and Windows, on x64 and arm64.

  1. 1

    Install your requirements in the image. There’s nothing else to fetch.

    RUN pip install -r requirements.txt

Pin one version

Keep baml.toml’s toolchain version, your SDK package and the generated code on the same version. If they drift, startup fails with a version skew error.

In production

BOUNDARY_API_KEY sends recordings to Boundary Cloud. BAML_TELEMETRY sets how much is recorded: off, low, medium (the default) or high.

What happens at startup

Only the first call pays for this. Along the way the runtime sets up its sysops: the way BAML reaches the outside world, like env vars, files and the network.

import baml_sdk

Once per process

Loads the native extension

Sets up its sysops

Decodes your compiled program

Checks the toolchain version

Starts BAML’s worker threads

Every call after this goes straight to BAML’s worker threads.

A sysop is a function in BAML’s standard library that reaches outside BAML. Sysops tie BAML into your language.

When BAML needs toit uses
  • Read an env varbaml.env.get("OPENAI_API_KEY")your process’s env varsthe ones os.environ sets
  • Get the working directorybaml.sys.current_dir()your working directoryos.chdir moves it
  • Make an HTTP requestbaml.http.fetch(url)BAML’s own HTTP clientit reads the proxy env vars
  • Call your codeescalate(body) // yoursyour Python functions
  • Exitbaml.sys.exit(1)ends your whole process

For example, change a variable between two calls:

os.environ["OPENAI_API_KEY"] = "sk-old"await classify_ticket(body)  # sends sk-oldos.environ["OPENAI_API_KEY"] = "sk-new"await classify_ticket(body)  # sends sk-new

What happens when BAML throws an exception

What if your BAML code has an error? It depends on the kind.

Expected

Your BAML function throws an error it declares

You get a normal Python exception. Catch it with try/except.

Panic

Your BAML code panics, like dividing by zero or a failed assert

You get a BamlPanic. It’s a BaseException, so a bare except Exception won’t swallow it by accident.

let arr = [1, 2, 3];
// panics with IndexOutOfBounds
let x = arr[5];

Also division by zero, a missing map key, a failed assert, stack overflow, integer overflow, and baml.sys.panic(…). None of these appear in a function’s throws.

Unhandled

A background task throws, and nothing waits for it

BAML prints it and exits, like any uncaught exception. Change that with set_unhandled_spawn_error_handler.

Bug in BAML

The BAML runtime itself fails

You get a BamlPanic carrying an SdkPanic, and your program keeps running. A crash during garbage collection still aborts.

⚠ This should never happen. BAML should never crash or segfault your process. If you see one, tell us on Discord and we’ll fix it.

What happens at shutdown

BAML finishes its work before your program exits.

  1. Python starts to exit
  2. BAML’s exit hook runsatexit
  3. Waits for calls and spawned tasks still running
  4. Reports errors nothing observed
  5. Finishes writing recordings
  6. Python exitsoptional timeout cancels the rest

What happens when you change an environment variable

The simplest way to see sysops at work: BAML reads the environment when it needs a value, instead of keeping its own copy.

your code

os.environ["OPENAI_API_KEY"] = "sk-old"await classify_ticket(body)  # sends sk-oldos.environ["OPENAI_API_KEY"] = "sk-new"await classify_ticket(body)  # sends sk-new
  • It reads at the request. env.OPENAI_API_KEY holds the variable’s name, not its value, so a secret you load after startup is used from the next call on.
  • BAML only reads. There’s no env.set: changes come from your code. For a single command, baml.sys.run takes env overrides that only that child process sees.
  • Clients never hold the key. Printing or recording a client can’t leak it.
  • Proxies follow too. Change HTTPS_PROXY, NO_PROXY or SSL_CERT_FILE and the next request uses them.
  • Set variables before starting calls on other threads. Changing the environment isn’t thread-safe in C, or in any language built on it.