How BAML works
BAML compiles when you build and runs inside your app as a library. No sidecar, no server.
What happens when you build?
When you build
On your machine, or in CI.
baml_src/
triage.baml · invoices.baml
baml_sdk/
src/lib.rsyour functions, typed
src/_inlinedbaml.binyour compiled program
src/_runtime.rsstarts the runtime
Cargo.tomla crate of its own
ships with
your app
When it runs
BAML runs inside your Rust program and loads on the first call, or when you call init(). Your code keeps its threads, BAML runs on its own, and the bridge connects them.
your Rust program
your code, on your threads
baml_sdk (generated)
baml_bridge crate · loads the BAML library at run time
A call crosses to BAML’s workers. Its result crosses back.
BAML runtime, one worker thread per CPU core
What happens at release?
The runtime is a shared library. By default it downloads once per machine and version, on the first call, into $BAML_HOME/bridges, checked against its SHA-256.
- 1
Warm the cache in your build stage.
baml_bridge::loader::ensure_library_cached()?;
- 2
Copy the cache into the image and turn downloads off.
COPY --from=build /root/.baml/bridges /opt/baml/bridges ENV BAML_HOME=/opt/baml ENV BAML_BRIDGE_DISABLE_DOWNLOAD=1
Pin one version
Keep baml.toml’s toolchain version, your SDK package and the generated code on the same version. If they drift, startup fails with a version skew error.
In production
BOUNDARY_API_KEY sends recordings to Boundary Cloud. BAML_TELEMETRY sets how much is recorded: off, low, medium (the default) or high.
What happens at startup?
classify_ticket(body).await
your first call
Downloads the BAML library
once per machineinto ~/.baml/bridges
Opens it
once per process
Sets up its sysops
once per process
Checks the toolchain version
once per process
Decodes your compiled program
once per process
This happens on the first call, or when you call init(), once. After that, calls go straight to the runtime.
What happens when you call a BAML function?
Your code calls classify_ticket
The bridge hands it to a BAML worker thread
the first call starts the pool
Builds the prompt, calls the model, parses the reply
Your code gets a typed result back
BAML does the work on its own threads, not yours.
What happens when BAML throws an exception?
Your BAML function throws an error it declares
You get an Err back.
Your BAML code panics, like dividing by zero or a failed assert
You get an Err back.
let arr = [1, 2, 3];
// panics with IndexOutOfBounds
let x = arr[5];Also division by zero, a missing map key, a failed assert, stack overflow, integer overflow, and baml.sys.panic(…). None of these appear in a function’s throws.
A background task throws, and nothing waits for it
Nothing. The Rust SDK doesn’t report these yet.
The BAML runtime itself fails
You get your language’s BAML panic, carrying an SdkPanic, and your program keeps running. A crash during garbage collection still aborts.
⚠ This should never happen. BAML should never crash or segfault your process. If you see one, tell us on Discord and we’ll fix it.
What happens at shutdown?
The program starts to exit
BAML’s exit hook runs
atexit
Waits for calls and spawned tasks still running
Finishes writing recordings
The program exits
BAML finishes its work before your program exits, as long as its exit hook runs.
What happens when a function crosses between your code and BAML?
You pass a function to make_widget
|name| name.to_uppercase()
It crosses as a handle
a reference to your function, not a copy
BAML calls it, and waits for the answer
Your function runs on your side
on the bridge’s dispatch pool
The result crosses back as a value
checked against the type BAML declared
make_widget returns a Widget
data copied, functions as handles you can call later
Data crosses as a copy. Functions cross as handles, freed when the side holding one lets go.
Why don’t you have docs?
Docs for humans are coming by the end of 2026. Until then, we’re more than happy to make time with you directly.
Meanwhile, we’ve worked hard to ensure agents can understand and use the toolchain well. Seriously, ask Claude: “Install BAML and be ambitious with it.”
I know how big a bet a new programming language is. We’re engineers too, and we’re happy to talk it through with you. Email me directly.
Vaibhav Gupta, co-founder and CEO
Meet the team →