Building a New Programming Language in 2024, pt. 1

Here at BoundaryML, we've spent the past year building BAML, a new programming language, with the goal of making it easy for developers to work with LLMs. We wanted to talk a bit about the engineering involved in this process (we'll discuss how we ideate on syntax in a future post).

Why make a language?

For argument's sake, let's assume we've agreed that building a new language is a good idea.

Requirements

At a minimum, users need to be able to:

• install it,
• write some code, and
• run it.

In addition, users also expect all of the following:

• a way to try it out without installing it
• easy-to-understand, actionable error messages
• a community: a place to ask questions and provide feedback
• IDE integration
  • syntax highlighting
  • inline diagnostics
  • jump-to-definition
  • autocomplete

BAML also isn't a general-purpose programming language, which means that we have one more critical requirement:

• interop with the languages our users write their software in (e.g. Python, TypeScript, Ruby)

This is by no means an exhaustive list (e.g. a standard library, toolchain manager, package manager, and package registry), but it's what we've started with.

Open-source and free

We also believe that making a language open-source, and committing to keeping keeping our toolchain open-source and freely available, is a core requirement for success.

That's why all the BAML source code is available on GitHub and licensed using Apache 2.0.

What we've built

Requirements are nice and all, but that doesn't say a lot about the actual engineering work involved in satisfying these tasks.

Choosing Rust 🦀

Building BAML in Python or Typescript was a no-go for us. Since we need BAML to be able to interface with any language, we need to provide a bare-bones foreign function interface (FFI) for each language we support, so we had to use something more low-level.

We wrote our first compiler in C++, but we've since migrated to Rust (even though we didn't know any Rust when we started). Here's a few reasons why:

• Running a team with maintainable C++ code is hard
• It's hype 🚀
• You can design your code such that if it compiles, it's probably correct
• We didn't have to start from scratch, and were able to use the Prisma implementation as a great reference for directory structure and organization
• We first thought that it wouldn't take more than 4 days to port our C++ code to Rust. It ended up taking 6 weeks 🤡

Other benefits we later discovered:

• There's a package for everything in Rust
• The UX in VSCode is great for writing and testing Rust (this actually inspired the way we thought about testing in BAML)
• Learning Rust is much easier with the help of tools like Github Copilot

BAML itself: a compiler, runtime, and CLI

To implement our compiler:

• we use a Pest grammar to lex the source code (inspired by Prisma),
• we do a first compiler pass to build an abstract syntax tree (AST), where we
  • resolve symbols and types,
  • collect diagnostics for malformed syntax trees (e.g. a partially-defined function), and
  • once we've verified the syntax tree is well-formed, we build an intermediate representation (IR)¹,
• we then do a second compiler pass using the IR to
  • apply static analysis to our template expansion on jinja (we use minijinja for our templates, and built our own type checking using its exposed AST), and
  • (optionally) generate types and bindings in our target languages (Python, TypeScript, Ruby) for the types and functions that our users have defined in BAML.

Once users generate bindings for their BAML functions, they can call those bindings, which then calls into the BAML runtime:

• we use pyo3 to expose our runtime in Python;
• we use napi-rs to expose our runtime in Node.js; (highly recommend napi-rs over neon)
• we use magnus and rb-sys to expose our runtime in Ruby; and
• we use wasm-bindgen to expose our runtime in WASM, for use in the VSCode extension and browser.

We use these same techniques to expose baml-cli in each target environment - the per-language customization is just how we retrieve argv (c.f. the Python implementation) - which allows us to guarantee the same behavior in each environment.

An installer

Since BAML isn't a general-purpose programming language, and is meant to be used via interop with user's preferred languages - e.g. Python, TypeScript, Ruby - we can rely on the package registries of those languages to distribute our compiler, runtime, and CLI.

(If this wasn't the case, we'd need to ship at least a portable installer script, like Docker does, or our own apt and brew sources.)

A way to try it out without installing it

This is promptfiddle.com, which is implemented using:

• CodeMirror, to provide IDE-like features (we considered Monaco, but wanted to keep as much of the logic in-browser as possible, and were inspired by repl.it's bet)
• a Lezer grammar for BAML, to back the syntax highlighting
• our own in-memory file tree, to provide an IDE-like experience
• a WASM build of our compiler and runtime, to provide real-time inline diagnostics

IDE integration

We provide IDE integration using a VSCode extension:

• a TextMate grammar powers syntax highlighting (if you're keeping count, yes, that's three different BAML grammars)
• the extension is implemented in TypeScript, and uses vscode-languageclient to communicate with the language server
• the language server (also implemented in TypeScript²) provides diagnostics, jump-to-definition, and autocomplete³, and is implemented using vscode-languageserver and the WASM bindings to the BAML compiler; and
• a React app in a VSCode webview powers the in-VSCode playground, which allows users to run individual BAML functions interactively.

Documentation

Our documentation is currently hosted on Mintlify, and is written using MDX (Markdown-React).

The current way we've broken down docs.boundaryml.com is to have sections for:

• quickstart,
• language tutorials,
• usage tutorials (i.e. how to actually call BAML functions from your Python/TypeScript/Ruby code), and
• reference documentation.

We'll likely have to migrate soon, though, because Mintlify doesn't support custom syntax highlighting and has no plans to do so.

Community

We provide a Discord server where users can ask questions and provide feedback in real-time, which is the primary way we engage with our community. (We would offer a community Slack, except Slack has no community pricing.)

Lately, we've also had users engage with us via GitHub Issues and Discussions (we've set up Slack notification bridges for these), as well as on Hacker News and Reddit - we recognize that we need to meet our users where they are.

More to come!

We'll dive more into the details of all this in the next posts.

Footnotes

1. The distinction between an AST, IR, or any other flavor (e.g. Rust MIR, LLVM LLIR) is somewhat arbitrary and tends to just reflect the abstractions used by a given compiler; our distinction, right now, is that we guarantee our IR is well-formed but we make no such guarantee for our AST. ↩

2. We considered using the NAPI-RS bindings to the BAML compiler - we know it's possible to ship VSCode extensions with native dependencies, but we were concerned about how much work we'd have to put into making our NAPI-RS bindings build for Electron. ↩

3. We've only actually implemented autocomplete suggestions for one specific case so far; doing it more generally is on our to-do list. ↩