Hyperlang compared with other BEAM isolation approaches

This document compares Hyperlang with the other ways to run untrusted or partly trusted code on, or next to, the BEAM. Every claim below rests on a probe or benchmark we ran, or on a cited primary source at a stated version. Where we could not check a claim, we say so.

Summary

• Same VM, ordinary Erlang, no ambient authority. A guest is Erlang source. Hyperlang compiles it through its own verifier into a BEAM module that can only call what the program declares. Every authority probe we ported (apply, external funs, computed calls, higher-order laundering, self(), list_to_atom) was rejected at compile time.
• Compute. Guest code is compiled BEAM code. A guest fib(25) took 0.35 ms, the same as native Erlang in a peer node or OS process (0.33 to 0.38 ms). wasmex took 0.68 ms; the interpreted approaches took 85 to 275 ms.
• Memory. An idle guest actor added 23 KiB of resident memory: the program is a shared literal, so actors only reference it. A Rust wasmex instance added 89 KiB, a Luerl state 71 KiB, a Dune session 109 KiB and erl_eval bindings 23 KiB. A peer node or an OS erl process added 74 to 76 MiB.
• Noisy neighbours. Next to 16 CPU-bound tenants, a latency-class victim's p99 rose 1.4x (271 to 370 µs) with Hyperlang.Scope, and 3.2x without scope controls. In Luerl it rose 3.0x, in erl_eval 7.4x and in Dune 36x. A victim in its own peer node rose 1.1x.
• Tenant start. A new tenant of a compiled program answered in 0.05 ms. Luerl took 0.03 ms, wasmex 0.08 ms and Dune 0.41 ms. A peer node or OS process took 111 to 123 ms to boot. Compiling a small program from source is a separate one-time step of about 17 ms.
• Cost. Each call into a guest has a fixed cost: about 21 µs for a run/4 in the caller, 29 µs for an actor call and 45 µs for a run/4 worker on this host. Workloads made of many tiny calls run faster in Luerl's in-caller mode or in wasmex.

All numbers come from one 16-core Apple Silicon host. Other jobs shared it, with a load average of 6 to 32 during the final runs. The BEAM ran with four schedulers (+S 4:4). The method and limitations sections give the detail.

The common wisdom, and where it comes from

The usual position is that the BEAM cannot isolate untrusted code in the same VM. The sources are clear and official:

• Erlang/OTP Secure Coding Guidelines (OTP 29, "What is not protected against"): "All loaded code is assumed to be trusted. There is no built-in sand-boxing mechanism for running untrusted Erlang code." and "Excessive resource usage is not prevented by default."
• EEF Security Working Group, Sandboxing (erlef/security-wg at e7f03719, docs/secure_coding_and_deployment_hardening/sandboxing.md): "It is therefore not possible to isolate 'untrusted' processes in some sort of sandbox." Its advice is "Use an embedded language runtime, such as Lua", and not to use Code.eval_string/1,2,3 on untrusted input.
• EEF, Atom exhaustion: the atom table holds "a default of 1.048.576 entries. If an attempt is made to add a new value while the table is at capacity, the virtual machine crashes." OTP's memory guide adds: "The atom table is not garbage-collected."
• Robert Virding, Elixir Forum, 2017, "How to create a sandbox to run untrusted code modules": "It is basically impossible to sandbox an erlang/elixir system if you allow it to run untested code", and "the only safe way is to interpret/run another language". In the same thread, Michał Muskała wrote: "OS-process-level sandboxing is definitely a much better idea."
• Bram Verburg (EEF Security WG), in the Dune announcement thread: "Personally I would stick with Luerl until the BEAM offers a native, runtime sandboxing feature".
• Dune's own README (v0.3.18): "Dune cannot offer strong security guarantees, and should not be considered a sufficient security layer by itself." and "Due to how the Erlang VM works, an attacker able to escape the sandbox could get full access to the VM without restriction."
• Erlang distribution gives no isolation between nodes. The OTP guidelines say: "All nodes connected through Erlang distribution are assumed to be trusted, and have unrestricted access to all other connected systems."

What is true in it

These parts of the wisdom are correct, and Hyperlang does not change them:

• Shared schedulers. All processes in one VM share the scheduler threads. A busy tenant takes CPU time from others unless something admits and prioritises work.
• Shared allocators and garbage collection. Heap pressure in one tenant costs time in others.
• One atom table. It is global, finite and never collected.
• One code server. Code loading is serialised.
• Native work. A long non-yielding BIF or NIF cannot be preempted.
• Loaded code is trusted. Any BEAM module, once loaded, can do anything.

What Hyperlang changes

The wisdom assumes that guest code becomes ordinary loaded code, or that it runs in an interpreter that needs hooks to stop it. Hyperlang does neither:

• The guest never loads its own code. Hyperlang loads a module that its own code generator wrote. That generator can only emit calls to a closed set of functions (see Isolation by lowering).
• Every atom a guest can cause comes from a fixed set bounded by host limits. Pool atoms are created on first use of each index, and the OTP compiler only ever creates names from a fixed set, because codegen lifts every fun, comprehension and receive into a fixed-name function. 1,000 distinct adversarial programs added no atoms after warm-up.
• Every run has its own reduction, heap and wall limits. Scopes add admission and priority classes on top.
• Every term that crosses the boundary is sanitized. Pids, ports, references and funs do not cross.

Residual effects that remain

The security specification lists these as Known Limitations. They are why hosts that need hard latency or memory bounds against hostile tenants add OS or node isolation:

• Guests share schedulers, allocators, the garbage collector and the code server.
• Heap-pressure latency with scope controls varied between 1.29x and 15.87x of baseline in three trials, so that bound is not reliable.
• Some native calls do not yield. A 32 MB binary:copy/2 ran for about 163 ms before it could be stopped.
• CPU, heap and churn enforcement is sampled and allows short bursts.
• Wall deadlines and timers are not hard real time.

Why the guest language is Erlang

Elixir compilation runs Elixir code. Module bodies, module attributes, macro expansion (defmacro, require), use and its __using__/1 callback, and @before_compile hooks all execute while the module compiles. Nothing has to be called for that code to run.

We checked each form on Elixir 1.20.0 (probes/elixir_compile_time.exs in the comparison harness). Each source wrote a file during Code.compile_string/1, before any function was called:

A sandbox that checks Elixir source or its AST from outside has two choices. It can forbid these constructs, or it can run untrusted compile-time code with the host's authority, with no time or memory bound. Dune 0.3.18 takes the first path. Our probes show that it rejects defmacro, quote, use, require and module attributes. It also does not create real modules: its defmodule is simulated with maps of anonymous functions, and the sanitized AST is evaluated with Code.eval_quoted_with_env/3 (lib/dune/eval.ex:93).

Erlang source has no user-defined code that runs at compile time. Hyperlang's front end does not run the preprocessor (epp). It expands one macro, ?MODULE, by token substitution (lib/hyperlang/otp_frontend.ex, expand_module_macro/1). It admits these attributes: -module, -export (both inert), -record, -behaviour, -vsn, -doc, -moduledoc, -spec, -callback and -optional_callbacks. It rejects every other attribute. Our probes confirmed the rejections: -include, -define, -compile({parse_transform, ...}), -on_load and -nifs all fail at program/1 as parse errors, and so does ?LINE.

The compile itself is metered. The whole parse, verify, code generation and OTP compile path runs in one worker. That worker is bounded by a reduction budget (max_compile_reductions, 200,000,000 by default), a wall-clock backstop and a heap cap. Reductions do not depend on host load, so a program is admitted or rejected the same way on an idle host and on a busy one.

The sound path to sandboxed Elixir

The sound way to sandbox Elixir source is to run the unchanged Elixir compiler inside the isolate. Macro expansion is then guest execution. The same reduction, heap and wall limits that bound any run then bound the compile, so a compile bomb dies by budget. Hyperlang provides the substrate this needs: metered guest execution, a bounded atom universe and capability-scoped surfaces.

Such an effort needs three things from the host. First, the OTP surfaces that the Elixir compiler uses, such as the code server, ETS and file reads, exposed as Hyperlang capabilities. Second, atom creation routed through a bounded encoder. Third, loading of the compiler's output through Hyperlang's verified code generation. Hyperlang does not include an Elixir guest compiler. If you want to work on one, see CONTRIBUTING.md.

Isolation by lowering

Hyperlang enforces isolation in the compiler front end. It does not apply surface checks to source text. The pipeline (lib/hyperlang/compiler.ex) is:

1. Parse. A vendored, renamed copy of OTP's erl_scan and erl_parse reads the source. Its atom-producing paths are patched so that names stay binaries. Parsing creates no atoms.
2. Lower to SafeCore. The parsed forms become Hyperlang's own IR. Unsupported forms are errors.
3. Verify (lib/hyperlang/verifier.ex) over the whole IR: function and record limits, guard-only expressions in guards, and record and variable rules.
4. Classify (lib/hyperlang/classifier.ex). Every call becomes exactly one kind: a local call, a call to a declared binding export, a call to an allowlisted stdlib MFA (lib/hyperlang/stdlib_allowlist.ex), or a Hyperlang-owned surface. The last clause rejects everything else ("remote call M:F/A is not declared"). Dynamic M:F(...), apply/2,3 and external funs are errors. The only dynamic dispatch is source-snapshot apply, which dispatches over a candidate set fixed at compile time.
5. Verify again after classification.
6. Generate code (lib/hyperlang/codegen.ex). Each call becomes a call into a Hyperlang runtime wrapper, which sanitizes arguments and results (lib/hyperlang/terms.ex), or a direct call to a function in a small, fixed set of pure stdlib functions. Receive clauses are wrapped in a message envelope. Atoms are mapped onto the program's pooled universe.
7. Check the generated forms (Codegen.verify_generated_forms/2). Every call in the output must pass one positive gate, generated_call_allowed?/3 in lib/hyperlang/stdlib.ex. Unknown tuple shapes, dynamic calls, !, bare receives and external funs are errors (generated_form_verify).
8. Compile and load the generated module only. compile:noenv_forms/2 runs on Hyperlang's own forms in the reduction-bounded worker. The result is loaded into a code slot.

The generated BEAM code therefore cannot express a call outside the program's closed call graph. This holds even if an earlier stage has a bug, because step 7 checks the output against an allowlist, not the input.

Two results from the sandbox suite show why this matters:

• Guard bodies (fix G1). The verifier once accepted fun, try, case, if and comprehensions inside guards. In one fuzzed program (grammar seed 364), a raw filename:basename/1 call came out of a fun in a guard. filename:* calls are only allowed through a Hyperlang wrapper, and this one had been emitted without it. The generated-form gate (step 7) rejected it before any code loaded. In 36 other seeds, the OTP compiler rejected the output. A check on the source surface would have seen an allowlisted stdlib call and passed it. The verifier now rejects these forms itself.
• Fuzzing. Grammar and mutation fuzzing produced no program that crashed the host, returned a pid, port, ref or fun, or escaped the fault taxonomy.

What the other approaches enforce, and where

We found no text, regex or string-blocklist sandbox among the systems we checked, and we do not describe any of them that way.

To our knowledge, Hyperlang is the first in-VM isolation design for the BEAM that lowers guest source through its own compiler into a closed, per-program call graph and checks the generated code against a positive allowlist before loading it. We checked Dune, Luerl, tv-labs lua, erl_eval with handlers, safeish, wa_embedder (it compiles WebAssembly to BEAM modules but makes no security claim and creates atoms from export names), ptc_runner, QuickBEAM, abacus, wasmex, Extism, Orb and Firefly. Our search was limited to public hex packages, GitHub and the forums cited here.

Partial trust in agent systems

Agent systems run code that is not entirely trusted. Some of it the agent writes itself. Some of it comes as tools and plugins, or as per-tenant logic. The agent's own behaviour can also be steered by untrusted input, through prompt injection or poisoned tool output. Most such code is fine. But it must not hold ambient authority. It must be metered and contained, so that one bad step cannot reach the host, other tenants, or capabilities it was not granted.

The actor model suits this work. Agents are long-lived, stateful, concurrent and message-driven. They fail independently and need supervision. BEAM processes and OTP give that directly. What the BEAM lacks is a way to run partly trusted code inside the VM as an actor with no ambient authority, metered and admitted. So teams leave the BEAM to get it:

• E2B gives every session "a hardware-isolated Firecracker microVM" (e2b.dev). Its docs state that resuming a sandbox "takes approximately 1 second" (Persistence).
• Firecracker's specification states that a microVM with 1 CPU and 128 MiB of RAM boots in at most 125 ms, with at most 5 MiB of VMM memory overhead (SPECIFICATION.md, Overhead).
• smolagents runs generated code in the host process by default. Its docs warn that "no local python sandbox can ever be completely secure" and point to E2B, Docker or other remote executors (Secure code execution).
• AutoGen's DockerCommandLineCodeExecutor "will create a Docker container and run all commands within that container". Its local executor carries the warning "Use it with caution." (Command line code executors).
• Pydantic's mcp-run-python ran Python in Pyodide inside Deno. Its archival notice says "there's just no safe way to run Python within pyodide safely with reasonable latency" (pydantic/mcp-run-python).
• Cloudflare's Dynamic Workers run code in V8 isolates. Cloudflare writes that "An isolate takes a few milliseconds to start and uses a few megabytes of memory" (Sandboxing AI agents, 100x faster). Its Worker Loader API documents load(code) "for one-time AI-generated tool calls".
• The Anthropic API code execution tool runs in a Linux container with 5 GiB of RAM and internet access disabled. Claude Code sandboxes commands with Seatbelt on macOS and bubblewrap on Linux.

Hyperlang gives the BEAM that primitive inside the VM. Guest actors are ordinary Erlang: callback actors, durable actors with storage bindings, channels and timers. They are capability-scoped and metered. In our harness, a new tenant of a compiled program answered in 0.05 ms, an idle guest actor added 23 KiB of resident memory, and a guest message round trip took 0.83 µs. A peer node, the nearest BEAM-native alternative, took 111 ms to boot and added 74 MiB. The README shows an agent running a generated tool as a guest.

The isolation strength differs. A microVM or a separate OS process gives OS-level separation, which Hyperlang does not provide. See Residual effects that remain.

Prior art

Versions are those on hex.pm or GitHub when we checked. Sources were read at the stated tag.

Dune (Elixir, AST rewriting)

• Version: hex 0.3.18, tag v0.3.18 (b82ff4aa), github.com/functional-rewire/dune.
• Mechanism: parse with an atom encoder, sanitize the AST against an allowlist, evaluate with Code.eval_quoted_with_env/3 in a spawned process. The process has priority: :low and max_heap_size: %{size: _, kill: true} (lib/dune/eval/process.ex:61-77).
• Limits: max_reductions (default 30,000), max_heap_size (default 50,000 words), timeout (default 50 ms) and atom_pool_size (default 5,000) (lib/dune/opts.ex:83-91). A watcher process polls reductions.
• Gaps we measured:
  • The heap cap does not set include_shared_binaries. A guest that collected 1 KB strings under a 10M-word (80 MB) cap was stopped after 1.3 s, when host memory had risen by 270 MB.
  • Guest code can read its own pid: self() returned a pid.
  • In a Dune.Session, defining a module with a one-letter name (K or A) and calling it in a later evaluation raised a KeyError in Dune's parser. The exit killed the calling host process (probes/dune_session_crash.exs). Longer names worked.
  • Existing atoms pass through (String.to_existing_atom/1 in lib/dune/parser/atom_encoder.ex:192), so guest code can name any atom already in the VM.
• Self-reported fixes (CHANGELOG): atom leaks (0.1.1), a VM crash via bitstrings (0.3.3), and security fixes in 0.3.14 and 0.3.17.
• Language limits (README): no structs, behaviours, protocols, processes or metaprogramming.

Luerl (Lua on the BEAM)

• Version: hex 1.5.1, tag v1.5.1 (772b25e9), github.com/rvirding/luerl.
• Mechanism: a Lua interpreter whose whole state is an Erlang term. luerl_sandbox removes io, file, parts of os, package, load, loadfile, require, dofile and loadstring (src/luerl_sandbox.erl:48-65). It leaves debug installed.
• Limits: max_time (default 100 ms), max_reductions (default none), and spawn_opts. There is no memory limit unless the caller passes max_heap_size in spawn_opts.
• Gaps we measured:
  • Reductions are polled every 100 ms (wait_reductions/2). An infinite loop under a 10M-reduction limit ran to 36.7M reductions before it was killed.
  • With max_reductions set, even a trivial return 1 took 100 ms, because the first poll sleeps.
  • When the runner died on its heap limit, luerl_sandbox:run/3 did not see the death. It returned {error, timeout} only when max_time (4 s) expired.
  • The runner is spawned without a link (start/4), and only the caller enforces max_time. When we killed the caller of an infinite loop, the runner kept one core busy past its 1 s max_time (probes/orphan_runs.exs).
• Atoms: Lua names stay binaries. 50,000 distinct string keys added 13 atoms after warm-up.

erl_eval with restricted function handlers

• Docs: OTP 29 erl_eval, "Non-Local Function Handler". The handler is called for fun calls, BIF calls, M:F calls and operators. Exceptions: "calls to erlang:apply/2,3" and functions defined by fun ... end expressions are hidden from it.
• Gaps we measured (probes/probes.exs, handler allowing only lists, maps and arithmetic BIFs):
  • F = fun os:cmd/1, F("echo pwned") ran the command.
  • lists:map(fun os:cmd/1, [...]) also ran it.
  • Scanning creates atoms: each program with 2,000 new atom literals added 2,000 atoms. Stock erl_scan calls list_to_atom/1.
  • erl_eval runs in the calling process. A receive in guest code read a message from the host's mailbox.
  • It has no limits. The host must add a process, max_heap_size and a kill timer. With an unlinked evaluator process, a guest loop kept running after its caller died (probes/orphan_runs.exs).
• Elixir: Code.eval_string/3 docs (Elixir 1.20): "Don't use eval_string/3 with untrusted input".

Peer nodes and OS processes

• Mechanism: a separate VM in its own OS process. The OTP peer module (OTP 25+) can connect over standard I/O, without distribution.
• What it gives: separate memory, schedulers and atom table. In our probe, 50,000 atoms created in a peer added 0 atoms to the host.
• What it does not give: any restriction inside the VM. In the peer, os:cmd/1 and file:read_file/1 worked. Over distribution, connected nodes have full access to each other (OTP Secure Coding Guidelines; EEF Distribution).
• OS tools: MuonTrap (hex 2.0.1) applies cgroup limits and kills child processes. Its README says that network sandboxing "is not supported. Look at tools like bubblewrap." ex_sandbox (hex 1.6.0) composes cgroups, namespaces, setpriv and bwrap on Linux.

WebAssembly: wasmex

• Version: hex 0.15.1 (c34f951a), github.com/tessi/wasmex. It pins wasmtime 47.0.3 (native/wasmex/Cargo.lock).
• Mechanism: a Rust NIF over wasmtime. Guest code runs on a Tokio runtime whose thread count equals the number of host CPU cores (native/wasmex/src/engine.rs:9-21). It does not run on the BEAM's schedulers.
• Limits:
  • Fuel is off by default (lib/wasmex/engine_config.ex).
  • Linear memory is unlimited by default; Wasmex.StoreLimits caps it.
  • call_function/4 has a timeout (default 5 s). In 0.15.0 and later, a timed-out call is interrupted by epoch. Our probe confirmed that CPU use stopped after the timeout.
• What we measured:
  • Fuel stopped an infinite loop.
  • A memory.grow loop stopped at the 80 MB store limit.
  • With 16 concurrent infinite loops (one per host core, the size of the Tokio pool), every call returned its 100 ms timeout, but the loops kept running: the BEAM still used 4 to 8 cores 10 to 12 s later, in two runs. With 8 or 15 loops, CPU use dropped to zero after the timeouts (probes/wasm_pool_saturation.exs). We did not find the cause.
  • In the 16-tenant noisy-neighbour cell, the first three victim calls each exceeded 5 s. The host BEAM used 11.0 cores while running with +S 4:4, and 11.8 cores after the cell ended (results/noisy_trial_with_wasm16.log).
• Advisories in range for wasmtime 47.0.3:
  • GHSA-32h6-97mm-8q3c (critical): component async-lifted callback results are unvalidated, causing a native stack buffer overflow; fixed in 48.0.4.
  • GHSA-mghf-wjfp-p5xm (critical): Cranelift on aarch64 can jump to the wrong target on a large br_table; fixed in 48.0.7.
  • We did not check whether wasmex's configuration reaches either path.

Other systems we checked

• safeish (hex 0.5.0, last commit 2022): a BEAM bytecode verifier, broken by bounty exploits (see above).
• tv-labs lua (hex 1.0.2): a pure-Elixir Lua VM. max_instructions is :infinity by default, and its budget is "recoverable via pcall".
• ptc_runner (hex 0.15.0): an interpreted Lisp with a closed atom allowlist. It uses max_heap_size with include_shared_binaries: true. Its docs ask hosts to add an OS or container memory limit for adversarial tenants (lib/ptc_runner/sandbox.ex:64-66).
• QuickBEAM (hex 0.11.3): QuickJS as a Zig NIF, plus a BEAM interpreter of QuickJS bytecode with step, memory and time limits.
• wa_embedder (hex 0.3.3): compiles WebAssembly to BEAM modules. It makes no security claim, creates atoms from export names (src/wa_embedder.erl:503), and has no fuel or reduction limits.
• Firefly: archived (last commit 2023). Extism Elixir SDK: last commit
  2024. Orb: a compiler from Elixir to WebAssembly, not a runtime.

Guarantee matrix

Each cell is labelled. P means a probe we ran (probes/ in the harness). M means a measurement. S means the project's source or docs at the stated version. Spec means Hyperlang's security specification and its tests.

Probe results

These are the outcomes of the ported probes (probes/probes.exs, results/probes.txt). The probes are a subset of Hyperlang's sandbox suite.

Benchmark method

Workloads

Every system runs the same kernels and returns the same values:

• id(42): a trivial call.
• fib(N): naive recursive Fibonacci (fib(20) = 6765, fib(25) = 75025).
• sort(1000): generate 1,000 integers with an LCG (seed 42, x' = (x * 1103515245 + 12345) rem 2^31), sort them, and return the first plus the last. wasmex has no list kernel.
• loop(500): count down from 500 (the noisy-neighbour victim).
• spin(_): loop forever (the noisy-neighbour load).

The sources are in compare/lib/compare.ex: Erlang for Hyperlang, the peer node and the OS process; Elixir for Dune; Lua for Luerl; Erlang expressions for erl_eval; WAT for wasmex.

Configurations

• Hyperlang: Hyperlang.run/4 with default policy (a worker process per run), run_in_caller?: true, and a callback actor from Hyperlang.module/1 called with GenServer.call/2.
• Dune: Dune.Session with the module defined once. Limits: max_reductions: 100_000_000, max_heap_size: 10_000_000, timeout: 5_000.
• Luerl: a luerl_sandbox:init/0 state with the functions loaded. Measured two ways: luerl:call_function_dec/3 in the caller (no limits), and luerl_sandbox:run/3 (spawned, max_time only, because max_reductions adds 100 ms to every run).
• erl_eval: funs bound once. Each call is evaluated in a process spawned with a 10M-word max_heap_size.
• wasmex: one instance per tenant, fuel on, 1 MB memory limit.
• Peer node: :peer.start_link/1 over standard I/O with +S 4:4; the kernel module loaded with code:load_binary/3; calls with :peer.call/4.
• OS process: erl -noshell -noinput +S 4:4 behind a {packet, 4} port; calls are term_to_binary RPCs.
• Native reference: the Erlang kernel compiled with compile:forms/2 and called directly in the host VM.

Procedure

• One host BEAM at a time, with +S 4:4. At most two child VMs.
• Each round runs every system, and the system order rotates between rounds.
• Results report the median and the range over rounds. The load column is the 1-minute load average at each sample.
• Warm calls: the mean over 5,000 (id), 50 (fib(20)), 100 (sort) or 5 (fib(25)) calls, after a warm-up.

Results

Host: 16-core Apple Silicon (macOS), OTP 29.0.2, Elixir 1.20.0, BEAM +S 4:4. Other jobs ran on the host during every measurement.

Tenant start from a compiled program

Each system prepares what it can once: Hyperlang compiles the program, wasmex compiles the module against a shared engine, Luerl loads the state, erl_eval builds the bindings and Dune evaluates the module into a session. Each measured start then creates one new tenant from that prepared form and answers id(42). Child VMs have nothing to prepare and pay their boot. Five rounds; the in-VM rows average 200 starts per round, the child VMs 3. Load average 12 to 15 (bench/prestart.exs).

Start from source

Time from source text to the first answer of id(42), measured on an earlier build of this release. Five rounds; the in-VM rows average 20 starts per round. Load average 7.5 to 7.6, except the Docker row (10.1).

Hyperlang's cold start is a full compile through the verifier and the OTP compiler. The interpreters only build a data structure. The container figure is a lower bound for container isolation: the image was linux/amd64, run under emulation in Docker Desktop's VM, with no BEAM inside.

Warm calls

Mean time per call, in µs. Three rounds; median, with the range in brackets where it matters. Load average 11 to 15.

• Two speed levels. Plain BEAM code ran fib(25) at about 0.37 ms or about 1.0 ms, depending on the run. The ranges above show both levels for the same row. This fits scheduler threads landing on different core types of this Apple Silicon host. We did not pin threads, so compare compute across rows with care.
• Guest against native, same load. bench/guest_cost.exs alternates single guest and native calls so that both see the same conditions. Over 50 pairs, a guest fib(25) through run/4 took a median 1.20x the native time in a fresh process (range 0.93x to 2.84x), and this includes the per-run cost. The medians were 0.39 ms (guest) and 0.36 ms (native) at load average 8.
• Interpreters. On fib(25), Dune, Luerl and erl_eval took 94 to 294 ms (median), against 0.35 to 1 ms for compiled code in the same runs.
• Per-call cost. Each Hyperlang call takes one Scope admission call, fresh per-run limits and sanitization: about 21 µs for a run/4 in the caller, 30 µs for an actor call and 46 µs for a run/4 worker. Luerl in the caller and wasmex have a lower fixed cost for tiny calls.

Throughput under concurrent tenants

Each tenant owns its own instance and calls fib(15) in a loop for 1.5 s. Calls per second for all tenants together (three rounds, median). Child VMs are capped at two by the lane rules.

Load average 10 to 17.

• The BEAM systems run on four schedulers. wasmex runs guest code on its own thread pool, sized to all 16 host cores. Each child VM brings four more schedulers. So the wasmex and child-VM rows use more CPU than the in-VM BEAM rows.
• For calls this small, per-call cost dominates. Hyperlang ran 8x to 20x more calls than Luerl and erl_eval, more than the child VMs, and fewer than wasmex, which uses all host cores.

Memory per tenant

Each row is one fresh BEAM per round, so allocator caches from another system do not leak into the result. The harness warms the system up, forces a garbage collection of every process, waits 1 s, and samples. It then creates N tenants, collects and waits again, and samples a second time. The numbers are the per-tenant difference: OS resident memory (RSS), and the BEAM's own counters. Three rounds, systems interleaved; median and range. Load average 9 to 11. The child-VM rows are from an earlier run at load 9.5 to 14.6.

What each number includes:

• Hyperlang actor. The loaded program is one shared literal in its code slot, which the actor pins while it lives, so actors only reference it. The figure covers the actor process and its scope and limits bookkeeping.
• Hyperlang program. A compiled %Hyperlang.Program{} value is about 35 KB of terms, and a loaded program's shared literal about 46 KB.
• Interpreters (Dune, Luerl, erl_eval): the tenant is a data structure held by the host. Each run then copies it into a fresh process.
• wasmex. erlang:memory/0 does not count NIF memory, so only the RSS column covers the wasmtime store and instance. The guest is the same kernels compiled from Rust (std, default 1 MiB stack, heap), and each instance ran id, fib and sort once, so its stack and heap pages are touched. RSS counts touched pages only; a larger guest touches more.
• Peer node and OS process: the RSS of one child VM started with +S 4:4, idle after loading the kernel module.

An idle Hyperlang actor costs about 3,300x less resident memory than a peer node or an OS erl process, and less than a Luerl state, a Dune session or a wasmex instance. Only erl_eval bindings, which hold no isolation, are as small.

Message round trip

• Hyperlang message ping-pong (bench/message_ping_pong.exs): a guest sends to an echo process through the host's message hook and receives the reply, 50,000 times. Three runs took 0.82, 0.83 and 0.90 µs per round trip at load average 7.4.
• Calls from the host into a tenant: see the id(42) column above. The peer node took 29 µs and the OS process 25 µs.

Victim latency next to noisy neighbours

The victim makes 1,000 sequential loop(500) calls. Each noisy tenant runs spin/1 over and over, and each run is stopped at 100 ms by the system's own control. We report the victim's p99 in µs (three rounds, median and range), the host BEAM's CPU use in cores, and victim calls that failed or exceeded 5 s.

Victim p99 in µs; three rounds, median and range. Load average 12.7 to 32.0. No victim call failed. After every cell, the VM's idle CPU use returned to at most 0.01 cores, so no cell left work running that could bias the next.

• With scope controls, the victim's p99 rose 1.4x from 0 to 16 noisy tenants. Without them it rose 3.2x. Luerl rose 3.0x, erl_eval 7.4x and Dune 36x. The scope rate quota held the 16 noisy tenants to 1.5 cores.
• The peer-node victim stayed flat because its noise ran in another VM with its own four schedulers, on a 16-core host. That is the OS-level separation Hyperlang does not provide.
• wasmex at 16 tenants is not in the table. Sixteen spinning calls fill wasmex's 16-thread pool, and the calls kept running after their timeouts (see WebAssembly: wasmex). In a single trial of that cell, the first three victim calls each exceeded 5 s. The host BEAM used 11.0 cores, and 11.8 cores after the cell ended. Those loops would have biased every later cell, so the final runs skip it.
• Every noisy tenant stops at a flag between runs. Killing a Luerl or erl_eval caller mid-run leaves its runner spinning (probes/orphan_runs.exs). An earlier attempt killed the callers, left orphaned runners, and we discarded its data (results/discarded/noisy_attempt2_orphan_leak.*).

These victims run a short loop, so they measure scheduling delay. Heap pressure and compile load are covered by the specification's own isolation measurements.

Limitations of this comparison

• One shared, loaded host. Other jobs ran during every measurement. The load average ranged from 6 to 32 in the final runs, and reached 100 during discarded attempts. We interleaved systems and report ranges. Absolute numbers will differ on a quiet host.
• Apple Silicon core types. Scheduler threads can land on performance or efficiency cores. Compare compute numbers within one execution context.
• Unequal CPU budgets. The in-VM BEAM systems ran on four schedulers. wasmex used its own pool sized to all host cores. Child VMs added four schedulers each.
• Child VMs capped at two. The lane rules allowed at most two child VMs, so the peer-node and OS-process throughput stops at two tenants.
• Container baseline. The Docker figure is a lower bound under emulation, with no BEAM inside the container.
• Our harness, our configurations. Each competitor ran with the configuration that the comparison describes. Other configurations may do better.
• Probes are a subset. The ported probes cover representative attacks. They are not each competitor's full attack surface, and a passing probe does not prove a system secure.
• Published claims. We did not reproduce vendor figures for E2B, Firecracker or Cloudflare. They are quoted from their docs.

Reproduction

The comparison harness is a separate Mix project. It depends on Hyperlang by path and on the competitors from hex. It is not part of this repository.

export MIX_BUILD_ROOT=$HOME/.cache/mixbuild/hyperlang_pitch/compare ELIXIR_ERL_OPTIONS='+S 4:4'
mix deps.get
mix run bench/compare_bench.exs cold 5
mix run bench/compare_bench.exs warm 3
mix run bench/compare_bench.exs tput 3
mix run bench/compare_bench.exs noisy 3
mix run bench/compare_bench.exs docker 5
./run_mem.sh            # one fresh BEAM per system and round
mix run probes/probes.exs
mix run probes/dune_session_crash.exs
mix run probes/elixir_compile_time.exs
export ELIXIR_ERL_OPTIONS='+S 4:4 +sbwt none +sbwtdcpu none +sbwtdio none'
mix run probes/orphan_runs.exs
mix run probes/wasm_runaway.exs
mix run probes/wasm_stop_midcall.exs
mix run probes/wasm_pool_saturation.exs 16
python3 scripts/summarize.py
python3 scripts/tables.py noisy
python3 scripts/tables.py mem

Hyperlang-only measurements live in this repository:

mix run bench/guest_cost.exs
mix run bench/message_ping_pong.exs
mix run bench/eval_hot_path.exs

The sandbox suite and the fuzz campaigns cited in the matrix run with:

mix test test/sandbox
HL_FUZZ_SEED=20261025 HL_FUZZ_MUT_SEED=77777 HL_FUZZ_N=2000 HL_FUZZ_MUT_N=800 \
  mix test test/sandbox/fuzz_probe_test.exs --include sandbox_fuzz

The grammar campaign generates 2,000 programs and the mutation campaign 800.