How Zero‑Copy GPU Inference From WebAssembly Works on Apple Silicon

# How Zero‑Copy GPU Inference From WebAssembly Works on Apple Silicon

It means a WebAssembly (Wasm) module can run inference while its “linear memory” is also the GPU’s buffer—so both CPU and GPU read and write the same physical DRAM pages, with no host↔GPU copies, no serialization steps, and no intermediate staging buffers. On Apple Silicon, this is practical because the CPU and GPU share memory (UMA), and because you can deliberately “thread the needle” through a specific chain: mmap-allocated pages → Metal MTLBuffer via bytesNoCopy → Wasmtime linear memory via a custom MemoryCreator. If each link is set up correctly, the Wasm guest writes tensors into its own memory, Metal kernels consume and produce results in-place, and the guest reads outputs back through the same pointer.

The Core Idea: One Buffer, Two Worlds

Most GPU pipelines—even on unified-memory systems—still end up copying because frameworks allocate a CPU-side array, then create a GPU buffer that (explicitly or implicitly) duplicates it. The “zero‑copy” claim here is stricter: the Wasm module’s linear memory and the Metal buffer are not merely “synchronized”; they are backed by the same physical bytes. That’s what lets a Wasm guest act as a sandboxed “control plane” while Metal does the heavy compute, without paying the data-movement tax every time you dispatch a kernel.

This is also why the technique is tied to Apple Silicon’s Unified Memory Architecture (UMA). On discrete GPUs over PCIe, the physical separation makes this exact notion of zero-copy generally unattainable: data must cross the bus somehow, so “zero-copy” usually becomes a looser term (like pinned memory or mapped buffers), not literal shared physical pages.

How the Pieces Fit: mmap, Metal bytesNoCopy, and Wasmtime

The pipeline has three parts, and each one matters.

1) mmap: allocating pages Metal can wrap

The starting point is allocating anonymous, private memory on macOS/ARM64 using mmap (e.g., with MAP_ANON | MAP_PRIVATE). The goal isn’t just “get some RAM”—it’s to get page-aligned backing memory that Metal will accept as the storage behind a GPU buffer. Reports around these demonstrations emphasize that alignment constraints can be stricter on Apple Silicon; community writeups cite roughly 16 KB alignment as important for achieving the true no-copy behavior.

If the allocation doesn’t satisfy what Metal expects, Metal may silently fall back to allocating its own storage and copying—breaking the whole premise.

2) Metal: MTLDevice.makeBuffer(bytesNoCopy:length:options:)

Next, you take the host pointer returned by mmap and wrap it with Metal using MTLDevice.makeBuffer(bytesNoCopy:length:options:) (or the equivalent bytesNoCopy API). This is the critical switch: bytesNoCopy tells Metal to use the provided pointer as the buffer’s storage, rather than allocating a new managed region and copying into it.

This is why using Metal’s default buffer creation (for example, allocating by length with options that allow managed behavior) can defeat “zero-copy.” The technique depends on preventing implicit duplication.

3) Wasmtime: adopting the same pages as Wasm linear memory

Finally, the Wasm runtime has to see that exact same memory as the module’s linear memory. Wasmtime provides a hook for this: the MemoryCreator trait (a custom allocator mechanism). By implementing MemoryCreator, the runtime can be instructed to use externally allocated pages—specifically, the same mmap region you already handed to Metal.

At that point, the Wasm guest and the GPU kernel are literally operating on the same backing storage. The demonstrators frame it plainly: “the CPU and the GPU read and write identical physical bytes.”

Why UMA Helps—But Isn’t Enough

UMA is the enabling condition, not the full solution. Apple Silicon’s CPU and GPU share DRAM, which makes a single physical allocation possible—but it doesn’t guarantee your software stack will actually share it. Copies can still happen if:

• the allocation doesn’t meet Metal’s requirements,
• you choose the wrong Metal buffer creation path,
• your runtime can’t adopt externally supplied pages for linear memory.

So the “zero-copy chain” is really about aligning the OS allocator (mmap), the GPU API (Metal bytesNoCopy), and the Wasm runtime (Wasmtime MemoryCreator) so they all agree on a single set of pages.

A Concrete Demonstration: Driftwood and the Reported Numbers

Abacus Noir’s Driftwood project is presented as an end-to-end proof: allocate with mmap, wrap with Metal bytesNoCopy, and expose those bytes to a Wasm guest via Wasmtime’s MemoryCreator. The demo flow is straightforward and persuasive: the guest writes a tensor, Metal compute kernels operate on it in-place, and the guest reads results back—without copying.

The reporting around Driftwood includes a specific datapoint: Llama‑3.2 1B inference at ~9 ms/token on an M1 using this approach. Whatever your baseline is, the practical point is consistent with the architectural claim: removing buffer duplication reduces both latency and memory overhead, which are often the limiting factors for on-device inference.

(For broader context on why inference cost and placement are becoming existential design constraints, see Soaring AI Inference Bills Push Computing to the Edge.)

Technical Caveats and Gotchas

This is not “flip a flag and it’s free.”

• Alignment/page-size constraints are real. If your mmap region doesn’t match what Metal expects (community reporting points to ~16 KB alignment on Apple Silicon), Metal may allocate elsewhere and copy, even if you thought you were doing no-copy.
• Use bytesNoCopy (or the equivalent) intentionally. Metal’s other buffer creation paths may involve managed storage and implicit transfers. The demonstrators stress that bytesNoCopy is essential to avoid hidden copies.
• Synchronization still matters. Shared physical pages don’t eliminate ordering problems. You still need correct CPU↔GPU synchronization so the Wasm guest and Metal kernels observe each other’s writes in the intended sequence. The demonstrations emphasize end-to-end correctness—guest writes, GPU reads/writes, guest reads—only when the proper fences/flushes/command-buffer sequencing are used.

Why It Matters Now

These writeups and demonstrations landed in April 18–19, 2026, and they matter because they turn what sounds like a niche memory trick into an enabling pattern for on-device AI: Wasm as a sandboxed orchestration layer that can directly manage GPU-resident tensors—without paying the copy tax. As more inference shifts to local devices for cost, latency, and privacy reasons, shaving memory movement can be as important as optimizing kernels.

In other words, this isn’t just “Metal is fast.” It’s a statement about system design: the Wasm module can become a modular, portable control plane for inference pipelines while still achieving low-level efficiency on UMA hardware. For a snapshot of the broader moment in tooling and incidents shaping developer priorities, see Today’s TechScan: From Vercel Breach to Voyager Power Cuts.

What to Watch

• OS/driver behavior changes: the approach depends on specific mmap and Metal bytesNoCopy semantics; macOS or Metal updates could change what qualifies for true no-copy.
• Wasm runtime support: Wasmtime’s MemoryCreator makes this possible; more documented examples (or higher-level abstractions) would lower the barrier.
• Independent benchmarks and scrutiny: the ~9 ms/token figure is compelling; broader reproduction and careful analysis will clarify when zero-copy dominates vs. when other bottlenecks take over—and what the security/memory-safety implications are when GPU drivers touch pages shared with a sandboxed runtime.

---

Sources: lilting.ch, abacusnoir.com, stefanosalvucci.com, pulse24.ai, moltbook.com, arxiv.org