Runtime Engineer - Mountain View, CA, United States

Originally posted 26 May 2026 by the employer.

About the role

MatX is seeking a Runtime Engineer to own the host-side stack and the contracts connecting compiler, RTL, simulator, ISA, and kernel teams. The role focuses on custom silicon for large-language-model inference and training.

What you'll do

- Build the host-side interface library handling device memory management, DMA, streams, events, and sync primitives. - Own and extend the executable format and compiler-to-runtime contract, including versioning and weight and quantization layouts. - Design the custom-kernel ABI, calling conventions, sync semantics, and host-side marshaling layers using DLPack, the buffer protocol, and NumPy. - Build Python bindings via PyO3 with a C-ABI shim for downstream integration paths. - Develop the LLM inference serving stack, including paged KV cache, continuous batching, request scheduling, token streaming, and cluster orchestration primitives. - Bring up host interconnect topology, failure detection, and clean-teardown recovery paths across racks, while designing host-side profiler and debugger interfaces.

What you'll need

- Strong systems programming experience in Rust, C, C++, or Go, covering memory management, allocator design, and FFI/ABI work. - Experience building production Python interop layers using PyO3, ctypes, pybind11, or C-ABI bridging. - Experience designing and maintaining API or ABI contracts between teams with versioning and breaking-change discipline. - Hands-on experience with accelerator programming models such as CUDA, ROCm, oneAPI Level Zero, or TPU for device memory, async execution, and kernel launch. - Working knowledge of ML systems, including training and inference loops, collectives, and tensor layouts.

Nice to have

- Knowledge of LLM inference internals such as vLLM, TensorRT-LLM, or SGLang (paged attention, scheduler design). - In-depth Rust experience, including proc macros, unsafe code with soundness reasoning, and complex lifetime/trait work. - Experience with custom allocator design (slab, paged, arena) or low-level memory systems. - ML framework integration experience with PyTorch custom backends, JAX/XLA, or ONNX runtime. - Profiler or tracing infrastructure experience using Perfetto, Nsight, or custom stacks. - Driver-adjacent or kernel-bypass experience, or prior new-silicon bring-up.