$160,000 - $250,000 USD yearly
Build the host-side runtime stack and custom-kernel ABI for custom LLM silicon using Rust and PyO3.
Originally posted May 27, 2026 by the employer.
About the role
MatX is building custom silicon for large-language-model inference and training with hardware/software co-design across ISA, RTL, simulator, compiler, and kernels. The Runtime Engineer owns the host-side interface library, the executable contracts, and the LLM inference serving stack.
What you'll do
- Build the host-side interface library covering device memory management, DMA, streams and events, and synchronization primitives.
- Own and extend the executable format, including compiler-to-runtime contracts, versioning, and weight/quantization layouts.
- Design the custom-kernel ABI and the host-side marshaling layer using DLPack, buffer protocol, and NumPy to transfer Python tensors to the device.
- Develop Python bindings via PyO3 alongside a C-ABI shim for downstream consumer integration.
- Build the LLM inference serving stack, including paged KV cache, continuous batching, request scheduling, token streaming, and cluster orchestration primitives.
- Bring up interconnect topology from the host and own the failure-detection and teardown path for recovery across racks.
What you'll need
- Strong experience in systems programming languages such as Rust, C, C++, or Go, including memory management, allocator design, and FFI/ABI implementations.
- Experience building production Python interop layers using PyO3, ctypes, pybind11, or equivalent C-ABI bridging.
- Track record of designing and maintaining API or ABI contracts between teams with disciplined versioning and evolution.
- Hands-on experience with accelerator programming models (CUDA, ROCm, oneAPI Level Zero, TPU, or comparable) regarding device memory, asynchronous execution, and kernel launches.
- Working knowledge of ML systems, including training and inference loops, collective operations, and tensor layouts.
Nice to have
- Knowledge of LLM inference internals such as vLLM, TensorRT-LLM, or SGLang (paged attention, scheduler design).
- Deep experience in Rust, including procedural macros, unsafe code with soundness reasoning, and complex lifetime/trait implementations.
- Experience with custom allocator design (slab, paged, arena) or other low-level memory engineering.
- Integration experience with ML frameworks such as PyTorch custom backends, JAX/XLA, or ONNX runtime.
- Experience building or using profiler and tracing infrastructure like Perfetto or Nsight.
- Driver-adjacent development, kernel-bypass engineering, or prior new-silicon bring-up experience.
More semiconductor jobs: Embedded software jobs
First seen in our index 87 days ago, well above the 65-day median for Systems Software Engineering roles.
Systems Software Engineering · Systems Software
|
Open roles in category
725
|
Median days since first seen
65 d
|
Category median salary (all employers)
$220k
|
See the full market breakdown ▾Category comparison, and who else is hiring
| Metric | MatX | All employers we track in this specialty (725 roles · 78 employers) |
|---|---|---|
| Open roles in this specialty | 2 | 725 |
| Open roles in the wider Software, Firmware & Systems family | 12 | 6787 · 180 employers |
| Median days since first seen | 87 d | 65 d (−22 d vs this employer) |
| Median salary (USD postings) | n/a | $220k |
Who's hiring in this category
- Qualcomm · 179 open roles · median 70 d
- NVIDIA · 174 open roles · median 61 d
- AMD · 59 open roles · median 58 d
- Arm Holdings · 33 open roles · median 19 d
- Mobileye · 25 open roles · median 101 d
- Intel Corporation · 16 open roles · median 26 d
How we counted: 725 open Systems Software Engineering (Systems Software) roles from 78 employers tracked in the SemiconductorJobs index, counted 16 Nov 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there, and is counted in the wider Software, Firmware & Systems family row. Days are counted from when our index first saw each posting, not from the employer's original posting date.