Senior Staff Inference Performance Engineer - Santa Clara, United States

Originally posted 15 October 2026 by the employer.

Join d-Matrix to optimize the end-to-end performance of LLM serving paths across Corsair accelerators and GPUs.

About the role

This role involves owning the end-to-end performance of workloads deployed on d-Matrix’s inference services, ensuring they perform at least as well as on bare metal. This includes optimizing speculative decoding and attention-FFN disaggregation across d-Matrix Corsair accelerators and GPUs.

What you'll do

• Diagnose performance problems across the full serving path, from orchestration to inference internals.
• Isolate problems to specific layers and owners, deciding whether to fix directly, route, or both.
• Write minimal, reproducible problem statements for cross-stack findings.
• Protect throughput, latency, and stability targets for orchestrated workloads.
• Assess what it takes to scale from single deployments to fleet operation.
• Capture each workload’s node requirements in the orchestration layer’s configuration.
• Build lightweight telemetry and benchmarks into prototype stacks to isolate problems.
• Turn recurring investigations into playbooks and mentor engineers in the profiling-first method.

What you'll need

• 8+ years of hands-on performance or systems engineering, including resolving hard regressions across multiple layers of a stack.
• Track record of finding performance overhead through profiling in unfamiliar, sparsely documented systems.
• Deep systems fundamentals, including Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths.
• Experience running performance-sensitive workloads on Kubernetes, including identifying and closing gaps due to scheduler defaults, network overlays, or orchestration abstractions.
• Working knowledge of LLM inference serving, including batching, KV-cache, prefill/decode, quantization, and speculative decoding.
• Proficiency in Python and C/C++, sufficient to instrument, debug, and patch code across the stack.
• Clear technical writing for hardware, firmware, or platform teams.
• Sound judgment about ownership, with examples of correctly handing off problems.
• Fluency with AI and agentic tools for investigation, code search, log triage, and drafting instrumentation.

Nice to have

• Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM.
• Hands-on experience building or tuning speculative decoding, disaggregated serving, or heterogeneous serving architectures.
• Experience tuning KV-cache management, request and batch scheduling, or weight prefetch and residency in an LLM serving system.
• GPU programming and profiling experience (such as CUDA, Nsight, NCCL).
• Experience with pre-production or bring-up hardware, or in organizations where root causes routinely cross team boundaries.
• Rust experience.

Skills: Inference Performance, LLM serving path, Kubernetes, Linux internals, GPU programming