Senior GPU Inference Performance Engineer - CA, Santa Clara, United States

Originally posted 9 July 2026 by the employer.

About the role

You will profile, diagnose, and explain performance while driving competitive positioning across AMD Instinct and alternative accelerator platforms.

What you'll do

- Instrument and analyze inference workloads across AMD Instinct (ROCm, rocProfiler, ROCm Systems Profiler, RGP, rocprof-compute, rocprof-sys, Omniperf) and NVIDIA (CUDA, Nsight Systems/Compute, DCGM) GPUs to identify bottlenecks across HBM bandwidth, compute utilization, kernel scheduling, memory allocation, and PCIe/Infinity Fabric data movement. - Profile and diagnose performance interactions between GPU runtimes, Linux operating systems, device drivers, container runtimes, memory subsystems, CPU scheduling, NUMA topology, and I/O pathways. - Design and execute head-to-head benchmarks on standardized AI and LLM workloads, producing data-backed explanations attributing gaps to hardware architecture, networking topology, communication libraries, software maturity, runtime behavior, or configuration differences. - Profile and optimize distributed inference topologies—including prefill-decode disaggregation, pipeline parallelism, and tensor parallelism across multi-node clusters—using RDMA/RoCE traces, NCCL/RCCL collective profiling, GPUDirect RDMA, and NIC-level counters. - Profile latency overhead, jitter, cold-start, and resource contention introduced by GPU operators, device plugins, container runtimes, and Kubernetes scheduling. - Build reproducible benchmarking harnesses, profiling scripts, and performance regression dashboards to automate trace collection across firmware, drivers, networking stacks, runtimes, and AI frameworks.

What you'll need

- Experience in GPU performance engineering, distributed systems, networking, operating systems, or systems performance analysis. - Demonstrated ability to read GPU traces and diagnose system-level bottlenecks impacting throughput, latency, and GPU utilization. - Strong Python and C/C++ skills with comfort reading GPU kernel code (HIP/CUDA), runtime code, or systems-level software. - Experience with Linux systems, device drivers, or container runtimes (Docker, containerd).

Nice to have

- Hands-on proficiency with either AMD (ROCm, rocProfiler, ROCm Systems Profiler, RGP, rocprof-compute, rocprof-sys, Omniperf/Omnitrace) or NVIDIA (CUDA, Nsight Systems/Compute, NCU) profiling toolchains, with deep understanding of GPU architecture (warp/wavefront execution, memory hierarchy, occupancy, and instruction-level parallelism). - Experience analyzing GPU communication and networking performance including NCCL/RCCL, RDMA/RoCE, GPUDirect RDMA, UCX, MPI, ConnectX, Pensando, or similar high-performance networking technologies. - Experience with multi-GPU and multi-node inference, training, or HPC environments including tensor parallelism, pipeline parallelism, distributed communication libraries, and network performance analysis tools. - Experience with Kubernetes GPU scheduling, MIG, GPU operator performance, or contributions to open-source infrastructure, systems, networking, inference, or profiling projects. - Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or a related technical field; advanced degree desired.

Skills: GPU performance analysis, AI inference workloads, LLM serving frameworks, HBM bandwidth, compute utilization