Software Engineer, GPU Inference - Sunnyvale, CA, United States

Originally posted 25 November 2025 by the employer.

Productionize and optimize the GPU serving stack for Cerebras' disaggregated AI inference systems.

About the role

This role involves productionizing and optimizing Cerebras' GPU serving stack across custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure.

What you'll do

• Productionize the GPU inference stack, covering API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.
• Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet.
• Define service-level indicators and objectives for GPU-backed inference, improving fault isolation, automated recovery, and incident response.
• Profile and optimize time to first token, request throughput, tail latency, GPU utilization, memory efficiency, and rack-level capacity.
• Optimize model-serving behavior, including scheduling, continuous batching, prefix caching, KV-cache management, and distributed communication.
• Debug complex failures and performance regressions across application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware, networking, and hardware.
• Build validation and regression infrastructure for model quality, numerical accuracy, precision changes, quantization, and determinism.

What you'll need

• 5+ years of software engineering experience with individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or demanding GPU workloads.
• Strong programming ability in C++ and Python, including multithreading, concurrency, memory management, and performance-sensitive software.
• Hands-on experience with vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent high-performance model-serving framework.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
• Experience debugging distributed systems across multiple layers.
• Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services.
• Ability to design rigorous benchmarks, interpret performance results, identify bottlenecks, and translate findings into production improvements.
• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Nice to have

• Experience with AMD Instinct accelerators and the ROCm ecosystem, including HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, or related libraries and tools.
• Deep CUDA experience demonstrating an ability to transfer GPU systems knowledge across accelerator platforms.
• Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT-LLM, or another open-source ML systems project.
• Experience optimizing prefill-heavy or disaggregated prefill/decode inference architectures.
• Understanding of KV-cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and memory-aware admission control.
• Experience with multi-GPU and multi-node inference, including tensor parallelism, pipeline parallelism, expert parallelism, RDMA, collective communication, and failure handling.
• Experience optimizing Mixture-of-Experts or multimodal models.
• Knowledge of GPU kernel optimization, operator fusion, graph capture, attention kernels, GEMM tuning, and communication/computation overlap.
• Experience with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of their numerical and model-quality effects.
• Experience building numerical-comparison, determinism, model-validation, or performance-regression test systems.
• Experience collaborating directly with accelerator vendors, framework maintainers, or open-source communities.