$129,600 - $220,300 USD yearly
Design, build, and operate production-grade AI infrastructure for large-scale GPU training and inference on Kubernetes.
Originally posted June 23, 2026 by the employer. Relisted by the employer on June 23, 2026.
About the role
Design, build, and operate production-grade AI infrastructure for large-scale GPU training and inference. The role owns cluster architecture, distributed workload orchestration, accelerator management, and performance engineering.
What you'll do
- Design and deploy scalable, multi-node GPU clusters on Kubernetes, covering compute, networking, storage, scheduling, security, and observability.
- Build distributed training and reinforcement learning platforms using Ray and NVIDIA NeMo RL to support frameworks such as PyTorch and JAX.
- Deploy and optimize high-throughput LLM inference utilizing vLLM, SGLang, and NVIDIA Dynamo.
- Implement GPU scheduling, quotas, isolation, autoscaling, health monitoring, and capacity management for multi-tenant environments.
- Profile and troubleshoot GPU workloads using NVIDIA Nsight Systems, Nsight Compute, DCGM, CUDA, NCCL, and related diagnostics.
- Automate cluster provisioning, upgrades, workload deployment, and operational recovery using infrastructure-as-code and GitOps practices.
What you'll need
- Bachelor's Degree and 8 years of software engineering experience.
- 4 years in software, cloud, platform, HPC, or systems engineering, including 2 years supporting distributed AI/ML workloads.
- Hands-on experience building GPU clusters from the ground up and operating them at production scale.
- Kubernetes expertise, including operators, CRDs, Helm, networking, storage, scheduling, and cluster lifecycle management.
- Production experience with Ray and at least two of the following: NVIDIA NeMo RL, vLLM, SGLang, or NVIDIA Dynamo.
- Knowledge of NVIDIA GPUs, CUDA, NCCL, GPU Operator, DCGM, MIG, RDMA, and multi-node collective communication.
- Proficiency in Python and Linux, along with experience in containers, CI/CD, GitOps, infrastructure-as-code, and platform observability.
Nice to have
- Experience with Google TPUs and TPU-oriented frameworks or distributed workloads.
- Experience with large language model training, fine-tuning, RLHF, or agentic reinforcement learning.
- Knowledge of TensorRT-LLM, Triton Inference Server, DeepSpeed, Megatron-LM, or similar performance-oriented frameworks.
- Experience operating secure, multi-tenant AI platforms in enterprise or regulated environments.
Skills: machine learning models, AI algorithms, Python, cloud platforms, production deployment
More semiconductor jobs: KLA Corporation jobs · Software engineer jobs
Tracked in our index since 1 Apr 2026. This is our observation date, not the employer posting date. Specialty: Software Engineering Semiconductor (Systems Software). 333 active roles in this specialty at 57 employers in our index, as of 22 Nov 2026.
How we counted: postings read from employer career sites. How we count