$224,000 - $431,250 USD yearly
Originally posted 6 November 2026 by the employer.
About the role
NVIDIA’s DGX Cloud Recovery and Resilience team is seeking a Senior System Software Engineer to own AI infrastructure services, tools, and open-source projects supporting resilient ML training workloads from design through production.
What you'll do
- Drive the design, build, launch, and operation of AI infrastructure services, tools, and open-source projects.
- Build resilient infrastructure on Kubernetes and Slurm for training workloads across supercomputers at major cloud providers and on-premises clusters.
- Build APIs interfacing with resiliency systems, including telemetry, break/fix automation, checkpointing, and execution systems.
- Develop a job submission layer that masks infrastructure failures and automatically restarts long-running training jobs.
- Build agentic tools to diagnose failures, identify bottlenecks, and recommend efficiency fixes for training workloads.
- Establish service operation standards including monitoring, alerts, CI/CD pipelines, integration test gates, and on-call plans.
What you'll need
- BS, MS, or PhD in Computer Science or a related field (or equivalent experience), with 12+ years of experience.
- Track record of delivering large systems from design through production.
- Experience with Kubernetes and/or Slurm at scale.
- Hands-on experience with PyTorch, TensorFlow, JAX, or Ray.
- Technical understanding of NVIDIA GPUs and networking, including failure modes.
- Experience running production services, including CI/CD, monitoring, alerting, and on-call responsibilities.
Skills: Kubernetes, Slurm, cloud infrastructure, GPU clusters, distributed training
More semiconductor jobs: NVIDIA jobs · California jobs
This role has been open 0 days — well below the 53-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · Infrastructure Platform
|
Open roles in category
1024
|
Median days open
53 d
|
Category median salary (all employers)
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | NVIDIA | All employers we track in this specialty (1024 roles · 93 employers) |
|---|---|---|
| Open roles in this specialty | 335 | 1024 |
| Open roles in the wider Software, Firmware & Systems family | 1223 | 6531 · 156 employers |
| Median days open | 57 d | 53 d (+4 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: Kubernetes, Slurm, cloud infrastructure, GPU clusters, distributed training
Who's hiring in this category
- NVIDIA (this employer) · 330 open roles · median 58 d
- Qualcomm · 58 open roles · median 60 d
- AMD · 55 open roles · median 35 d
- Graphcore · 48 open roles · median 100 d
- Intel Corporation · 38 open roles · median 23 d
- Arm Holdings · 37 open roles · median 10 d
How we counted: 1024 open Infrastructure/Platform Software (Infrastructure Platform) roles from 93 employers tracked in the SemiconductorJobs index, counted 7 Nov 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.