Software Development Engineer — GPU Fleet Management & AI Infrastructure - CA, San Jose, United States

Originally posted 10 September 2026 by the employer.

This role focuses on building Fleet Manager, a secure control plane for operating large-scale AMD GPU infrastructure.

About the role

This role involves building Fleet Manager, a secure control plane for operating large-scale AMD GPU infrastructure, providing scheduling, workload orchestration, hardware health management, interactive development environments, and model-serving capabilities across GPU clusters.

What you'll do

• Design and develop Fleet Manager’s distributed control-plane services, APIs, schedulers, inference gateway, and command-line tools.
• Build reliable orchestration for GPU training, inference, custom jobs, and interactive development workloads.
• Develop scalable scheduling and admission-control capabilities, including priority, fairness, topology-aware placement, quotas, backfilling, and multi-node workload coordination.
• Implement durable reconciliation, lifecycle management, retries, idempotency, and recovery across PostgreSQL and external execution systems.
• Integrate Fleet Manager with Kubernetes and technologies such as Kueue, JobSet, container runtimes, storage systems, and observability platforms.
• Develop GPU health, diagnostics, quarantine, and controlled-remediation capabilities using ROCm and AMD hardware telemetry.

What you'll need

• Strong systems-software development experience.
• Experience designing and operating distributed systems, control planes, schedulers, or cloud infrastructure.
• Strong understanding of concurrency, asynchronous programming, state machines, and failure recovery.
• Experience building reliable services using REST, streaming, WebSocket, or gRPC APIs.
• Experience with Kubernetes internals, controllers, operators, scheduling, resource management, or custom resources.
• Ability to write high-quality, maintainable code with careful attention to correctness, testing, and operational behavior.

Nice to have

• Production Rust experience.
• Familiarity with workload scheduling technologies such as Kueue, JobSet, Slurm, or other batch and cluster schedulers.
• Experience with PostgreSQL-backed services, schema evolution, transactions, leader election, and optimistic concurrency.
• Understanding of container security, multi-tenant isolation, authentication, authorization, and secrets management.
• Experience with production observability, including metrics, structured logging, tracing, alerting, and incident diagnosis.
• Experience with source control, continuous integration, automated testing, profiling, and debugging tools.
• Experience in one or more of: AMD GPU architecture, ROCm, HIP, amd-smi, RCCL, GPU device plugins, distributed AI training, multi-node collective communication, large-model inference, GPU topology, capacity management, performance analysis, hardware diagnostics, Kubernetes networking, storage, admission control, workload isolation, OpenAI-compatible inference APIs, request routing, streaming, rate limiting, high-performance shared storage, large model or checkpoint management, bare-metal, virtualized, cloud GPU infrastructure, production security, threat modeling for multi-tenant compute platforms.

Skills: Fleet Manager, GPU infrastructure, Kubernetes, GPU scheduling, inference infrastructure, distributed control-plane