Director/Sr. Manager, AI Inference Model Scaling - Sunnyvale, CA, United States
Originally posted 23 July 2026 by the employer.
About the role
This role involves leading the Inference Model Scaling organization, focusing on ML model compilation and optimization, and developing high-performance kernels. You will define the technical vision, organizational strategy, and execution roadmap for a globally distributed engineering team.
What you'll do
- Define the technical roadmap and strategy for the team.
- Lead design reviews and establish engineering standards.
- Drive support for emerging LLM architectures and inference workloads.
- Hire, mentor, and grow a high-performing engineering team.
- Partner with Cloud Platform, ML, and Hardware teams in planning and delivering for end-to-end service enablement in Cloud and On-Premise settings.
- Own planning, prioritization, and execution across multiple concurrent initiatives.
What you'll need
- BS, MS, or PhD in Computer Science, Computer Engineering or related field.
- 12+ years building compiler, ML systems, or infrastructure software.
- 5+ years leading engineering teams.
- Deep experience with modern compiler infrastructure (LLVM, MLIR, XLA, TVM, Torch FX, or similar).
- Strong understanding of graph compilation and optimization.
- Experience with Python and C++.
- Experience delivering production-quality software.
Nice to have
- Experience building compiler frontends for AI accelerators.
- Experience supporting PyTorch, JAX, TensorFlow, or ONNX.
- Experience with LLM inference or training systems.
- Familiarity with distributed compilation.
- Experience working with hardware architects.
- Experience leading teams through rapid growth.
Skills: ML model compilation, graph optimization, compiler infrastructure, AI accelerators, LLM inference
This role has been open 84 days — well above the 52-day median for Infrastructure/Platform Software roles.
Infrastructure/Platform Software · Infrastructure Platform
|
Open roles in category
968
|
Median days open
52 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | Cerebras | All employers we track in this specialty (968 roles · 86 employers) |
|---|---|---|
| Open roles in this specialty | 36 | 968 |
| Open roles in the wider Software, Firmware & Systems family | 73 | 6118 · 145 employers |
| Median days open | 84 d | 52 d (+32 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: ML model compilation, graph optimization, compiler infrastructure, AI accelerators, LLM inference
Who's hiring in this category
- NVIDIA · 278 open roles · median 47 d
- Qualcomm · 62 open roles · median 54 d
- AMD · 54 open roles · median 20 d
- Graphcore · 44 open roles · median 94 d
- Intel Corporation · 43 open roles · median 13 d
- Cerebras (this employer) · 36 open roles · median 84 d
How we counted: 968 open Infrastructure/Platform Software (Infrastructure Platform) roles from 86 employers tracked in the SemiconductorJobs index, counted 19 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.