Senior Product Manager –AI Inference Software - CA, Santa Clara, United States
$179,970 - $308,520 USD yearly
Originally posted 1 July 2026 by the employer.
Drive framework-layer inference product strategy for ROCm across AMD Instinct hardware.
About the role
This role owns the framework-layer inference product strategy for ROCm, translating customer, ecosystem, and engineering signals into roadmap decisions for production AI inference on AMD Instinct hardware.
What you'll do
- Own the product strategy and roadmap for ROCm inference frameworks software capabilities across single-GPU through rack-scale deployments. - Partner with engineering teams across inference engines, serving and orchestration, memory systems, GPU libraries, runtimes, kernels, and communications to prioritize technical requirements. - Identify inference performance bottlenecks and evaluate where framework-level software investment drives performance and competitive differentiation. - Act as an AMD representative in the open-source inference community, interfacing with maintainers, developers, and researchers. - Engage directly with AI infrastructure customers and cloud providers to translate deployment requirements and serving constraints into product decisions. - Provide software-informed input to future GPU architecture and platform decisions relevant to inference performance at scale.
What you'll need
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related technical field. - Deep technical expertise across AI inference systems, inference engines, serving, orchestration, memory management, and GPU software interactions. - Experience driving complex technical initiatives across multiple engineering teams or organizations. - Background in AI inference infrastructure, GPU computing, distributed systems, cloud infrastructure, high-performance computing, developer platforms, research, or technical product leadership. - Ability to communicate technical tradeoffs to engineering leaders, customer stakeholders, open-source contributors, and executive audiences.
Nice to have
- Deep understanding of production AI inference engines such as vLLM, SGLang, and AMD ATOM, and serving systems such as llm-d, along with memory management and distributed inference. - Familiarity with AMD Instinct hardware and the ROCm software ecosystem. - Advanced degree in Computer Science, Electrical Engineering, or a related technical field. - Product management experience in deeply technical products. - History of open-source contributions, technical writing, conference talks, or sustained community engagement.
Skills: ROCm, GPU software stack, AI inference, PyTorch, JAX, LLM inference
This role has been open 112 days — well above the 47-day median for Product Management roles.
Product Management · AI ML Hardware
|
Open roles in category
71
|
Median days open
47 d
|
Median salary
$248k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | AMD | All employers we track in this specialty (71 roles · 22 employers) |
|---|---|---|
| Open roles in this specialty | 8 | 71 |
| Median days open | 55 d | 47 d (+8 d vs this employer) |
| Median salary (USD postings) | — | $248k |
Skills observed across this category: ROCm, GPU software stack, AI inference, PyTorch, JAX, LLM inference
Who's hiring in this category
- NVIDIA · 23 open roles · median 35 d
- Qualcomm · 10 open roles · median 69 d
- AMD (this employer) · 8 open roles · median 54 d
- NXP Semiconductors · 4 open roles · median 20 d
- Analog Devices · 3 open roles · median 39 d
- SambaNova · 3 open roles · median 41 d
How we counted: 71 open Product Management (AI ML Hardware) roles from 22 employers tracked in the SemiconductorJobs index, counted 21 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there. Figures refresh nightly.