Principal GenAI Inference Optimization Engineer - CA, San Jose, United States
Originally posted 27 March 2026 by the employer.
Optimize generative AI inference workloads on AMD GPU platforms across single-node and distributed environments.
About the role
This role focuses on improving performance, efficiency, and scalability of generative AI inference workloads on AMD GPU platforms. You will contribute to optimizing latency, throughput, and cost efficiency for real-world deployment of large-scale models across the software-hardware stack.
What you'll do
- Optimize performance of GenAI inference workloads on AMD GPU platforms across single-node and distributed environments. - Improve latency, throughput, and cost efficiency for LLM and multimodal model serving in production. - Analyze and resolve bottlenecks across compute, memory, and communication, including kernel efficiency, KV-cache usage, memory bandwidth, and scheduling. - Contribute to cross-stack optimizations spanning kernels, runtimes, communication libraries, and inference/serving frameworks such as vLLM, SGLang, or Triton. - Implement and evaluate inference optimization techniques including batching strategies, quantization, prefix caching, and speculative decoding. - Develop and use profiling, benchmarking, and performance analysis tools for inference workloads.
What you'll need
- Expertise in GenAI inference optimization, GPU performance, and large-scale serving systems. - Understanding of GPU architecture, memory systems, and communication patterns. - Experience working across layers from kernels and runtimes to frameworks and serving systems. - Equivalent industry experience or relevant degree.
Nice to have
- Understanding of GPU architecture and performance fundamentals covering compute, memory hierarchy, and interconnects such as PCIe, Infinity Fabric, or RDMA. - Hands-on experience with inference and serving frameworks such as vLLM, SGLang, Triton, or TensorRT-LLM. - Experience working on LLM or multimodal inference workloads and serving architectures. - Experience with ML frameworks such as PyTorch, JAX, or TensorFlow for inference. - Proficiency in Python and at least one systems language such as C++, CUDA, or HIP. - B.S., M.S. or Ph.D. in Computer Science, Computer Engineering, or a related field.
Skills: GenAI inference optimization, AMD GPU platforms, LLM, multimodal model serving, GPU architecture, kernel efficiency, KV-cache usage, inference optimization techniques
This role has been open 173 days — well above the 88-day median for AI/ML Hardware Engineering roles.
AI/ML Hardware Engineering · AI ML Hardware
|
Open roles in category
504
|
Median days open
88 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | AMD | All employers we track in this specialty (504 roles · 77 employers) |
|---|---|---|
| Open roles in this specialty | 33 | 504 |
| Open roles in the wider Software, Firmware & Systems family | 389 | 6250 · 144 employers |
| Median days open | 108 d | 88 d (+20 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: GenAI inference optimization, AMD GPU platforms, LLM, multimodal model serving, GPU architecture, kernel efficiency
Who's hiring in this category
- Qualcomm · 104 open roles · median 117 d
- NVIDIA · 88 open roles · median 59 d
- AMD (this employer) · 33 open roles · median 108 d
- Micron Technology · 26 open roles · median 60 d
- Mobileye · 20 open roles · median 87 d
- Analog Devices · 18 open roles · median 44 d
How we counted: 504 open AI/ML Hardware Engineering (AI ML Hardware) roles from 77 employers tracked in the SemiconductorJobs index, counted 26 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.