Advanced AI Workloads - Performance and Scalability Engineer - CA, San Jose, United States
Originally posted 11 March 2026 by the employer.
Drive AI inference performance on AMD GPUs across multi-node distributed serving frameworks and custom GPU kernels.
About the role
AMD is seeking an engineer to drive AI inference performance across the software and systems stack on AMD GPUs. You will lead a technical team to profile, diagnose, and optimize models across customer-relevant serving configurations, targeting kernel-level and distributed systems bottlenecks.
What you'll do
- Drive performance optimization end-to-end across the stack on models and customer-relevant serving configurations through kernel and systems-level optimizations. - Profile, diagnose, and resolve cross-stack performance bottlenecks, spanning GPU kernels, operator dispatch, framework-level scheduling, and multi-node communication. - Diagnose kernel-level performance issues using profiling tools to identify occupancy limitations, L2 cache thrashing, register pressure, and memory coalescing issues. - Integrate and optimize custom kernels using Triton, Gluon, CK, PyDSL, ASM, or AITER within serving frameworks, analyzing dispatch paths, shape extraction, and backend selection. - Optimize multi-node distributed inference, addressing communication-compute overlap, parallelism strategies, and scale-out performance. - Upstream optimizations into open-source frameworks including vLLM, SGLang, and PyTorch while leading customer-facing technical engagements.
What you'll need
- Bachelor's, Master's, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent. - Software development background in GPU computing, AI systems, or high-performance computing with proficiency in Python and C++. - Deep hands-on experience with AI serving frameworks such as vLLM, SGLang, or TensorRT-LLM and their runtime internals. - End-to-end workload profiling experience tracing execution from user request to GPU kernel execution. - Understanding of GPU kernel performance characteristics, including occupancy, register and LDS pressure, memory coalescing, cache utilization, wavefront scheduling, and instruction-level bottlenecks. - Understanding of distributed systems and topologies, including multi-GPU and multi-node scale-up and scale-out, RCCL/NCCL, RDMA, and communication-compute overlap.
Nice to have
- Experience with custom kernel development or integration using HIP, CUDA, Triton, or CK. - Understanding of model architectures (transformers, MoE, diffusion) and inference paradigms such as speculative decoding, prefill-decode disaggregation, and continuous batching. - System and rack-level design awareness covering performance tradeoffs across deployment stacks. - Strong Linux systems knowledge and experience with AI-assisted development workflows.
Skills: AI inference performance, AMD GPUs, profiling, optimizing leading models, distributed serving deployment, GPU kernel performance
This role has been open 179 days — well above the 88-day median for AI/ML Hardware Engineering roles.
AI/ML Hardware Engineering · AI ML Hardware
|
Open roles in category
504
|
Median days open
88 d
|
Median salary
$236k
|
See the full market breakdown ▾Category comparison, skills in demand, and who else is hiring
| Metric | AMD | All employers we track in this specialty (504 roles · 77 employers) |
|---|---|---|
| Open roles in this specialty | 32 | 504 |
| Open roles in the wider Software, Firmware & Systems family | 391 | 6302 · 146 employers |
| Median days open | 109 d | 88 d (+21 d vs this employer) |
| Median salary (USD postings) | — | $236k |
Skills observed across this category: AI inference performance, AMD GPUs, profiling, optimizing leading models, distributed serving deployment, GPU kernel performance
Who's hiring in this category
- Qualcomm · 104 open roles · median 117 d
- NVIDIA · 88 open roles · median 59 d
- AMD (this employer) · 33 open roles · median 108 d
- Micron Technology · 26 open roles · median 60 d
- Mobileye · 20 open roles · median 87 d
- Analog Devices · 18 open roles · median 44 d
How we counted: 504 open AI/ML Hardware Engineering (AI ML Hardware) roles from 77 employers tracked in the SemiconductorJobs index, counted 26 Oct 2026. Specialty figures count only roles carrying this exact specialty label, so an employer's related work in neighbouring specialties is not included there — it is counted in the wider Software, Firmware & Systems family row. Figures refresh nightly.